Taaghche Feelings Analyzer

This project is Homework #3 of the NLP Course at the CE Department of Sharif University of Technology, lectured by Dr. Ehsaneddin Asgari. The aim is to train some models on taaghche comments.

Data

For this project we used different datasets, including taaghche sentiment analysis, Arman, Persian-NER, and we have crawled the website to collect some more data.

Requirements

• python
• Jupyter Notebook
• Additional libraries specified in requirements.txt

Installation

1. Clone the repository.
   git clone https://github.com/miirzamiir/taaghcheh-feelings-analyzer.git

2. Install the dependencies.
   pip install -r requirements.txt `

Models

For this project we have trained 4 models:

1. First one is a Naïve Bayes model which had been trained on taaghche sentiment analysis. Here is the result of the trained model:
   Accuracy: 0.7341618882320847
   Precision (Macro): 0.7327678622348551
   Recall (Macro): 0.7341135534607575
   F1 Score (Macro): 0.732412323288311
   F1 Score (Micro): 0.7341618882320847
   

2. For the second model, a transformer-based model named pars-bert has been trained. You can see the model here. Here is the result of the trained model:
   F1 Score: 0.799
   Accuracy: 0.800
   Precision: 0.801
   Recall: 0.800
   

3. For the third model we have trained a HMM model, which you can see its results:
   Accuracy: 58.74%
   F1-macro score: 5.37%
   F1-micro score: 58.73%
   Precision-macro: 5.94%
   Precision-micro: 58.73%
   Recall-macro: 5.94%
   Recall-micro: 58.73%
   

4. And finally, for the last model again we have used the pars-bert model for BIO tags. You can see the model here. Here is the result of the model.
   F1 Score: 0.3534761203704847
   Accuracy: 0.8980074083535573
   Precision: 0.48302243550652585
   Recall: 0.3239270904426163
   

Contributors

Here is the list of all contributors of this project:
Amirmohamad Shakuri - Hamed Jahantigh - Zahra Maleki