This repository is created with the aim to provide better models for NLI in persian, with the transparent codes for training I hope you guys find it inspiring and build better model in the future. for more details about the task and methods used for training check the [medium post](https://haddadhesam.medium.com/) and notebooks.


# Dataset

The dataset used for training is Wiki D/Similar dataset (wiki-d-similar.zip), obtained from [Sentence Transformers](https://github.com/m3hrdadfi/sentence-transformers) repository.

# Model

The proposed model is published at HuggingFace Hub with the name of ``demoversion/bert-fa-base-uncased-haddad-wikinli``. You can download and use the model from [HuggingFace Website](https://huggingface.co/demoversion/bert-fa-base-uncased-haddad-wikinli) or directly in transformers library like this:

    from transformers import pipeline
    model = pipeline("zero-shot-classification", model="demoversion/bert-fa-base-uncased-haddad-wikinli")
    labels = ["ورزشی",
			    "سیاسی",
			    "علمی",
			    "فرهنگی"]
		template_str = "این یک متن {} است."
		str_sentence = "مرحله مقدماتی جام جهانی حاشیه‌های زیادی داشت."
		model(str_sentence, labels, hypothesis_template=template_str)
The result of this code snippet is:

    Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
    {'labels': ['فرهنگی', 'علمی', 'سیاسی', 'ورزشی'],
     'scores': [0.25921085476875305,
       0.25713297724723816,
       0.24884170293807983,
       0.23481446504592896],
       'sequence': 'مرحله مقدماتی جام جهانی حاشیه\u200cهای زیادی داشت.'}
Yep, the right label (highest score) without training.
# Results

The result comparing to the original model published for this dataset is available in the table bellow.


|Model|dev_accuracy| dev_f1|test_accuracy|test_f1|
|--|--|--|--|--|
|[m3hrdadfi/bert-fa-base-uncased-wikinli](https://huggingface.co/m3hrdadfi/bert-fa-base-uncased-wikinli)|77.88|77.57|76.64|75.99|
|[demoversion/bert-fa-base-uncased-haddad-wikinli](https://huggingface.co/demoversion/bert-fa-base-uncased-haddad-wikinli)|**78.62**|**79.74**|**77.04**|**78.56**|

# Notebooks
Notebooks used for training and evaluation are available below.

[Training ![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DemoVersion/persian-nli-trainer/blob/main/notebooks/training.ipynb)

[Evaluation ![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DemoVersion/persian-nli-trainer/blob/main/notebooks/evaluation.ipynb)


{"language": "fa", "license": "apache-2.0"}
transformers pytorch safetensors bert text-classification fa license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #safetensors #bert #text-classification #fa #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

This repository is created with the aim to provide better models for NLI in persian, with the transparent codes for training I hope you guys find it inspiring and build better model in the future. for more details about the task and methods used for training check the medium post and notebooks.


Dataset
=======


The dataset used for training is Wiki D/Similar dataset (URL), obtained from Sentence Transformers repository.


Model
=====


The proposed model is published at HuggingFace Hub with the name of ''demoversion/bert-fa-base-uncased-haddad-wikinli''. You can download and use the model from HuggingFace Website or directly in transformers library like this:



```
from transformers import pipeline
model = pipeline("zero-shot-classification", model="demoversion/bert-fa-base-uncased-haddad-wikinli")
labels = ["ورزشی",
		    "سیاسی",
		    "علمی",
		    "فرهنگی"]
	template_str = "این یک متن {} است."
	str_sentence = "مرحله مقدماتی جام جهانی حاشیه‌های زیادی داشت."
	model(str_sentence, labels, hypothesis_template=template_str)

```

The result of this code snippet is:



```
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
{'labels': ['فرهنگی', 'علمی', 'سیاسی', 'ورزشی'],
 'scores': [0.25921085476875305,
   0.25713297724723816,
   0.24884170293807983,
   0.23481446504592896],
   'sequence': 'مرحله مقدماتی جام جهانی حاشیه\u200cهای زیادی داشت.'}

```

Yep, the right label (highest score) without training.


Results
=======


The result comparing to the original model published for this dataset is available in the table bellow.



Notebooks
=========


Notebooks used for training and evaluation are available below.


Training ![Open In Colab](URL


Evaluation ![Open In Colab](URL



TAGS
#transformers #pytorch #safetensors #bert #text-classification #fa #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #safetensors #bert #text-classification #fa #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
