

DAL-BERT: Another pre-trained language model for Persian
---

DAL-BERT is a transformer-based model trained on more than 80 gigabytes of Persian text including both formal and informal (conversational) contexts. The architecture of this model follows the original BERT [[Devlin et al.](https://arxiv.org/abs/1810.04805)].

How to use the Model
---
```python
from transformers import BertForMaskedLM, BertTokenizer, pipeline
model = BertForMaskedLM.from_pretrained('sharif-dal/dal-bert')
tokenizer = BertTokenizer.from_pretrained('sharif-dal/dal-bert')
fill_sentence = pipeline('fill-mask', model=model, tokenizer=tokenizer)
fill_sentence('اینجا جمله مورد نظر خود را بنویسید و کلمه موردنظر را [MASK] کنید')
```

The Training Data
---
The abovementioned model was trained on a bunch of newspapers, news agencies' websites, technology-related sources, people's comments, magazines, literary criticism, and some blogs.

Evaluation
---

| Training Loss | Epoch | Step  |
|:-------------:|:-----:|:-----:|
| 2.1855        | 13   | 7649486 |

Contributors
---
- Arman Malekzadeh [[Github](https://github.com/arm-on)]
- Amirhossein Ramazani, Master's Student in AI @ Sharif University of Technology [[Linkedin](https://www.linkedin.com/in/amirhossein-ramazani/)] [[Github](https://github.com/amirhossein1376)]

{"language": "fa", "license": "apache-2.0", "tags": ["bert-fa", "bert-persian"], "widget": [{"text": "\u0627\u0632 \u0647\u0631 \u062f\u0633\u062a\u06cc \u0628\u06af\u06cc\u0631\u06cc \u0627\u0632 \u0647\u0645\u0648\u0646 [MASK] \u0645\u06cc\u062f\u06cc"}, {"text": "\u0627\u06cc\u0646 \u0622\u062e\u0631\u06cc\u0646 \u0628\u0627\u0631\u0647 \u0628\u0647\u062a [MASK] \u0645\u06cc\u06af\u0645"}, {"text": "\u0686\u0631\u0627 \u0622\u0646 \u062c\u0648\u0627\u0646 \u0628\u06cc\u0686\u0627\u0631\u0647 \u0631\u0627 \u0628\u0647 \u0633\u062e\u0631\u0647 [MASK]"}, {"text": "\u0622\u062e\u0647 \u0645\u062d\u0633\u0646 [MASK] \u0647\u0645 \u0634\u062f \u062e\u0648\u0627\u0646\u0646\u062f\u0647\u061f"}, {"text": "\u067e\u0633\u0631 \u0639\u062c\u0628 [MASK] \u0632\u062f"}], "model-index": [{"name": "dal-bert", "results": []}]}
transformers pytorch bert fill-mask bert-fa bert-persian fa arxiv:1810.04805 license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #bert-fa #bert-persian #fa #arxiv-1810.04805 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

DAL-BERT: Another pre-trained language model for Persian
--------------------------------------------------------


DAL-BERT is a transformer-based model trained on more than 80 gigabytes of Persian text including both formal and informal (conversational) contexts. The architecture of this model follows the original BERT [Devlin et al.].


How to use the Model
--------------------


The Training Data
-----------------


The abovementioned model was trained on a bunch of newspapers, news agencies' websites, technology-related sources, people's comments, magazines, literary criticism, and some blogs.


Evaluation
----------



Contributors
------------


* Arman Malekzadeh [Github]
* Amirhossein Ramazani, Master's Student in AI @ Sharif University of Technology [Linkedin] [Github]



TAGS
#transformers #pytorch #bert #fill-mask #bert-fa #bert-persian #fa #arxiv-1810.04805 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #bert #fill-mask #bert-fa #bert-persian #fa #arxiv-1810.04805 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
