
# BERT BASE (cased)

Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
[this paper](https://arxiv.org/abs/1810.04805) and first released in
[this repository](https://github.com/google-research/bert). This model is cased: it does make a difference
between bulgarian and Bulgarian. The training data is Bulgarian text from [OSCAR](https://oscar-corpus.com/post/oscar-2019/), [Chitanka](https://chitanka.info/) and [Wikipedia](https://bg.wikipedia.org/).

The model was compressed via [progressive module replacing](https://arxiv.org/abs/2002.02925).

### How to use

Here is how to use this model in PyTorch:

```python
>>> from transformers import pipeline
>>> 
>>> model = pipeline(
>>>     'fill-mask',
>>>     model='rmihaylov/bert-base-theseus-bg',
>>>     tokenizer='rmihaylov/bert-base-theseus-bg',
>>>     device=0,
>>>     revision=None)
>>> output = model("София е [MASK] на България.")
>>> print(output)

[{'score': 0.1586454212665558,
  'sequence': 'София е столица на България.',
  'token': 76074,
  'token_str': 'столица'},
 {'score': 0.12992817163467407,
  'sequence': 'София е  столица на България.',
  'token': 2659,
  'token_str': 'столица'},
 {'score': 0.06064048036932945,
  'sequence': 'София е Перлата на България.',
  'token': 102146,
  'token_str': 'Перлата'},
 {'score': 0.034687548875808716,
  'sequence': 'София е представителката на България.',
  'token': 105456,
  'token_str': 'представителката'},
 {'score': 0.03053216263651848,
  'sequence': 'София е присъединяването на България.',
  'token': 18749,
  'token_str': 'присъединяването'}]
```

{"language": ["bg"], "license": "mit", "tags": ["torch"], "datasets": ["oscar", "chitanka", "wikipedia"], "inference": false}
transformers pytorch bert fill-mask torch bg dataset:oscar dataset:chitanka dataset:wikipedia arxiv:1810.04805 arxiv:2002.02925 license:mit autotrain_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #torch #bg #dataset-oscar #dataset-chitanka #dataset-wikipedia #arxiv-1810.04805 #arxiv-2002.02925 #license-mit #autotrain_compatible #region-us 


# BERT BASE (cased)

Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
this paper and first released in
this repository. This model is cased: it does make a difference
between bulgarian and Bulgarian. The training data is Bulgarian text from OSCAR, Chitanka and Wikipedia.

The model was compressed via progressive module replacing.

### How to use

Here is how to use this model in PyTorch:



# BERT BASE (cased)

Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
this paper and first released in
this repository. This model is cased: it does make a difference
between bulgarian and Bulgarian. The training data is Bulgarian text from OSCAR, Chitanka and Wikipedia.

The model was compressed via progressive module replacing. ### How to use

Here is how to use this model in PyTorch:
TAGS
#transformers #pytorch #bert #fill-mask #torch #bg #dataset-oscar #dataset-chitanka #dataset-wikipedia #arxiv-1810.04805 #arxiv-2002.02925 #license-mit #autotrain_compatible #region-us 
 # BERT BASE (cased)

Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
this paper and first released in
this repository. This model is cased: it does make a difference
between bulgarian and Bulgarian. The training data is Bulgarian text from OSCAR, Chitanka and Wikipedia.

The model was compressed via progressive module replacing. ### How to use

Here is how to use this model in PyTorch:
passage: TAGS
#transformers #pytorch #bert #fill-mask #torch #bg #dataset-oscar #dataset-chitanka #dataset-wikipedia #arxiv-1810.04805 #arxiv-2002.02925 #license-mit #autotrain_compatible #region-us 
# BERT BASE (cased)

Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
this paper and first released in
this repository. This model is cased: it does make a difference
between bulgarian and Bulgarian. The training data is Bulgarian text from OSCAR, Chitanka and Wikipedia.

The model was compressed via progressive module replacing.### How to use

Here is how to use this model in PyTorch: