
# bert-base-bg-cs-pl-ru-cased

SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.

08.11.2021: upload model with MLM and NSP heads

\[1\]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. \(2019\). [Tuning Multilingual Transformers for Language-Specific Named Entity Recognition](https://www.aclweb.org/anthology/W19-3712/). ACL anthology W19-3712.

{"language": ["bg", "cs", "pl", "ru"]}
transformers pytorch jax bert feature-extraction bg cs pl ru endpoints_compatible region:us
TAGS
#transformers #pytorch #jax #bert #feature-extraction #bg #cs #pl #ru #endpoints_compatible #region-us 


# bert-base-bg-cs-pl-ru-cased

SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.

08.11.2021: upload model with MLM and NSP heads

\[1\]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. \(2019\). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.

# bert-base-bg-cs-pl-ru-cased

SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.

08.11.2021: upload model with MLM and NSP heads

\[1\]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. \(2019\). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.
TAGS
#transformers #pytorch #jax #bert #feature-extraction #bg #cs #pl #ru #endpoints_compatible #region-us 
 # bert-base-bg-cs-pl-ru-cased

SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.

08.11.2021: upload model with MLM and NSP heads

\[1\]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. \(2019\). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.
passage: TAGS
#transformers #pytorch #jax #bert #feature-extraction #bg #cs #pl #ru #endpoints_compatible #region-us 
# bert-base-bg-cs-pl-ru-cased

SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.

08.11.2021: upload model with MLM and NSP heads

\[1\]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. \(2019\). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.