
# gpt2-wechsel-uyghur

Model trained with WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.

See the code here: https://github.com/CPJKU/wechsel

And the paper here: https://arxiv.org/abs/2112.06598

## Performance

| Model | PPL |
|---|---|
| `gpt2-wechsel-sundanese` | **111.72** |
| `gpt2` (retrained from scratch)  | 149.46 |

| Model | PPL |
|---|---|
| `gpt2-wechsel-scottish-gaelic` | **16.43** |
| `gpt2` (retrained from scratch)  | 19.53 |

| Model | PPL |
|---|---|
| `gpt2-wechsel-uyghur` | **34.33** |
| `gpt2` (retrained from scratch)  | 42.82 |

| Model | PPL |
|---|---|
| `gpt2-wechsel-malagasy` | **14.01** |
| `gpt2` (retrained from scratch)  | 15.93 |

See our paper for details.

## Citation

Please cite WECHSEL as

```
@misc{minixhofer2021wechsel,
      title={WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models}, 
      author={Benjamin Minixhofer and Fabian Paischer and Navid Rekabsaz},
      year={2021},
      eprint={2112.06598},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

{"language": "ug", "license": "mit"}
transformers pytorch gpt2 text-generation ug arxiv:2112.06598 license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-generation #ug #arxiv-2112.06598 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

gpt2-wechsel-uyghur
===================


Model trained with WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.


See the code here: URL


And the paper here: URL


Performance
-----------






See our paper for details.


Please cite WECHSEL as



TAGS
#transformers #pytorch #gpt2 #text-generation #ug #arxiv-2112.06598 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

passage: TAGS
#transformers #pytorch #gpt2 #text-generation #ug #arxiv-2112.06598 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
