
# GPT-2 recycled for Italian (small, adapted lexical embeddings)
[Wietse de Vries](https://www.semanticscholar.org/author/Wietse-de-Vries/144611157) •
[Malvina Nissim](https://www.semanticscholar.org/author/M.-Nissim/2742475)

## Model description

This model is based on the small OpenAI GPT-2 ([`gpt2`](https://huggingface.co/gpt2)) model.

The Transformer layer weights in this model are identical to the original English, model but the lexical layer has been retrained for an Italian vocabulary.

For details, check out our paper on [arXiv](https://arxiv.org/abs/2012.05628) and the code on [Github](https://github.com/wietsedv/gpt2-recycle).


## Related models

### Dutch
 - [`gpt2-small-dutch-embeddings`](https://huggingface.co/GroNLP/gpt2-small-dutch-embeddings): Small model size with only retrained lexical embeddings.
 - [`gpt2-small-dutch`](https://huggingface.co/GroNLP/gpt2-small-dutch):  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (**Recommended**)
 - [`gpt2-medium-dutch-embeddings`](https://huggingface.co/GroNLP/gpt2-medium-dutch-embeddings): Medium model size with only retrained lexical embeddings.

### Italian
 - [`gpt2-small-italian-embeddings`](https://huggingface.co/GroNLP/gpt2-small-italian-embeddings): Small model size with only retrained lexical embeddings.
 - [`gpt2-small-italian`](https://huggingface.co/GroNLP/gpt2-small-italian):  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (**Recommended**)
 - [`gpt2-medium-italian-embeddings`](https://huggingface.co/GroNLP/gpt2-medium-italian-embeddings): Medium model size with only retrained lexical embeddings.


## How to use

```python
from transformers import pipeline

pipe = pipeline("text-generation", model="GroNLP/gpt2-small-italian-embeddings")
```

```python
from transformers import AutoTokenizer, AutoModel, TFAutoModel

tokenizer = AutoTokenizer.from_pretrained("GroNLP/gpt2-small-italian-embeddings")
model = AutoModel.from_pretrained("GroNLP/gpt2-small-italian-embeddings")  # PyTorch
model = TFAutoModel.from_pretrained("GroNLP/gpt2-small-italian-embeddings")  # Tensorflow
```

## BibTeX entry

```bibtex
@misc{devries2020good,
      title={As good as new. How to successfully recycle English GPT-2 to make models for other languages}, 
      author={Wietse de Vries and Malvina Nissim},
      year={2020},
      eprint={2012.05628},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

{"language": "it", "tags": ["adaption", "recycled", "gpt2-small"], "pipeline_tag": "text-generation"}
transformers pytorch tf jax safetensors gpt2 text-generation adaption recycled gpt2-small it arxiv:2012.05628 autotrain_compatible endpoints_compatible has_space text-generation-inference region:us
TAGS
#transformers #pytorch #tf #jax #safetensors #gpt2 #text-generation #adaption #recycled #gpt2-small #it #arxiv-2012.05628 #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 


# GPT-2 recycled for Italian (small, adapted lexical embeddings)
Wietse de Vries •
Malvina Nissim

## Model description

This model is based on the small OpenAI GPT-2 ('gpt2') model.

The Transformer layer weights in this model are identical to the original English, model but the lexical layer has been retrained for an Italian vocabulary.

For details, check out our paper on arXiv and the code on Github.


## Related models

### Dutch
 - 'gpt2-small-dutch-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-dutch':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-dutch-embeddings': Medium model size with only retrained lexical embeddings.

### Italian
 - 'gpt2-small-italian-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-italian':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-italian-embeddings': Medium model size with only retrained lexical embeddings.


## How to use





## BibTeX entry



# GPT-2 recycled for Italian (small, adapted lexical embeddings)
Wietse de Vries •
Malvina Nissim ## Model description

This model is based on the small OpenAI GPT-2 ('gpt2') model.

The Transformer layer weights in this model are identical to the original English, model but the lexical layer has been retrained for an Italian vocabulary.

For details, check out our paper on arXiv and the code on Github. ## Related models ### Dutch
 - 'gpt2-small-dutch-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-dutch':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-dutch-embeddings': Medium model size with only retrained lexical embeddings. ### Italian
 - 'gpt2-small-italian-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-italian':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-italian-embeddings': Medium model size with only retrained lexical embeddings. ## How to use ## BibTeX entry
TAGS
#transformers #pytorch #tf #jax #safetensors #gpt2 #text-generation #adaption #recycled #gpt2-small #it #arxiv-2012.05628 #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
 # GPT-2 recycled for Italian (small, adapted lexical embeddings)
Wietse de Vries •
Malvina Nissim ## Model description

This model is based on the small OpenAI GPT-2 ('gpt2') model.

The Transformer layer weights in this model are identical to the original English, model but the lexical layer has been retrained for an Italian vocabulary.

For details, check out our paper on arXiv and the code on Github. ## Related models ### Dutch
 - 'gpt2-small-dutch-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-dutch':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-dutch-embeddings': Medium model size with only retrained lexical embeddings. ### Italian
 - 'gpt2-small-italian-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-italian':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-italian-embeddings': Medium model size with only retrained lexical embeddings. ## How to use ## BibTeX entry
TAGS
#transformers #pytorch #tf #jax #safetensors #gpt2 #text-generation #adaption #recycled #gpt2-small #it #arxiv-2012.05628 #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
# GPT-2 recycled for Italian (small, adapted lexical embeddings)
Wietse de Vries •
Malvina Nissim## Model description

This model is based on the small OpenAI GPT-2 ('gpt2') model.

The Transformer layer weights in this model are identical to the original English, model but the lexical layer has been retrained for an Italian vocabulary.

For details, check out our paper on arXiv and the code on Github.## Related models### Dutch
 - 'gpt2-small-dutch-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-dutch':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-dutch-embeddings': Medium model size with only retrained lexical embeddings.### Italian
 - 'gpt2-small-italian-embeddings': Small model size with only retrained lexical embeddings.
 - 'gpt2-small-italian':  Small model size with retrained lexical embeddings and additional fine-tuning of the full model. (Recommended)
 - 'gpt2-medium-italian-embeddings': Medium model size with only retrained lexical embeddings.## How to use## BibTeX entry