## LuxGPT-2 based GER

GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of RTL.lu news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an German base model, feature space mapping from LB on Base feature space and gradual layer freezing.
The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 53.4 hours
- for 20 epochs and 7 cycles
- using the fastai library


## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("laurabernardy/LuxGPT2-basedGER")

model = AutoModelForCausalLM.from_pretrained("laurabernardy/LuxGPT2-basedGER")
```
## Limitations and Biases
See the [GPT2 model card](https://huggingface.co/gpt2) for considerations on limitations and bias. See the [GPT2 documentation](https://huggingface.co/transformers/model_doc/gpt2.html) for details on GPT2.

{"language": ["lb"], "license": "mit", "tags": ["luxembourgish", "l\u00ebtzebuergesch", "text generation", "transfer learning"], "model-index": [{"name": "LuxGPT2-basedGER", "results": [{"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "Luxembourgish Test Dataset", "type": "LuxembourgishTestDataset"}, "metrics": [{"type": "accuracy", "value": "0.34"}, {"type": "perplexity", "value": "45.89"}]}]}]}
transformers pytorch gpt2 text-generation luxembourgish lëtzebuergesch text generation transfer learning lb license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-generation #luxembourgish #lëtzebuergesch #text generation #transfer learning #lb #license-mit #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

## LuxGPT-2 based GER

GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of URL news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an German base model, feature space mapping from LB on Base feature space and gradual layer freezing.
The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 53.4 hours
- for 20 epochs and 7 cycles
- using the fastai library


## Usage

## Limitations and Biases
See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.

## LuxGPT-2 based GER

GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of URL news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an German base model, feature space mapping from LB on Base feature space and gradual layer freezing.
The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 53.4 hours
- for 20 epochs and 7 cycles
- using the fastai library ## Usage ## Limitations and Biases
See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.
TAGS
#transformers #pytorch #gpt2 #text-generation #luxembourgish #lëtzebuergesch #text generation #transfer learning #lb #license-mit #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ## LuxGPT-2 based GER

GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of URL news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an German base model, feature space mapping from LB on Base feature space and gradual layer freezing.
The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 53.4 hours
- for 20 epochs and 7 cycles
- using the fastai library ## Usage ## Limitations and Biases
See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.
passage: TAGS
#transformers #pytorch #gpt2 #text-generation #luxembourgish #lëtzebuergesch #text generation #transfer learning #lb #license-mit #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
## LuxGPT-2 based GER

GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of URL news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an German base model, feature space mapping from LB on Base feature space and gradual layer freezing.
The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 53.4 hours
- for 20 epochs and 7 cycles
- using the fastai library## Usage## Limitations and Biases
See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.