
# mgfrantz/distilgpt2-finetuned-reddit-tifu

This model was trained to as practice for fine-tuning a causal language model.
There was no intended use case for this model besides having some fun seeing how different things might be screwed up.

## Data

This model was trained on "short" subset of [`reddit_tifu`](https://huggingface.co/datasets/reddit_tifu) dataset.
The data was split into 90% train and 10% validation using `dataset.train_test_split`, with a seed of 0.

To prepare the data for training, the `"tldr"` and `"documents"` fields were joined by `"\n\n"`.
When multiple items were in the `"tldr"` or `"documents"` fields, only the first item was selected for joining.
These joined documents were tokenized using the `"distilgpt2"` tokenizer.

Finally, tokenized texts were concatenated end-to-end and split into blocks of 128 tokens.

**TODO:** Add a different separation token between documents that can be used to stop generation.

## Training

This model was trained in Colab by fine-tuning [`distilgpt2`](https://huggingface.co/distilgpt2) for 174390 steps (3 epochs).
Default training arguments were used, except for `learning_rate=2e-5` and `weight_decay=0.01`.
At the conclusion of training, a training loss of 3.52 and a validation loss of 3.44 were observed.
{"language": ["en"], "license": "mit", "datasets": ["reddit_tifu (subset: short)"], "thumbnail": "https://styles.redditmedia.com/t5_2to41/styles/communityIcon_qedoavxzocr61.png?width=256&s=9c7c19b81474c3788279b8d6d6823e791d0524fc", "widget": [{"text": "I told my friend"}]}
transformers pytorch tensorboard safetensors gpt2 text-generation en license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #tensorboard #safetensors #gpt2 #text-generation #en #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


# mgfrantz/distilgpt2-finetuned-reddit-tifu

This model was trained to as practice for fine-tuning a causal language model.
There was no intended use case for this model besides having some fun seeing how different things might be screwed up.

## Data

This model was trained on "short" subset of 'reddit_tifu' dataset.
The data was split into 90% train and 10% validation using 'dataset.train_test_split', with a seed of 0.

To prepare the data for training, the '"tldr"' and '"documents"' fields were joined by '"\n\n"'.
When multiple items were in the '"tldr"' or '"documents"' fields, only the first item was selected for joining.
These joined documents were tokenized using the '"distilgpt2"' tokenizer.

Finally, tokenized texts were concatenated end-to-end and split into blocks of 128 tokens.

TODO: Add a different separation token between documents that can be used to stop generation.

## Training

This model was trained in Colab by fine-tuning 'distilgpt2' for 174390 steps (3 epochs).
Default training arguments were used, except for 'learning_rate=2e-5' and 'weight_decay=0.01'.
At the conclusion of training, a training loss of 3.52 and a validation loss of 3.44 were observed.
# mgfrantz/distilgpt2-finetuned-reddit-tifu

This model was trained to as practice for fine-tuning a causal language model.
There was no intended use case for this model besides having some fun seeing how different things might be screwed up. ## Data

This model was trained on "short" subset of 'reddit_tifu' dataset.
The data was split into 90% train and 10% validation using 'dataset.train_test_split', with a seed of 0.

To prepare the data for training, the '"tldr"' and '"documents"' fields were joined by '"\n\n"'.
When multiple items were in the '"tldr"' or '"documents"' fields, only the first item was selected for joining.
These joined documents were tokenized using the '"distilgpt2"' tokenizer.

Finally, tokenized texts were concatenated end-to-end and split into blocks of 128 tokens.

TODO: Add a different separation token between documents that can be used to stop generation. ## Training

This model was trained in Colab by fine-tuning 'distilgpt2' for 174390 steps (3 epochs).
Default training arguments were used, except for 'learning_rate=2e-5' and 'weight_decay=0.01'.
At the conclusion of training, a training loss of 3.52 and a validation loss of 3.44 were observed.
TAGS
#transformers #pytorch #tensorboard #safetensors #gpt2 #text-generation #en #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # mgfrantz/distilgpt2-finetuned-reddit-tifu

This model was trained to as practice for fine-tuning a causal language model.
There was no intended use case for this model besides having some fun seeing how different things might be screwed up. ## Data

This model was trained on "short" subset of 'reddit_tifu' dataset.
The data was split into 90% train and 10% validation using 'dataset.train_test_split', with a seed of 0.

To prepare the data for training, the '"tldr"' and '"documents"' fields were joined by '"\n\n"'.
When multiple items were in the '"tldr"' or '"documents"' fields, only the first item was selected for joining.
These joined documents were tokenized using the '"distilgpt2"' tokenizer.

Finally, tokenized texts were concatenated end-to-end and split into blocks of 128 tokens.

TODO: Add a different separation token between documents that can be used to stop generation. ## Training

This model was trained in Colab by fine-tuning 'distilgpt2' for 174390 steps (3 epochs).
Default training arguments were used, except for 'learning_rate=2e-5' and 'weight_decay=0.01'.
At the conclusion of training, a training loss of 3.52 and a validation loss of 3.44 were observed.
passage: TAGS
#transformers #pytorch #tensorboard #safetensors #gpt2 #text-generation #en #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# mgfrantz/distilgpt2-finetuned-reddit-tifu

This model was trained to as practice for fine-tuning a causal language model.
There was no intended use case for this model besides having some fun seeing how different things might be screwed up.## Data

This model was trained on "short" subset of 'reddit_tifu' dataset.
The data was split into 90% train and 10% validation using 'dataset.train_test_split', with a seed of 0.

To prepare the data for training, the '"tldr"' and '"documents"' fields were joined by '"\n\n"'.
When multiple items were in the '"tldr"' or '"documents"' fields, only the first item was selected for joining.
These joined documents were tokenized using the '"distilgpt2"' tokenizer.

Finally, tokenized texts were concatenated end-to-end and split into blocks of 128 tokens.

TODO: Add a different separation token between documents that can be used to stop generation.## Training

This model was trained in Colab by fine-tuning 'distilgpt2' for 174390 steps (3 epochs).
Default training arguments were used, except for 'learning_rate=2e-5' and 'weight_decay=0.01'.
At the conclusion of training, a training loss of 3.52 and a validation loss of 3.44 were observed.