
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilgpt2-finetune-acl22

This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on the [ACL-anthology-corpus](https://github.com/shauryr/ACL-anthology-corpus) dataset.
It achieves the following results on the evaluation set:
- Loss: 3.4835

## Model description

We finetune the gpt2 LLM on the full-text from ACL-anthology-corpus


### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 256
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 3.6676        | 1.0   | 9852  | 3.5623          |
| 3.5959        | 2.0   | 19704 | 3.4995          |
| 3.5719        | 3.0   | 29556 | 3.4835          |


### Framework versions

- Transformers 4.21.2
- Pytorch 1.12.1
- Datasets 2.4.0
- Tokenizers 0.12.1

## What can it do?

Write introductions/abstract

- Prompt : Toward Annotator Group Bias in Crowdsourcing. Introduction 
- Generation : Toward Annotator Group Bias in Crowdsourcing. Introduction Online platforms for crowdsourcing have received increasing scrutiny in recent years as platforms for online data analytics require an additional layer of content that allows users to interact and be informed about their quality.

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "widget": [{"text": "Toward Annotator Group Bias in Crowdsourcing. Introduction", "example_title": "Introduction"}, {"text": "Over the last few years, there has been a move towards data", "example_title": "Over the last few years"}, {"text": "We introduce a new language representation", "example_title": "new language representation"}, {"text": "Acknowledgements. This research is supported by the National Science Foundation", "example_title": "Acknowledgements"}, {"text": "We hope that our work serves not only to inform the NLP ", "example_title": "We hope that"}], "model-index": [{"name": "distilgpt2-finetune-acl22", "results": []}]}
transformers pytorch safetensors gpt2 text-generation generated_from_trainer license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #safetensors #gpt2 #text-generation #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

distilgpt2-finetune-acl22
=========================


This model is a fine-tuned version of distilgpt2 on the ACL-anthology-corpus dataset.
It achieves the following results on the evaluation set:


* Loss: 3.4835


Model description
-----------------


We finetune the gpt2 LLM on the full-text from ACL-anthology-corpus


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 256
* eval\_batch\_size: 64
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3


### Training results



### Framework versions


* Transformers 4.21.2
* Pytorch 1.12.1
* Datasets 2.4.0
* Tokenizers 0.12.1


What can it do?
---------------


Write introductions/abstract


* Prompt : Toward Annotator Group Bias in Crowdsourcing. Introduction
* Generation : Toward Annotator Group Bias in Crowdsourcing. Introduction Online platforms for crowdsourcing have received increasing scrutiny in recent years as platforms for online data analytics require an additional layer of content that allows users to interact and be informed about their quality.



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 256
* eval\_batch\_size: 64
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3 ### Training results ### Framework versions


* Transformers 4.21.2
* Pytorch 1.12.1
* Datasets 2.4.0
* Tokenizers 0.12.1


What can it do?
---------------


Write introductions/abstract


* Prompt : Toward Annotator Group Bias in Crowdsourcing. Introduction
* Generation : Toward Annotator Group Bias in Crowdsourcing. Introduction Online platforms for crowdsourcing have received increasing scrutiny in recent years as platforms for online data analytics require an additional layer of content that allows users to interact and be informed about their quality.
TAGS
#transformers #pytorch #safetensors #gpt2 #text-generation #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 256
* eval\_batch\_size: 64
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3 ### Training results ### Framework versions


* Transformers 4.21.2
* Pytorch 1.12.1
* Datasets 2.4.0
* Tokenizers 0.12.1


What can it do?
---------------


Write introductions/abstract


* Prompt : Toward Annotator Group Bias in Crowdsourcing. Introduction
* Generation : Toward Annotator Group Bias in Crowdsourcing. Introduction Online platforms for crowdsourcing have received increasing scrutiny in recent years as platforms for online data analytics require an additional layer of content that allows users to interact and be informed about their quality.
passage: TAGS
#transformers #pytorch #safetensors #gpt2 #text-generation #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 256
* eval\_batch\_size: 64
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3### Training results### Framework versions


* Transformers 4.21.2
* Pytorch 1.12.1
* Datasets 2.4.0
* Tokenizers 0.12.1


What can it do?
---------------


Write introductions/abstract


* Prompt : Toward Annotator Group Bias in Crowdsourcing. Introduction
* Generation : Toward Annotator Group Bias in Crowdsourcing. Introduction Online platforms for crowdsourcing have received increasing scrutiny in recent years as platforms for online data analytics require an additional layer of content that allows users to interact and be informed about their quality.