
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# distilgpt2-finetuned-eap

This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.3557
- Train Accuracy: 0.0010
- Validation Loss: 7.4820
- Validation Accuracy: 0.0
- Epoch: 49

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 0.0002, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: float32

### Training results

| Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Epoch |
|:----------:|:--------------:|:---------------:|:-------------------:|:-----:|
| 4.6280     | 0.0015         | 4.3578          | 0.0013              | 0     |
| 4.2572     | 0.0018         | 4.2845          | 0.0013              | 1     |
| 4.0662     | 0.0018         | 4.2587          | 0.0012              | 2     |
| 3.9066     | 0.0016         | 4.2470          | 0.0012              | 3     |
| 3.7578     | 0.0016         | 4.2493          | 0.0013              | 4     |
| 3.6177     | 0.0018         | 4.2728          | 0.0013              | 5     |
| 3.4741     | 0.0017         | 4.3208          | 0.0012              | 6     |
| 3.3366     | 0.0017         | 4.3542          | 0.0012              | 7     |
| 3.1946     | 0.0016         | 4.3973          | 0.0012              | 8     |
| 3.0581     | 0.0016         | 4.4947          | 0.0014              | 9     |
| 2.9171     | 0.0017         | 4.5970          | 0.0013              | 10    |
| 2.7766     | 0.0016         | 4.6691          | 0.0014              | 11    |
| 2.6373     | 0.0015         | 4.7961          | 0.0012              | 12    |
| 2.4986     | 0.0014         | 4.8906          | 0.0002              | 13    |
| 2.3600     | 0.0015         | 4.9836          | 0.0002              | 14    |
| 2.2307     | 0.0014         | 5.1439          | 0.0001              | 15    |
| 2.1054     | 0.0015         | 5.3017          | 0.0001              | 16    |
| 1.9798     | 0.0015         | 5.4037          | 0.0001              | 17    |
| 1.8679     | 0.0014         | 5.5184          | 0.0001              | 18    |
| 1.7544     | 0.0013         | 5.6429          | 0.0001              | 19    |
| 1.6486     | 0.0013         | 5.7368          | 0.0001              | 20    |
| 1.5492     | 0.0013         | 5.8070          | 0.0001              | 21    |
| 1.4525     | 0.0013         | 5.9248          | 0.0001              | 22    |
| 1.3725     | 0.0013         | 5.9879          | 0.0                 | 23    |
| 1.2901     | 0.0011         | 6.1063          | 0.0001              | 24    |
| 1.2178     | 0.0014         | 6.1828          | 0.0001              | 25    |
| 1.1429     | 0.0012         | 6.2581          | 0.0001              | 26    |
| 1.0831     | 0.0011         | 6.3003          | 0.0000              | 27    |
| 1.0266     | 0.0012         | 6.3558          | 0.0001              | 28    |
| 0.9673     | 0.0012         | 6.4831          | 0.0000              | 29    |
| 0.9116     | 0.0012         | 6.5555          | 0.0                 | 30    |
| 0.8652     | 0.0012         | 6.6239          | 0.0                 | 31    |
| 0.8198     | 0.0013         | 6.6751          | 0.0                 | 32    |
| 0.7795     | 0.0013         | 6.7499          | 0.0                 | 33    |
| 0.7410     | 0.0010         | 6.7741          | 0.0000              | 34    |
| 0.7015     | 0.0012         | 6.8395          | 0.0000              | 35    |
| 0.6679     | 0.0011         | 6.9150          | 0.0                 | 36    |
| 0.6367     | 0.0010         | 6.9847          | 0.0                 | 37    |
| 0.6034     | 0.0011         | 7.0334          | 0.0001              | 38    |
| 0.5756     | 0.0010         | 7.0516          | 0.0000              | 39    |
| 0.5442     | 0.0011         | 7.1220          | 0.0                 | 40    |
| 0.5188     | 0.0010         | 7.1494          | 0.0000              | 41    |
| 0.4971     | 0.0010         | 7.2100          | 0.0                 | 42    |
| 0.4711     | 0.0010         | 7.2883          | 0.0001              | 43    |
| 0.4501     | 0.0011         | 7.2946          | 0.0                 | 44    |
| 0.4274     | 0.0011         | 7.3313          | 0.0001              | 45    |
| 0.4066     | 0.0011         | 7.3620          | 0.0000              | 46    |
| 0.3898     | 0.0011         | 7.4119          | 0.0000              | 47    |
| 0.3679     | 0.0010         | 7.4769          | 0.0001              | 48    |
| 0.3557     | 0.0010         | 7.4820          | 0.0                 | 49    |


### Framework versions

- Transformers 4.21.1
- TensorFlow 2.8.2
- Datasets 2.4.0
- Tokenizers 0.12.1

{"license": "apache-2.0", "tags": ["generated_from_keras_callback"], "model-index": [{"name": "distilgpt2-finetuned-eap", "results": []}]}
transformers tf gpt2 text-generation generated_from_keras_callback license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #tf #gpt2 #text-generation #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

distilgpt2-finetuned-eap
========================


This model is a fine-tuned version of distilgpt2 on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 0.3557
* Train Accuracy: 0.0010
* Validation Loss: 7.4820
* Validation Accuracy: 0.0
* Epoch: 49


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': 0.0002, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.21.1
* TensorFlow 2.8.2
* Datasets 2.4.0
* Tokenizers 0.12.1



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': 0.0002, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.21.1
* TensorFlow 2.8.2
* Datasets 2.4.0
* Tokenizers 0.12.1
TAGS
#transformers #tf #gpt2 #text-generation #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': 0.0002, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.21.1
* TensorFlow 2.8.2
* Datasets 2.4.0
* Tokenizers 0.12.1
passage: TAGS
#transformers #tf #gpt2 #text-generation #generated_from_keras_callback #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'AdamWeightDecay', 'learning\_rate': 0.0002, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight\_decay\_rate': 0.01}
* training\_precision: float32### Training results### Framework versions


* Transformers 4.21.1
* TensorFlow 2.8.2
* Datasets 2.4.0
* Tokenizers 0.12.1