| # Shakespeare GPT Model | |
| This is a GPT model trained on Shakespeare's text. The model has the following specifications: | |
| - Vocabulary Size: 65 | |
| - Number of Layers: 12 | |
| - Number of Heads: 16 | |
| - Embedding Dimension: 1024 | |
| - Context Length: 256 | |
| - Total Parameters: 151.37M | |
| ## Usage | |
| ```python | |
| from transformers import AutoModel | |
| model = AutoModel.from_pretrained("jatingocodeo/shakespeare-decoder") | |
| ``` | |
| ## Training | |
| This model was trained on Shakespeare's text with a target loss of < 0.099999. | |