Spaces:
Sleeping
Sleeping
Commit
·
c132a96
1
Parent(s):
f4e648b
fix: README.md file format
Browse files
README.md
CHANGED
|
@@ -1,3 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |

|
| 2 |
|
| 3 |
# QuillGPT
|
|
@@ -111,21 +124,21 @@ To train the GPT model, follow these steps:
|
|
| 111 |
2. Write the configurations for transformer and save the file.
|
| 112 |
<br>For example:
|
| 113 |
`json
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
|
| 130 |
3. Train model using script `scripts/train_gpt.py`
|
| 131 |
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: QuillGPT
|
| 3 |
+
emoji: 📉
|
| 4 |
+
colorFrom: yellow
|
| 5 |
+
colorTo: yellow
|
| 6 |
+
sdk: streamlit
|
| 7 |
+
sdk_version: 1.40.2
|
| 8 |
+
app_file: app.py
|
| 9 |
+
pinned: false
|
| 10 |
+
license: mit
|
| 11 |
+
short_description: Implementation of the GPT decoder block in PyTorch
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |

|
| 15 |
|
| 16 |
# QuillGPT
|
|
|
|
| 124 |
2. Write the configurations for transformer and save the file.
|
| 125 |
<br>For example:
|
| 126 |
`json
|
| 127 |
+
{
|
| 128 |
+
"data_path": "data/corpus.txt",
|
| 129 |
+
"vocab_size": 135,
|
| 130 |
+
"batch_size": 32,
|
| 131 |
+
"block_size": 256,
|
| 132 |
+
"max_iters": 3000,
|
| 133 |
+
"eval_interval": 300,
|
| 134 |
+
"learning_rate": 3e-5,
|
| 135 |
+
"eval_iters": 50,
|
| 136 |
+
"n_embd": 1024,
|
| 137 |
+
"n_head": 12,
|
| 138 |
+
"n_layer": 18,
|
| 139 |
+
"dropout": 0.3,
|
| 140 |
+
}
|
| 141 |
+
`
|
| 142 |
|
| 143 |
3. Train model using script `scripts/train_gpt.py`
|
| 144 |
|