Spaces:
Sleeping
Sleeping
Commit
·
c132a96
1
Parent(s):
f4e648b
fix: README.md file format
Browse files
README.md
CHANGED
@@ -1,3 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |

|
2 |
|
3 |
# QuillGPT
|
@@ -111,21 +124,21 @@ To train the GPT model, follow these steps:
|
|
111 |
2. Write the configurations for transformer and save the file.
|
112 |
<br>For example:
|
113 |
`json
|
114 |
-
|
115 |
-
|
116 |
-
|
117 |
-
|
118 |
-
|
119 |
-
|
120 |
-
|
121 |
-
|
122 |
-
|
123 |
-
|
124 |
-
|
125 |
-
|
126 |
-
|
127 |
-
|
128 |
-
|
129 |
|
130 |
3. Train model using script `scripts/train_gpt.py`
|
131 |
|
|
|
1 |
+
---
|
2 |
+
title: QuillGPT
|
3 |
+
emoji: 📉
|
4 |
+
colorFrom: yellow
|
5 |
+
colorTo: yellow
|
6 |
+
sdk: streamlit
|
7 |
+
sdk_version: 1.40.2
|
8 |
+
app_file: app.py
|
9 |
+
pinned: false
|
10 |
+
license: mit
|
11 |
+
short_description: Implementation of the GPT decoder block in PyTorch
|
12 |
+
---
|
13 |
+
|
14 |

|
15 |
|
16 |
# QuillGPT
|
|
|
124 |
2. Write the configurations for transformer and save the file.
|
125 |
<br>For example:
|
126 |
`json
|
127 |
+
{
|
128 |
+
"data_path": "data/corpus.txt",
|
129 |
+
"vocab_size": 135,
|
130 |
+
"batch_size": 32,
|
131 |
+
"block_size": 256,
|
132 |
+
"max_iters": 3000,
|
133 |
+
"eval_interval": 300,
|
134 |
+
"learning_rate": 3e-5,
|
135 |
+
"eval_iters": 50,
|
136 |
+
"n_embd": 1024,
|
137 |
+
"n_head": 12,
|
138 |
+
"n_layer": 18,
|
139 |
+
"dropout": 0.3,
|
140 |
+
}
|
141 |
+
`
|
142 |
|
143 |
3. Train model using script `scripts/train_gpt.py`
|
144 |
|