Commit
·
921e3b3
1
Parent(s):
1a735fe
Update README.md
Browse files
README.md
CHANGED
@@ -35,7 +35,7 @@ Polyglot-Ko was trained on 1.2TB Korean Dataset, a large-scale curated dataset c
|
|
35 |
|
36 |
## Training procedure
|
37 |
|
38 |
-
Polyglot-Ko was trained for 213 billion tokens over 102,000 steps on 256 * A100 GPUs. It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token correctly.
|
39 |
|
40 |
## How to use
|
41 |
|
|
|
35 |
|
36 |
## Training procedure
|
37 |
|
38 |
+
Polyglot-Ko was trained for 213 billion tokens over 102,000 steps on 256 * A100 GPUs with [GPT-NeoX framework](https://github.com/EleutherAI/gpt-neox). It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token correctly.
|
39 |
|
40 |
## How to use
|
41 |
|