hyunwoongko commited on
Commit
921e3b3
·
1 Parent(s): 1a735fe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -35,7 +35,7 @@ Polyglot-Ko was trained on 1.2TB Korean Dataset, a large-scale curated dataset c
35
 
36
  ## Training procedure
37
 
38
- Polyglot-Ko was trained for 213 billion tokens over 102,000 steps on 256 * A100 GPUs. It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token correctly.
39
 
40
  ## How to use
41
 
 
35
 
36
  ## Training procedure
37
 
38
+ Polyglot-Ko was trained for 213 billion tokens over 102,000 steps on 256 * A100 GPUs with [GPT-NeoX framework](https://github.com/EleutherAI/gpt-neox). It was trained as an autoregressive language model, using cross-entropy loss to maximize the likelihood of predicting the next token correctly.
39
 
40
  ## How to use
41