neavo
/

modern_bert_multilingual

Model card Files Files and versions

neavo commited on Jan 31

Commit

e99e560

·

verified ·

1 Parent(s): 1a0b784

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -49,7 +49,7 @@ license: apache-2.0
 - 在 `L40*7` 的设备上训练了大约 `100` 个小时，训练量大约 `60B` Token
 - 主要训练参数
   - Batch Size : 1792
-  - Learing Rate : 4e-05
   - Maximum Sequence Length : 512
   - Optimizer : adamw_torch
   - LR Scheduler: warmup_stable_decay

 - 在 `L40*7` 的设备上训练了大约 `100` 个小时，训练量大约 `60B` Token
 - 主要训练参数
   - Batch Size : 1792
+  - Learing Rate : 5e-04
   - Maximum Sequence Length : 512
   - Optimizer : adamw_torch
   - LR Scheduler: warmup_stable_decay