Update README.md
Browse files
README.md
CHANGED
|
@@ -69,7 +69,12 @@ pipeline("Xin chào!")
|
|
| 69 |
#### Training Hyperparameters
|
| 70 |
|
| 71 |
- **Training regime:** bf16 mixed precision <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 72 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
|
| 74 |
## Evaluation
|
| 75 |
|
|
|
|
| 69 |
#### Training Hyperparameters
|
| 70 |
|
| 71 |
- **Training regime:** bf16 mixed precision <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 72 |
+
- **Data sequence length:** 8192
|
| 73 |
+
- **Tensor model parallel size:** 4
|
| 74 |
+
- **Pipelinemodel parallel size:** 1
|
| 75 |
+
- **Context parallel size:** 1
|
| 76 |
+
- **Micro batch size:** 1
|
| 77 |
+
- **Global batch size:** 512
|
| 78 |
|
| 79 |
## Evaluation
|
| 80 |
|