readme
Browse files
README.md
CHANGED
|
@@ -103,6 +103,29 @@ Epoch 1 | iter 512 step 8 | loss train: 11.938, val: n/a | iter time: 363.84 ms
|
|
| 103 |
Epoch 1 | iter 576 step 9 | loss train: 11.920, val: n/a | iter time: 362.75 ms (step) remaining time: 3 days, 0:13:59
|
| 104 |
Epoch 1 | iter 640 step 10 | loss train: 11.900, val: n/a | iter time: 363.46 ms (step) remaining time: 2 days, 23:07:06
|
| 105 |
# ...
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
```
|
| 107 |
|
| 108 |
Backup `wandb`:
|
|
|
|
| 103 |
Epoch 1 | iter 576 step 9 | loss train: 11.920, val: n/a | iter time: 362.75 ms (step) remaining time: 3 days, 0:13:59
|
| 104 |
Epoch 1 | iter 640 step 10 | loss train: 11.900, val: n/a | iter time: 363.46 ms (step) remaining time: 2 days, 23:07:06
|
| 105 |
# ...
|
| 106 |
+
Epoch 1 | iter 643264 step 10051 | loss train: 2.834, val: 2.669 | iter time: 360.50 ms (step) remaining time: 0:03:59
|
| 107 |
+
Epoch 2 | iter 643328 step 10052 | loss train: 2.837, val: 2.669 | iter time: 359.53 ms (step) remaining time: 0:03:37
|
| 108 |
+
Epoch 2 | iter 643392 step 10053 | loss train: 2.768, val: 2.669 | iter time: 362.83 ms (step) remaining time: 0:03:15
|
| 109 |
+
Epoch 2 | iter 643456 step 10054 | loss train: 2.695, val: 2.669 | iter time: 363.85 ms (step) remaining time: 0:02:53
|
| 110 |
+
Epoch 2 | iter 643520 step 10055 | loss train: 2.768, val: 2.669 | iter time: 365.40 ms (step) remaining time: 0:02:30
|
| 111 |
+
Epoch 2 | iter 643584 step 10056 | loss train: 2.710, val: 2.669 | iter time: 364.72 ms (step) remaining time: 0:02:08
|
| 112 |
+
Epoch 2 | iter 643648 step 10057 | loss train: 2.749, val: 2.669 | iter time: 365.00 ms (step) remaining time: 0:01:46
|
| 113 |
+
Epoch 2 | iter 643712 step 10058 | loss train: 2.748, val: 2.669 | iter time: 363.42 ms (step) remaining time: 0:01:24
|
| 114 |
+
Epoch 2 | iter 643776 step 10059 | loss train: 2.710, val: 2.669 | iter time: 364.49 ms (step) remaining time: 0:01:02
|
| 115 |
+
Epoch 2 | iter 643840 step 10060 | loss train: 2.738, val: 2.669 | iter time: 364.43 ms (step) remaining time: 0:00:39
|
| 116 |
+
Epoch 2 | iter 643904 step 10061 | loss train: 2.734, val: 2.669 | iter time: 364.94 ms (step) remaining time: 0:00:17
|
| 117 |
+
Validating ...
|
| 118 |
+
Final evaluation | val loss: 2.669 | val ppl: 14.422
|
| 119 |
+
Saving checkpoint to '../out/pretrain-core-0/final/lit_model.pth'
|
| 120 |
+
----------------------------------------
|
| 121 |
+
| Performance
|
| 122 |
+
| - Total tokens : 5,275,279,360
|
| 123 |
+
| - Training Time : 223314.37 s
|
| 124 |
+
| - Tok/sec : 5541.09 tok/s
|
| 125 |
+
| ----------------------------------------
|
| 126 |
+
| Memory Usage
|
| 127 |
+
| - Memory Used : 22.33 GB
|
| 128 |
+
----------------------------------------
|
| 129 |
```
|
| 130 |
|
| 131 |
Backup `wandb`:
|