pretrain core 2

README.md

@@ -275,6 +275,25 @@ litgpt convert_pretrained_checkpoint ../out/pretrain-core-1/final ../out/pretrai
 CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True litgpt pretrain --config pretrain_core_model_2.yaml
 ```
 
+```
+Seed set to 23
+Time to instantiate model: 0.30 seconds.
+Total parameters: 234,914,304
+Validating ...
+Measured TFLOPs: 13880.23
+Epoch 1 | iter 256 step 1 | loss train: 2.857, val: 3.078 | iter time: 518.53 ms (step) remaining time: 8:49:46
+Epoch 1 | iter 512 step 2 | loss train: 2.839, val: 3.078 | iter time: 461.04 ms (step) remaining time: 7:47:05
+Epoch 1 | iter 768 step 3 | loss train: 2.835, val: 3.078 | iter time: 462.82 ms (step) remaining time: 7:27:16
+Epoch 1 | iter 1024 step 4 | loss train: 2.872, val: 3.078 | iter time: 464.55 ms (step) remaining time: 7:16:22
+Epoch 1 | iter 1280 step 5 | loss train: 2.867, val: 3.078 | iter time: 462.06 ms (step) remaining time: 7:09:02
+Epoch 1 | iter 1536 step 6 | loss train: 2.899, val: 3.078 | iter time: 465.26 ms (step) remaining time: 7:03:30
+Epoch 1 | iter 1792 step 7 | loss train: 2.878, val: 3.078 | iter time: 465.57 ms (step) remaining time: 6:59:00
+Epoch 1 | iter 2048 step 8 | loss train: 2.919, val: 3.078 | iter time: 464.37 ms (step) remaining time: 6:55:10
+Epoch 1 | iter 2304 step 9 | loss train: 2.922, val: 3.078 | iter time: 464.24 ms (step) remaining time: 6:51:45
+Epoch 1 | iter 2560 step 10 | loss train: 2.924, val: 3.078 | iter time: 464.71 ms (step) remaining time: 6:48:39
+# ...
+```
+
 ```bash
 cp ../config-2.json ../out/pretrain-core-2/final/config.json
 ```