cc-100-done Done

README.md

@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model was trained from scratch on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6233
+- Loss: 1.6150
 
 ## Model description
 
@@ -33,14 +33,14 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-07
-- train_batch_size: 32
+- learning_rate: 1e-07
+- train_batch_size: 16
 - eval_batch_size: 8
 - seed: 42
 - distributed_type: multi-GPU
 - num_devices: 8
 - gradient_accumulation_steps: 16
-- total_train_batch_size: 4096
+- total_train_batch_size: 2048
 - total_eval_batch_size: 64
 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
@@ -50,7 +50,9 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 25.4141       | 0.5424 | 2500 | 1.6233          |
+| 25.6476       | 0.2712 | 2500 | 1.6198          |
+| 25.4537       | 0.5424 | 5000 | 1.6171          |
+| 25.8678       | 0.8137 | 7500 | 1.6150          |
 
 
 ### Framework versions