tangledgroup
/

tangled-alpha-0.9-core

Text Generation

Model card Files Files and versions

mtasic85 commited on Mar 15

Commit

a2e4903

·

1 Parent(s): 8b6e5e1

cpt core 4

Files changed (1) hide show

scripts/cpt_core_model_4.py +1 -1

scripts/cpt_core_model_4.py CHANGED Viewed

@@ -111,7 +111,7 @@ trainer = UnslothTrainer(
         bf16=is_bfloat16_supported(),
         logging_steps=1,
         # optim='adamw_8bit',
-        optim='adamw',
         weight_decay=0.01,
         lr_scheduler_type='cosine',
         seed=23,

         bf16=is_bfloat16_supported(),
         logging_steps=1,
         # optim='adamw_8bit',
+        optim='adamw_torch_fused',
         weight_decay=0.01,
         lr_scheduler_type='cosine',
         seed=23,