Spaces:

nroggendorff
/

train-llama

Runtime error

nroggendorff commited on Aug 1, 2024

Commit

d2b80b3

verified ·

1 Parent(s): 3fa9d6e

Update train.py

Files changed (1) hide show

train.py CHANGED Viewed

@@ -16,9 +16,9 @@ VOCAB_SIZE = 32000
 INPUT_DATASET = "nroggendorff/openhermes"
 OUTPUT_REPO = "smallama"
 FP16 = True
-WARMUP_STEPS = 500
 DECAY = 0.01
-GRADIENT_ACCUMULATION_STEPS = 4
 CLIPPING = 1.0
 PUSH_TO_HUB = True

 INPUT_DATASET = "nroggendorff/openhermes"
 OUTPUT_REPO = "smallama"
 FP16 = True
+WARMUP_STEPS = 200
 DECAY = 0.01
+GRADIENT_ACCUMULATION_STEPS = 1
 CLIPPING = 1.0
 PUSH_TO_HUB = True