Spaces:

nroggendorff
/

train-llama

Runtime error

nroggendorff commited on Aug 4, 2024

Commit

678a941

verified ·

1 Parent(s): 9a8cbf2

Update train.py

Files changed (1) hide show

train.py CHANGED Viewed

@@ -11,8 +11,8 @@ MAX_SEQ_LENGTH = 2048
 BATCH_SIZE = 1
 EPOCHS = 2
 LEARNING_RATE = 1e-4
-FACTOR = 1024
-VOCAB_SIZE = 32000
 INPUT_DATASET = "HuggingFaceTB/smollm-corpus"
 INSTRUCT_DATASET = "nroggendorff/openhermes"
 OUTPUT_REPO = "smallama"

 BATCH_SIZE = 1
 EPOCHS = 2
 LEARNING_RATE = 1e-4
+FACTOR = 2048
+VOCAB_SIZE = 3200
 INPUT_DATASET = "HuggingFaceTB/smollm-corpus"
 INSTRUCT_DATASET = "nroggendorff/openhermes"
 OUTPUT_REPO = "smallama"