Spaces:

nroggendorff
/

train-llama

Runtime error

nroggendorff commited on Jul 31, 2024

Commit

82ca3e9

verified ·

1 Parent(s): 721a451

Update train.py

Files changed (1) hide show

train.py CHANGED Viewed

@@ -11,7 +11,7 @@ MAX_SEQ_LENGTH = 512
 BATCH_SIZE = 64
 EPOCHS = 3
 LEARNING_RATE = 2e-2
-FACTOR = 16
 VOCAB_SIZE = 32000
 INPUT_DATASET = "nroggendorff/elephant"
 OUTPUT_REPO = "smallama"

 BATCH_SIZE = 64
 EPOCHS = 3
 LEARNING_RATE = 2e-2
+FACTOR = 256
 VOCAB_SIZE = 32000
 INPUT_DATASET = "nroggendorff/elephant"
 OUTPUT_REPO = "smallama"