Update README.md
Browse files
README.md
CHANGED
@@ -83,7 +83,7 @@ For ReFT, the nodes in the last 8 layers were unfrozen with attention to allow t
|
|
83 |
|
84 |
After 3 to 4 epochs, the model began to overfit regardless of the strategies employed. Increasing both batch size and the number of epochs resulted in higher final training and evaluation cross-entropy.
|
85 |
|
86 |
-
Following an extensive grid search, supervised fine-tuning of Llama 3.1-8B with LoRA+ and the parameters mentioned
|
87 |
|
88 |
#### Preprocessing [optional]
|
89 |
|
|
|
83 |
|
84 |
After 3 to 4 epochs, the model began to overfit regardless of the strategies employed. Increasing both batch size and the number of epochs resulted in higher final training and evaluation cross-entropy.
|
85 |
|
86 |
+
Following an extensive grid search, supervised fine-tuning of [Llama 3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) with LoRA+ and the parameters mentioned below yielded the best training and evaluation cross-entropy.
|
87 |
|
88 |
#### Preprocessing [optional]
|
89 |
|