nenad1002
/

quantum-research-bot-v1.0

Text Generation

text-generation-inference

Model card Files Files and versions Community

nenad1002 commited on Sep 2, 2024

Commit

e81abeb

·

verified ·

1 Parent(s): 82430f0

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -92,7 +92,7 @@ Following an extensive grid search, supervised fine-tuning of [Llama 3.1-8B](htt
 #### Training Hyperparameters
 - **Training regime:**
-- bfloat16 precision
 - LORA rank: 8
 - LORA alpha: 16
 - LORA droput: 0.1

 #### Training Hyperparameters
 - **Training regime:**
+- bfloat16 precision (nf4 for qLoRA)
 - LORA rank: 8
 - LORA alpha: 16
 - LORA droput: 0.1