Update README.md

README.md

@@ -44,53 +44,6 @@ This instruction-following llm was built via parameter-efficient QLoRA finetunin
 
 We use Eleuther.AI's [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above, the same version as Hugging Face's [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
 
-## Training
-
-It took ~1 hour to train 1 epoch on 1x A100.
-
-Prompt format:
-This model (and all my future releases) uses the [ChatML](https://huggingface.co/docs/transformers/chat_templating#what-template-should-i-use) prompt format, which was developed by OpenAI.
-
-```
-<|im_start|>system
-You are a helpful assistant.<|im_end|>
-<|im_start|>user
-{prompt}<|im_end|>
-<|im_start|>assistant
-```
-
-### Training Hyperparameters
-
-
-We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune llms on instruction-following datasets.
-
-The following `TrainingArguments` config was used:
-
-- num_train_epochs = 1
-- auto_find_batch_size = True
-- gradient_accumulation_steps = 1
-- optim = "paged_adamw_32bit"
-- save_strategy = "epoch"
-- learning_rate = 3e-4
-- lr_scheduler_type = "cosine"
-- warmup_ratio = 0.03
-- logging_strategy = "steps"
-- logging_steps = 25
-- bf16 = True
-
-The following `bitsandbytes` quantization config was used:
-
-- quant_method: bitsandbytes
-- load_in_8bit: False
-- load_in_4bit: True
-- llm_int8_threshold: 6.0
-- llm_int8_skip_modules: None
-- llm_int8_enable_fp32_cpu_offload: False
-- llm_int8_has_fp16_weight: False
-- bnb_4bit_quant_type: nf4
-- bnb_4bit_use_double_quant: False
-- bnb_4bit_compute_dtype: bfloat16
-
 ## How to Get Started with the Model
 
 Use the code below to get started with the model.
@@ -180,6 +133,53 @@ Remember, when writing emails, always keep in mind your audience and their prefe
 |:-----------------------------:|:----------------------:|:---------------------:|:-------------:|:-----------------------:|
 | 3.1                        | 1x A100 (40 GB SXM)  | torch               | fp16    | 13                    |
 
+## Training
+
+It took ~1 hour to train 1 epoch on 1x A100.
+
+Prompt format:
+This model (and all my future releases) uses the [ChatML](https://huggingface.co/docs/transformers/chat_templating#what-template-should-i-use) prompt format, which was developed by OpenAI.
+
+```
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+### Training Hyperparameters
+
+
+We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune llms on instruction-following datasets.
+
+The following `TrainingArguments` config was used:
+
+- num_train_epochs = 1
+- auto_find_batch_size = True
+- gradient_accumulation_steps = 1
+- optim = "paged_adamw_32bit"
+- save_strategy = "epoch"
+- learning_rate = 3e-4
+- lr_scheduler_type = "cosine"
+- warmup_ratio = 0.03
+- logging_strategy = "steps"
+- logging_steps = 25
+- bf16 = True
+
+The following `bitsandbytes` quantization config was used:
+
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: False
+- bnb_4bit_compute_dtype: bfloat16
+
 
 ## Model Card Contact