Update README.md

README.md

@@ -76,6 +76,35 @@ scores = batch_scores[:, 1].exp().tolist()
 print(scores) # [0.0020704232156276703, 0.9999990463256836] first document is not relevant, as expected
 ```
 
+# Training
+
+We used [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) to fine-tune Mistral to create FollowIR-7B, with the following training script:
+```bash
+#!/bin/bash
+accelerate launch src/train_bash.py \
+    --stage sft \
+    --do_train \
+    --model_name_or_path "mistralai/Mistral-7B-Instruct-v0.2" \
+    --dataset followIR-train \
+    --template mistral \
+    --output_dir OUTPUT \
+    --finetuning_type lora \
+    --lora_target q_proj,v_proj,o_proj,k_proj \
+    --overwrite_cache \
+    --per_device_train_batch_size 32 \
+    --gradient_accumulation_steps 1 \
+    --lr_scheduler_type cosine \
+    --logging_steps 2 \
+    --save_steps 29 \
+    --learning_rate 3e-5 \
+    --num_train_epochs 8.0 \
+    --plot_loss \
+    --max_length 2048 \
+    --lora_rank 8 \
+    --lora_alpha 16 \
+    --bf16 
+```
+
 # Citation
 
 ```bibtex