Update README.md

README.md

@@ -1,22 +1,56 @@
 ---
-base_model: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
+base_model: meta-llama/meta-llama-3.1-8b-instruct
 tags:
-- text-generation-inference
-- transformers
-- unsloth
-- llama
+- llama adapter
 - trl
+- llama3.1 8b
 license: apache-2.0
 language:
 - en
 ---
+## Model Overview
+A LoRA (Low-Rank Adaptation) fine-tuned adapter for the Llama-3.1-8B language model.
 
-# Uploaded  model
+## Model Details
+- Base Model: meta-llama/Llama-3.1-8B-instruct
+- Adaptation Method: LoRA
 
-- **Developed by:** kevin009
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
+## Training Configuration
+### Training Hyperparameters
+- Learning Rate: 50e-6
+- Batch Size: 2
+- Number of Epochs: 1
+- Training Steps: ~2,000
+- Precision: "BF16"
 
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+### LoRA Configuration
+- Rank (r): 16
+- Alpha: 16
+- Target Modules:
+  - `q_proj` (Query projection)
+  - `k_proj` (Key projection)
+  - `v_proj` (Value projection)
+  - `o_proj` (Output projection)
+  - `up_proj` (Upsampling projection)
+  - `down_proj` (Downsampling projection)
+  - `gate_proj` (Gate projection)
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+## Usage
+This adapter must be used in conjunction with the base Llama-3.1-8B model. 
+
+### Loading the Model
+```python
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load base model
+base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-instruct")
+tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-instruct")
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, "path_to_adapter")
+```
+
+## Limitations and Biases
+- This adapter might inherits some limitations and biases present in the base Llama-3.1-8B-instruct model
+- The training dataset size (~1k steps) is relatively small, which may limit the adapter's effectiveness