Update README.md

README.md

@@ -11,12 +11,122 @@ language:
 - en
 ---
 
-# Uploaded  model
+# Model Card for Mistral-7B-Instruct-v0.1-Unsloth-MedicalQA
 
-- **Developed by:** bouthros
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/mistral-7b-instruct-v0.1-bnb-4bit
+<img src="https://files.oaiusercontent.com/file-SRkkqbc6KKUWGAfvWfrZpA?se=2025-01-11T20%3A14%3A07Z&sp=r&sv=2024-08-04&sr=b&rscc=max-age%3D604800%2C%20immutable%2C%20private&rscd=attachment%3B%20filename%3D9f951e1f-ad60-431b-b016-e4d79f30a3ab.webp&sig=PwbELJUHXlMlgk3T4MoDPH7nVYfPEXN0ypjadk1DuEc%3D" alt="drawing" width="400"/>
 
-This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+<font color="FF0000" size="5"><b>
+This is a medical question-answering model fine-tuned for healthcare domain</b></font>
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+<br><b>Foundation Model: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1<br/>
+Dataset: https://huggingface.co/datasets/Laurent1/MedQuad-MedicalQnADataset_128tokens_max<br/></b>
+
+The model has been fine-tuned using CUDA-enabled GPU hardware with optimized training through [Unsloth](https://github.com/unslothai/unsloth).
+
+[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="100"/>](https://github.com/unslothai/unsloth)
+
+
+## Model Details
+
+The model is based upon the foundation model: Mistral-7B-Instruct-v0.1.<br/>
+It has been tuned with Supervised Fine-tuning Trainer using the Unsloth optimization framework for faster and more efficient training.
+
+### Libraries
+- unsloth
+- transformers
+- torch
+- trl
+- peft
+- einops
+- bitsandbytes
+- datasets
+
+## Training Configuration
+
+### Model Parameters
+- max_sequence_length = 2048
+- load_in_4bit = True
+- LoRA rank (r) = 32
+- lora_alpha = 16
+- lora_dropout = 0
+
+### Target Modules for LoRA
+- q_proj
+- k_proj
+- v_proj
+- o_proj
+- gate_proj
+- up_proj
+- down_proj
+
+### Training Hyperparameters
+- per_device_train_batch_size = 2
+- gradient_accumulation_steps = 16
+- warmup_steps = 5
+- warmup_ratio = 0.03
+- max_steps = 1600
+- learning_rate = 1e-4
+- weight_decay = 0.01
+- lr_scheduler_type = "linear"
+- optimizer = "paged_adamw_32bit"
+
+## Training Statistics
+
+### Hardware Utilization
+- Training duration: 10,561.28 seconds (approximately 176.02 minutes)
+- Peak reserved memory: 5.416 GB
+- Peak reserved memory for training: 0.748 GB
+- Peak reserved memory % of max memory: 13.689%
+- Peak reserved memory for training % of max memory: 1.891%
+
+### Dataset
+The model was trained on the MedQuad dataset, which contains medical questions and answers. The training data was processed using a chat template format for instruction-tuning.
+
+## Bias, Risks, and Limitations
+
+<font color="FF0000">
+Users (both direct and downstream) should be aware of the following:
+
+1. This model is intended for medical question-answering but should not be used as a substitute for professional medical advice.
+2. The model's responses should be verified by healthcare professionals before making any medical decisions.
+3. Generation of plausible yet incorrect medical information remains a possibility.
+4. The model's knowledge is limited to its training data and may not cover all medical conditions or recent medical developments.
+</font>
+
+## Usage
+
+The model can be loaded and used with the Unsloth library:
+
+```python
+from unsloth import FastLanguageModel
+max_seq_length = 2048  # Choose any! We auto support RoPE Scaling internally!
+dtype = (
+    None  # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
+)
+model, tokenizer = FastLanguageModel.from_pretrained(
+    "bouthros/Mistral-7B-Instruct-v0.1-Unsloth-MedicalQA",
+    max_seq_length=2048,
+    load_in_4bit=True,
+)
+```
+
+Example usage:
+```python
+messages = [
+    {"from": "human", "value": "What are the types of liver cancer?"},
+]
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to("cuda")
+```
+
+## Model Access
+
+The model is available on Hugging Face Hub at: bouthros/Mistral-7B-Instruct-v0.1-Unsloth-MedicalQA
+
+## Citation
+
+If you use this model, please cite the original Mistral-7B-Instruct-v0.1 model and the MedQuad dataset.