README.md

---
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
library_name: peft
---


# Model Card for Fine-Tuned LLaMA Empathy

## Model Summary

Fine-Tuned LLaMA Empathy is a large language model fine-tuned to enhance emotional understanding and generate needs-based responses. This model is designed for use in psychology, therapy, conflict resolution, human-computer interaction, and online moderation. It is based on the Meta-Llama-3.1-8B-Instruct model and utilizes LoRA (Low-Rank Adaptation) for efficient fine-tuning.

## Model Details

### Model Description

- **Developed by:** AI Medical in collaboration with Ruslanmv.com
- **Funded by:** 
- **Shared by:** AI Medical
- **Model type:** Fine-tuned Meta-Llama-3.1-8B-Instruct
- **Language(s) (NLP):** English
- **License:** Creative Commons Attribution 4.0 International License (CC BY 4.0)
- **Fine-tuned from model:** meta-llama/Meta-Llama-3.1-8B-Instruct

### Model Sources
- **Repository:** [Hugging Face Model Repository](https://huggingface.co/ruslanmv/fine_tuned_llama_empathy)


## Uses

### Direct Use
- **Psychology & Therapy:** Assisting professionals in understanding and responding empathetically to patient emotions.
- **Conflict Resolution:** Helping mediators decode emotional expressions and address underlying needs.
- **Human-Computer Interaction:** Enhancing chatbots and virtual assistants with emotionally aware responses.
- **Social Media Moderation:** Reducing toxicity and improving online discourse through need-based responses.
- **Education:** Supporting emotional intelligence training and communication skill development.

### Downstream Use
- Fine-tuning for specialized applications in mental health, conflict resolution, or AI-driven assistance.
- Integration into virtual therapists, mental health applications, and online support systems.

### Out-of-Scope Use
- Not a substitute for professional psychological evaluation or medical treatment.
- Not suitable for high-risk applications requiring absolute accuracy in emotional interpretation.

## Bias, Risks, and Limitations

- **Bias:** As with any NLP model, biases may exist due to the dataset and training methodology.  LLaMA models, in particular, have shown biases.
- **Risk of Misinterpretation:** Emotional expressions are subjective and may be misclassified in complex scenarios.
- **Generalization Limitations:** May not fully capture cultural and contextual variations in emotional expressions.

### Recommendations
Users should verify outputs before applying them in professional or high-stakes settings. Continuous evaluation and user feedback are recommended.

## How to Get Started with the Model

```python
from transformers import pipeline

model_name = "ruslanmv/fine_tuned_llama_empathy"
model = pipeline("text-generation", model=model_name)

prompt = "I feel betrayed."
response = model(prompt, max_length=50)
print(response)
```

## Training Details

### Training Data
- **Dataset:** Annotated dataset mapping evaluative expressions to emotions and needs.
- **Annotations:** 1,500+ labeled examples linking expressions to emotional states and corresponding needs.

### Training Procedure

#### Preprocessing
- Tokenized using Hugging Face `transformers` library.
- Augmented with synonym variations and paraphrased sentences.

#### Training Hyperparameters
- **Training regime:** Mixed precision training using LoRA.
- **Batch size:** 32
- **Learning rate:** 2e-5
- **Training steps:** 1k
- **Hardware:** 1x A100 GPU using DeepSpeed ZeRO-3

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data
- Held-out dataset containing unseen evaluative expressions.

#### Factors
- Performance across different emotional expression categories.
- Sensitivity to nuanced phrasing and variations.

#### Metrics
- **Accuracy:** Measures correct classification of emotions and needs.
- **Precision & Recall:** Evaluates the balance between capturing true emotions and avoiding false positives.
- **F1-Score:** Measures the balance between precision and recall.

### Results

- **Accuracy:** 89.5%
- **F1-Score:** 87.2%
- **Latency:** <500ms response time

## Environmental Impact

- **Hardware Type:** A100 GPUs
- **Training Time:** hours
- **Carbon Emitted:** Estimated using [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute).

## Technical Specifications

### Model Architecture and Objective
- Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
- Fine-tuned using LoRA for parameter-efficient training.  Key LoRA parameters: `r=8`, `lora_alpha=16`, `lora_dropout=0.2`, `target_modules=["v_proj", "q_proj"]`

### Compute Infrastructure
- **Hardware:** AWS spot instances (1x A100 GPUs)
- **Software:** Hugging Face `transformers`, PEFT, PyTorch

## Citation

If you use this model, please cite:

```bibtex
@misc{ai-medical_2025,
  author = {AI Medical, ruslanmv.com},
  title = {Fine-Tuned LLaMA Empathy},
  year = {2025},
  howpublished = {\url{[https://huggingface.co/ruslanmv/fine_tuned_llama_empathy](https://huggingface.co/ruslanmv/fine_tuned_llama_empathy)}}
}
```

## More Information
- **Model Card Authors:** AI Medical Team, ruslanmv.com
- **Framework Versions:** PEFT 0.14.0