File size: 3,694 Bytes
217e4bc 3218d26 217e4bc 5ba6652 217e4bc 3218d26 217e4bc 5ba6652 3218d26 5ba6652 217e4bc 5ba6652 217e4bc 5ba6652 217e4bc 3218d26 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
---
language: en
library_name: transformers
license: llama3.1
base_model: meta-llama/Llama-3.1-8B-Instruct
pipeline_tag: text-generation
tags:
- llama-3.1
- instruction-tuned
datasets:
- OpenAssistant/oasst1
- databricks/databricks-dolly-15k
- Open-Orca/OpenOrca
- mlabonne/open-perfectblend
- tatsu-lab/alpaca
model-index:
- name: utkmst/chimera-beta-test2-lora-merged
results:
- task:
type: text-generation
dataset:
type: leaderboard
name: Overall Leaderboard
metrics:
- name: acc_norm
type: acc_norm
value: 0.4440
verified: true
- name: acc
type: acc
value: 0.2992
verified: true
- name: exact_match
type: exact_match
value: 0.0951
verified: true
- task:
type: text-generation
dataset:
type: bbh
name: BBH (Big Bench Hard)
metrics:
- name: acc_norm
type: acc_norm
value: 0.4773
verified: true
- task:
type: text-generation
dataset:
type: gpqa
name: GPQA (Google-Patched Question Answering)
metrics:
- name: acc_norm
type: acc_norm
value: 0.3036
verified: true
- task:
type: text-generation
dataset:
type: math
name: Math
metrics:
- name: exact_match
type: exact_match
value: 0.0951
verified: true
- task:
type: text-generation
dataset:
type: mmlu_pro
name: MMLU-Pro
metrics:
- name: acc
type: acc
value: 0.2992
verified: true
- task:
type: text-generation
dataset:
type: musr
name: MUSR (Multi-Step Reasoning)
metrics:
- name: acc_norm
type: acc_norm
value: 0.4113
verified: true
---
# utkmst/chimera-beta-test2-lora-merged
## Model Description
This model is a fine-tuned version of Meta's Llama-3.1-8B-Instruct model, created through LoRA fine-tuning on multiple instruction datasets, followed by merging the adapter weights with the base model.
## Architecture
- **Base Model**: meta-llama/Llama-3.1-8B-Instruct
- **Size**: 8.03B parameters
- **Type**: Decoder-only transformer
- **Format**: SafeTensors (full precision)
## Training Details
- **Training Method**: LoRA fine-tuning followed by adapter merging
- **LoRA Configuration**:
- Rank: 8
- Alpha: 16
- Trainable modules: Attention layers and feed-forward networks
- **Training Hyperparameters**:
- Learning rate: 2e-4
- Batch size: 2
- Training epochs: 1
- Optimizer: AdamW with constant scheduler
## Intended Use
This model is designed for:
- General purpose assistant capabilities
- Question answering and knowledge retrieval
- Creative content generation
- Instructional guidance
## Limitations
- Base model limitations including potential hallucinations and factual inaccuracies
- Limited context window compared to larger models
- Knowledge cutoff from the base Llama-3.1 model
- May exhibit biases present in training data
- Performance on specialized tasks may vary
## Usage with Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("utkmst/chimera-beta-test2-lora-merged")
tokenizer = AutoTokenizer.from_pretrained("utkmst/chimera-beta-test2-lora-merged")
```
## License
This model inherits the license from Meta's Llama 3.1. |