|
--- |
|
base_model: |
|
- Sao10K/L3.3-70B-Euryale-v2.3 |
|
- Sao10K/70B-L3.3-mhnnn-x1 |
|
- EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1 |
|
- Doctor-Shotgun/L3.3-70B-Magnum-v4-SE |
|
- Sao10K/70B-L3.3-Cirrus-x1 |
|
- meta-llama/Llama-3.3-70B-Instruct |
|
library_name: transformers |
|
tags: |
|
- mergekit |
|
- merge |
|
license: llama3.3 |
|
--- |
|
Part of a multi merge experiment. |
|
The idea behind it is to create 3 individual models: |
|
- Pathos: For ERP and uncensored NSFW content |
|
- Ethos: For prose and storytelling |
|
- Logos: For intelligence and awareness |
|
|
|
The three models above will then be combined into: |
|
- Kairos: The best of all three hopefully. |
|
|
|
I will be using differnet merge methods for these merges in an attempt to find the best combinations hence the Alpha, Beta and Delta tags you will see on each which represent different merge methods. |
|
# merge |
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
|
|
|
## Merge Details |
|
### Merge Method |
|
|
|
This model was merged using the [Linear DELLA](https://arxiv.org/abs/2406.11617) merge method using [meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) as a base. |
|
|
|
### Models Merged |
|
|
|
The following models were included in the merge: |
|
* [Sao10K/L3.3-70B-Euryale-v2.3](https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3) |
|
* [Sao10K/70B-L3.3-mhnnn-x1](https://huggingface.co/Sao10K/70B-L3.3-mhnnn-x1) |
|
* [EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1](https://huggingface.co/EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1) |
|
* [Doctor-Shotgun/L3.3-70B-Magnum-v4-SE](https://huggingface.co/Doctor-Shotgun/L3.3-70B-Magnum-v4-SE) |
|
* [Sao10K/70B-L3.3-Cirrus-x1](https://huggingface.co/Sao10K/70B-L3.3-Cirrus-x1) |
|
|
|
### Configuration |
|
|
|
The following YAML configuration was used to produce this model: |
|
|
|
```yaml |
|
models: |
|
- model: Doctor-Shotgun/L3.3-70B-Magnum-v4-SE |
|
parameters: |
|
weight: 0.20 |
|
density: 0.7 |
|
- model: Sao10K/70B-L3.3-mhnnn-x1 |
|
parameters: |
|
weight: 0.20 |
|
density: 0.7 |
|
- model: Sao10K/70B-L3.3-Cirrus-x1 |
|
parameters: |
|
weight: 0.20 |
|
density: 0.7 |
|
- model: EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1 |
|
parameters: |
|
weight: 0.20 |
|
density: 0.7 |
|
- model: Sao10K/L3.3-70B-Euryale-v2.3 |
|
parameters: |
|
weight: 0.20 |
|
density: 0.7 |
|
merge_method: della_linear |
|
base_model: meta-llama/Llama-3.3-70B-Instruct |
|
parameters: |
|
epsilon: 0.2 |
|
lambda: 1.1 |
|
out_dtype: bfloat16 |
|
tokenizer: |
|
source: Sao10K/L3.3-70B-Euryale-v2.3 |
|
``` |
|
|
