README.md

---
base_model:
- SentientAGI/Dobby-Unhinged-Llama-3.3-70B
- Black-Ink-Guild/Pernicious_Prophecy_70B
- LatitudeGames/Wayfarer-Large-70B-Llama-3.3
- TheDrummer/Fallen-Llama-3.3-R1-70B-v1
- Nexesenex/Llama_3.x_70b_SmarTracks_V1.01
- Nexesenex/Llama_3.1_70b_TearDrops_V1.11
library_name: transformers
tags:
- mergekit
- merge
license: llama3.3
---
# about

After a lot of "lego merges" to experiment, let's start a basket merge series!
Base is the third version of Smarteaz, Smartracks, in which the R1 model is itself a merge between R1, R1 without chinese censorship, and R1 Fallen Llama.
That base has shown itself excellent to empower any model thrown at it. Nemotron and Tulu complete the mix.

My 5 favorite L3.3 (Negative Llama, EVA, Dobby, Fallen Llama ofc and Wayfarer) are included in submerges, starting with the well doted Permiscious Prophecy (Including a bit of Sao10K's Euryale 2.2 through the 70Blivion model).
Hermes and Tess are also included in submerges, in their abliterated version. Hermes has also its Gutemberg Doppel version.
Some abliterated or uncensored L3 are also wrapped in, like Lumitron Abliterated (including some NeverSleep work) or Creative Llama.

---
# benchs

Benchs are traded for creativity in this merge, so :
- PPL Wikitext Eng 512 : 3.54 (good)
- ARC-C : 59.20 (good)
- ARC-E : 80.70 (good also)

---
# merge

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Nexesenex/Llama_3.x_70b_SmarTracks_V1.01](https://huggingface.co/Nexesenex/Llama_3.x_70b_SmarTracks_V1.01) as a base.
- The "Smart base" of the model, a 3 levels merge-stock mix of Llama 3.3 abliterated finetuned (the root), Deepseek R1 Distill based Fallen Llama, Nemotron and Tulu.

### Models Merged

The following models were included in the merge:
* [SentientAGI/Dobby-Unhinged-Llama-3.3-70B](https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B)
- For its.. unhinged "personality traits".
* [Black-Ink-Guild/Pernicious_Prophecy_70B](https://huggingface.co/Black-Ink-Guild/Pernicious_Prophecy_70B)
- A balanced healed merge-stock steering with Eva (creativity), Negative Llama (debiasing), L3.1 Oblivion (general intelligence), Open-Bio (anatomy and medicine).
* [LatitudeGames/Wayfarer-Large-70B-Llama-3.3](https://huggingface.co/LatitudeGames/Wayfarer-Large-70B-Llama-3.3)
- To darken the model and the RP scenarios.
* [TheDrummer/Fallen-Llama-3.3-R1-70B-v1](https://huggingface.co/TheDrummer/Fallen-Llama-3.3-R1-70B-v1)
- To highlight and consolidate the R1 capabilities and spice-up/darken the model.
* [Nexesenex/Llama_3.1_70b_TearDrops_V1.11](https://huggingface.co/Nexesenex/Llama_3.1_70b_TearDrops_V1.11)
- A legacy 3.1 merge, led by Tess R1, including the Hermes based Gutemberg Doppel and an uncensored creative finetune.

### Configuration

The following YAML configuration was used to produce this model:

```yaml
merge_method: model_stock
models:
  - model: Nexesenex/Llama_3.1_70b_TearDrops_V1.11
    parameters:
      weight: 1.0
  - model: Black-Ink-Guild/Pernicious_Prophecy_70B
    parameters:
      weight: 1.0
  - model: SentientAGI/Dobby-Unhinged-Llama-3.3-70B
    parameters:
      weight: 1.0
  - model: TheDrummer/Fallen-Llama-3.3-R1-70B-v1
    parameters:
      weight: 1.0
  - model: LatitudeGames/Wayfarer-Large-70B-Llama-3.3
    parameters:
      weight: 1.0
base_model: Nexesenex/Llama_3.x_70b_SmarTracks_V1.01
dtype: bfloat16
out_dtype: bfloat16
parameters:
  int8_mask: true
  normalize: true
  rescale: false
chat_template: auto
tokenizer:
  source: union
```