---
base_model:
- Nitral-Archive/Virtuoso-Lite-chatmlified-10B_r16-ep1
- Nitral-Archive/NightWing3-10B-v0.1
library_name: transformers
tags:
- mergekit
- merge
license: other
language:
- en
---
# This has not yet been extensively tested.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/0QE2gG0eheTSto_iO-RY0.png)

# Base model: (Falcon3-10B-deepseekv3-distill)[[Virtuoso_Lite]](https://huggingface.co/arcee-ai/Virtuoso-Lite)

[IQ4 GGUF test quant here](https://huggingface.co/Nitrals-Quants/NightWing3_Virtuoso-10B-v0.2-IQ4_NL-GGUF)

# ST Presets [using nera's as a placeholder until i test further] [Here](https://huggingface.co/Nitral-AI/NightWing3_Virtuoso-10B-v0.2/tree/main/ST)
# Prompt format: ChatML
```
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```

### Models Merged:
* [Nitral-Archive/Virtuoso-Lite-chatmlified-10B_r16-ep1](https://huggingface.co/Nitral-Archive/Virtuoso-Lite-chatmlified-10B_r16-ep1)
* [Nitral-Archive/NightWing3-10B-v0.1](https://huggingface.co/Nitral-Archive/NightWing3-10B-v0.1)

### The following YAML configuration was used to produce this model:
```yaml
slices:
  - sources:
      - model: Nitral-Archive/Virtuoso-Lite-chatmlified-10B_r16-ep1
        layer_range: [0, 40]
      - model: Nitral-Archive/NightWing3-10B-v0.1
        layer_range: [0, 40]
merge_method: slerp
base_model: Nitral-Archive/Virtuoso-Lite-chatmlified-10B_r16-ep1
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.420
dtype: bfloat16

```
# Notes: The goal of this merge was to make use of both the falcon3-10B base model I trained earlier (nightwing3) and my more recent training run over Arcee's distillation of DeepSeekV3, which also falcon3-10B as a base (Virtuoso-Lite-chatmlified-10B_r16-ep1). Initially, I wasn't entirely satisfied with the results of either model on their own. However, with limited testing, this merged version appears to have smoothed out some of the rough edges present in the originals. Further evaluation is needed to fully assess its performance.