merged

This is a merge of pre-trained language models created using mergekit.

You can use ChatML format.

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

base_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
dtype: bfloat16
merge_method: slerp
parameters:
  t:
  - filter: self_attn
    value: [0.0, 0.5, 0.3, 0.7, 1.0]
  - filter: mlp
    value: [1.0, 0.5, 0.7, 0.3, 0.0]
  - value: 0.5
slices:
- sources:
  - layer_range: [0, 28]
    model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
  - layer_range: [0, 28]
    model: Cran-May/T.E-8.1

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 27.34
IFEval (0-Shot) 69.25
BBH (3-Shot) 36.50
MATH Lvl 5 (4-Shot) 0.83
GPQA (0-shot) 8.17
MuSR (0-shot) 10.98
MMLU-PRO (5-shot) 38.28

Buy Me A Coffee

Downloads last month
59
Safetensors
Model size
7.62B params
Tensor type
BF16
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for suayptalha/Rombos-2.5-T.E-8.1

Merge model
this model
Quantizations
3 models

Collection including suayptalha/Rombos-2.5-T.E-8.1

Evaluation results