README.md

---
tags:
- merge
- mergekit
- lazymergekit
- datatab/Yugo45-GPT
- FlexingD/yarn-mistral-7B-64k-instruct-alpaca-cleaned-origin
base_model:
- datatab/YugoGPT-Alpaca-v1
- FlexingD/yarn-mistral-7B-64k-instruct-alpaca-cleaned-origin
license: cc-by-4.0
datasets:
- datatab/alpaca-cleaned-serbian-full
language:
- sr
---

# Yugo45-GPT *(7b)

This **Yugo45-GPT (7b)** model has been fine-tuned on the Alpaca dataset using the **[gordicaleksa/YugoGPT](https://huggingface.co/gordicaleksa/YugoGPT/)** as the zero ground base model.

- **Finetune performed by**: datatab
- **License**: CC-BY-4.0
- **Original model author**: [gordicaleksa/YugoGPT](https://huggingface.co/gordicaleksa/YugoGPT/)

Yugo45-GPT is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [datatab/YugoGPT-Alpaca-v1](https://huggingface.co/datatab/YugoGPT-Alpaca-v1)
* [FlexingD/yarn-mistral-7B-64k-instruct-alpaca-cleaned-origin](https://huggingface.co/FlexingD/yarn-mistral-7B-64k-instruct-alpaca-cleaned-origin)

## 📌 Note

Special thanks for idea [**Stopwolf**](https://huggingface.co/Stopwolf) and this **X** post  [**@TheStopwolf**](https://twitter.com/TheStopwolf/status/1761350502212599890)

## 🧩 Configuration

```yaml
slices:
  - sources:
      - model: datatab/YugoGPT-Alpaca-v1
        layer_range: [0, 32]
      - model: FlexingD/yarn-mistral-7B-64k-instruct-alpaca-cleaned-origin
        layer_range: [0, 32]
merge_method: slerp
base_model: datatab/YugoGPT-Alpaca-v1
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
```
## 🏋🏼 Benchmarks
```python
# TBD
```

## 💻 Usage

```python
# TBD
```