File size: 1,505 Bytes
5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 429c1fb 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 5a10e60 fbcd3b9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
---
library_name: transformers
license: llama3
base_model:
- nbeerbower/llama-3-Daredevil-Mahou-8B
datasets:
- flammenai/MahouMix-v1
---

# Mahou-1.3-llama3-8B
Mahou is our attempt to build a production-ready conversational/roleplay LLM.
Future versions will be released iteratively and finetuned from flammen.ai conversational data.
### License
This model is based on Meta Llama-3-8B and is governed by the [META LLAMA 3 COMMUNITY LICENSE AGREEMENT](LICENSE).
### Chat Format
This model has been trained to use ChatML format. Note the additional tokens in [tokenizer_config.json](tokenizer_config.json).
```
<|im_start|>system
{{system}}<|im_end|>
<|im_start|>{{char}}
{{message}}<|im_end|>
<|im_start|>{{user}}
{{message}}<|im_end|>
```
### Roleplay Format
- Speech without quotes.
- Actions in `*asterisks*`
```
*leans against wall cooly* so like, i just casted a super strong spell at magician academy today, not gonna lie, felt badass.
```
### ST Settings
1. Use ChatML for the Context Template.
2. Enable Instruct Mode.
3. Use the [Mahou preset](https://huggingface.co/datasets/flammenai/Mahou-ST-ChatML-Instruct/raw/main/Mahou.json).
4. Recommended: Add newline as a stopping string: `["\n"]`
### Method
Finetuned for 10 epochs using an A100 on Google Colab.
[Fine-tune Llama 3 with ORPO](https://huggingface.co/blog/mlabonne/orpo-llama-3) - [Maxime Labonne](https://huggingface.co/mlabonne) |