Update README.md
Browse files
README.md
CHANGED
@@ -13,8 +13,28 @@ tags:
|
|
13 |
Converted version of [Mixtral Instruct](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.
|
14 |
|
15 |
### Impact on performance
|
16 |
-
Impact of quantization on a set of models.
|
17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
|
19 |
### Prompt Pattern
|
20 |
Here is a reminder of the command pattern to interact with the model if the add_special_tokens option is disabled (otherwise, do not include the BOS symbol and space at the beginning of the sequence):
|
|
|
13 |
Converted version of [Mixtral Instruct](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.
|
14 |
|
15 |
### Impact on performance
|
16 |
+
Impact of quantization on a set of models.
|
17 |
+
|
18 |
+
Evaluation of the model was conducted using the PoLL (Pool of LLM) technique, assessing performance on **100 French questions** with scores aggregated from six evaluations
|
19 |
+
(two per evaluator). The evaluators included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet.
|
20 |
+
|
21 |
+
**Performance Scores (on a scale of 5):**
|
22 |
+
| Model | Score | # params |
|
23 |
+
|---------------------------------------------:|:-------:|:--------:|
|
24 |
+
| gpt-4o | 4.13 | N/A |
|
25 |
+
| mistralai/Mixtral-8x7B-Instruct-v0.1 | 3.71 | 46.7b |
|
26 |
+
| cmarkea/Mixtral-8x7B-Instruct-v0.1-4bit | 3.68 | 46.7b |
|
27 |
+
| gpt-3.5-turbo | 3.66 | 175b |
|
28 |
+
| TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ | 3.56 | 46.7b |
|
29 |
+
| mistralai/Mistral-7B-Instruct-v0.2 | 1.98 | 7.25b |
|
30 |
+
| cmarkea/bloomz-7b1-mt-sft-chat | 1.69 | 7.1b |
|
31 |
+
| cmarkea/bloomz-3b-dpo-chat | 1.68 | 3b |
|
32 |
+
| cmarkea/bloomz-3b-sft-chat | 1.51 | 3b |
|
33 |
+
| croissantllm/CroissantLLMChat-v0.1 | 1.19 | 1.3b |
|
34 |
+
| cmarkea/bloomz-560m-sft-chat | 1.04 | 0.56b |
|
35 |
+
| OpenLLM-France/Claire-Mistral-7B-0.1 | 0.38 | 7.25b |
|
36 |
+
|
37 |
+
The impact of quantization is negligible.
|
38 |
|
39 |
### Prompt Pattern
|
40 |
Here is a reminder of the command pattern to interact with the model if the add_special_tokens option is disabled (otherwise, do not include the BOS symbol and space at the beginning of the sequence):
|