Update README.md
Browse files
README.md
CHANGED
|
@@ -77,6 +77,21 @@ Minotaur was fine-tuned from the base model MPT-7B, please refer to its model ca
|
|
| 77 |
|
| 78 |
## Benchmarks
|
| 79 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
|
| 82 |
## Examples - results may vary based on temperature and other settings
|
|
|
|
| 77 |
|
| 78 |
## Benchmarks
|
| 79 |
|
| 80 |
+
hf-causal-experimental (pretrained=openaccess-ai-collective/minotaur-13b), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
|
| 81 |
+
| Task |Version| Metric |Value | |Stderr|
|
| 82 |
+
|-------------|------:|--------|-----:|---|-----:|
|
| 83 |
+
|arc_challenge| 0|acc |0.4556|_ |0.0146|
|
| 84 |
+
| | |acc_norm|0.4795|_ |0.0146|
|
| 85 |
+
|arc_easy | 0|acc |0.7736|_ |0.0086|
|
| 86 |
+
| | |acc_norm|0.7298|_ |0.0091|
|
| 87 |
+
|boolq | 1|acc |0.8141|_ |0.0068|
|
| 88 |
+
|hellaswag | 0|acc |0.5893|_ |0.0049|
|
| 89 |
+
| | |acc_norm|0.7726|_ |0.0042|
|
| 90 |
+
|openbookqa | 0|acc |0.3280|_ |0.0210|
|
| 91 |
+
| | |acc_norm|0.4420|_ |0.0222|
|
| 92 |
+
|piqa | 0|acc |0.7992|_ |0.0093|
|
| 93 |
+
| | |acc_norm|0.7960|_ |0.0094|
|
| 94 |
+
|winogrande | 0|acc |0.7198|_ |0.0126|
|
| 95 |
|
| 96 |
|
| 97 |
## Examples - results may vary based on temperature and other settings
|