allenai
/

OLMo-1B-0724-hf

Text Generation

Transformers

Safetensors

English

olmo

Model card Files Files and versions Community

OyvindTafjord commited on Jun 24, 2024

Commit

4951643

verified ·

1 Parent(s): 6e1cdb7

Update README.md

Browse files

Files changed (1) hide show

README.md +6 -6

README.md CHANGED Viewed

@@ -133,16 +133,16 @@ And for the 1B model:
 | task       | random | [StableLM 2 1.6b](https://huggingface.co/stabilityai/stablelm-2-1_6b)\* | [Pythia 1B](https://huggingface.co/EleutherAI/pythia-1b) | [TinyLlama 1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T) | OLMo 1B | **OLMo 1.7-1B** (ours) |
 | ------------- | ------ | ----------------- | --------- | -------------------------------------- | ------- | ---- |
-| arc_challenge | 25     | 43.81             | 33.11     | 34.78                                  | 34.45   | 36.5 |
-| arc_easy      | 25     | 63.68             | 50.18     | 53.16                                  | 58.07   | 55.3 |
 | boolq         | 50     | 76.6              | 61.8      | 64.6                                   | 60.7    | 67.5 |
-| copa          | 50     | 84                | 72        | 78                                     | 79      | 83.0 |
 | hellaswag     | 25     | 68.2              | 44.7      | 58.7                                   | 62.5    | 66.9 |
 | openbookqa    | 25     | 45.8              | 37.8      | 43.6                                   | 46.4    | 46.4 |
-| piqa          | 50     | 74                | 69.1      | 71.1                                   | 73.7    | 74.9 |
-| sciq          | 25     | 94.7              | 86        | 90.5                                   | 88.1    | 93.4 |
 | winogrande    | 50     | 64.9              | 53.3      | 58.9                                   | 58.9    | 61.4 |
-| Average       | 36.11  | 68.41             | 56.44     | 61.48                                  | 62.42   | 65.0 |
 \*Unlike OLMo, Pythia, and TinyLlama, StabilityAI has not disclosed yet the data StableLM was trained on, making comparisons with other efforts challenging.

 | task       | random | [StableLM 2 1.6b](https://huggingface.co/stabilityai/stablelm-2-1_6b)\* | [Pythia 1B](https://huggingface.co/EleutherAI/pythia-1b) | [TinyLlama 1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T) | OLMo 1B | **OLMo 1.7-1B** (ours) |
 | ------------- | ------ | ----------------- | --------- | -------------------------------------- | ------- | ---- |
+| arc_challenge | 25     | 43.8              | 33.1      | 34.8                                   | 34.5    | 36.5 |
+| arc_easy      | 25     | 63.7              | 50.2      | 53.2                                   | 58.1    | 55.3 |
 | boolq         | 50     | 76.6              | 61.8      | 64.6                                   | 60.7    | 67.5 |
+| copa          | 50     | 84.0              | 72.0      | 78.0                                   | 79.0    | 83.0 |
 | hellaswag     | 25     | 68.2              | 44.7      | 58.7                                   | 62.5    | 66.9 |
 | openbookqa    | 25     | 45.8              | 37.8      | 43.6                                   | 46.4    | 46.4 |
+| piqa          | 50     | 74.0              | 69.1      | 71.1                                   | 73.7    | 74.9 |
+| sciq          | 25     | 94.7              | 86.0      | 90.5                                   | 88.1    | 93.4 |
 | winogrande    | 50     | 64.9              | 53.3      | 58.9                                   | 58.9    | 61.4 |
+| Average       | 36.1   | 68.4              | 56.4      | 61.5                                   | 62.4    | 65.0 |
 \*Unlike OLMo, Pythia, and TinyLlama, StabilityAI has not disclosed yet the data StableLM was trained on, making comparisons with other efforts challenging.