switch to flat inflection benchmark

src/about.py

@@ -14,9 +14,7 @@ class Tasks(Enum):
     # task_key in the json file, metric_key in the json file, name to display in the leaderboard 
     task0 = Task("icelandic_winogrande_stringmatch", "exact_match,get-answer", "WinoGrande-IS (3-shot)")
     task1 = Task("icelandic_sentences_ged_stringmatch", "exact_match,get-answer", "GED")
-    task2 = Task("icelandic_inflection_easy", "json_metric,get-answer", "Inflection (common)")
-    task3 = Task("icelandic_inflection_medium", "json_metric,get-answer", "Inflection (uncommon)")
-    task4 = Task("icelandic_inflection_hard", "json_metric,get-answer", "Inflection (rare)")
+    task2 = Task("icelandic_inflection_all", "exact_match,get-answer", "Inflection (1-shot)")
     task5 = Task("icelandic_belebele", "exact_match,get-answer", "Belebele (IS)")
     task6 = Task("icelandic_arc_challenge", "exact_match,get-answer", "ARC-Challenge-IS")
 
@@ -54,13 +52,9 @@ This is a benchmark for binary sentence-level Icelandic grammatical error detect
 Each example consists of a sentence that may contain one or more grammatical errors, and the task is to predict whether the sentence contains an error.
 - Link to dataset: https://huggingface.co/datasets/mideind/icelandic-sentences-gec
 
-### Inflection benchmarks
-The inflection benchmarks test the model's ability to generate inflected forms of Icelandic adjective-noun pairs. They are divided into three levels of difficulty by
-commonness: common (100 examples), uncommon (100 examples), and rare (100 examples). The model gets a point for an example if it generates error-free json with the 
-correct inflected forms in all cases, singular and plural.
-- Link to dataset (common): https://huggingface.co/datasets/mideind/icelandic-inflection-easy
-- Link to dataset (uncommon): https://huggingface.co/datasets/mideind/icelandic-inflection-medium
-- Link to dataset (rare): https://huggingface.co/datasets/mideind/icelandic-inflection-hard
+### Inflection benchmark
+The inflection benchmark tests models' ability to generate inflected forms of 300 Icelandic adjective-noun pairs for all four cases, singular and plural.
+- Link to dataset: https://huggingface.co/datasets/mideind/icelandic-inflection-all-flat
 
 ### Belebele (IS)
 This is the Icelandic subset (900 examples) of the Belebele benchmark, a multiple-choice reading comprehension task. The task is to answer questions about a given passage.