clefourrier HF staff commited on
Commit
b17e273
·
1 Parent(s): d03b4ca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -7,4 +7,23 @@ sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  pinned: false
8
  ---
9
 
10
+ This repository is a demo leaderboard template.
11
+ You can copy the leaderboard space and the two datasets (results and requests) to your org to get started with your own leaderboard!
12
+
13
+ The space does 3 things:
14
+ - stores users submissions, and sends them to the `requests` dataset
15
+ - reads the submissions depending on their status/date of creation, and launches evaluations through the `main_backend.py` file, using the Eleuther AI Harness. Results of running evaluations are then sent to `results`
16
+ - reads the results and displays them in a leaderboard.
17
+
18
+ To get started on your own leaderboard, you will need to edit 2 files:
19
+ - `src/envs.py` to define your own environment variable (like the org name in which this has been copied)
20
+ - `src/about.py` with the tasks and number of few_shots you want for your tasks
21
+
22
+ Once this is done, you need to edit the "fake results" file to fit the format of your tasks: in the sub dictionary `results`, replace task_name1 and metric_name by the correct values you defined in Tasks above.
23
+ ```
24
+ "results": {
25
+ "task_name1": {
26
+ "metric_name": 0
27
+ }
28
+ }
29
+ ```