|
|
--- |
|
|
title: MTEB Leaderboard |
|
|
emoji: 🥇 |
|
|
colorFrom: blue |
|
|
colorTo: indigo |
|
|
sdk: gradio |
|
|
sdk_version: 4.20.0 |
|
|
app_file: app.py |
|
|
pinned: false |
|
|
tags: |
|
|
- leaderboard |
|
|
startup_duration_timeout: 1h |
|
|
fullWidth: true |
|
|
--- |
|
|
|
|
|
## The MTEB Leaderboard repository |
|
|
|
|
|
This repository contains the code for pushing and updating the MTEB leaderboard daily. |
|
|
|
|
|
| Relevant Links | Decription | |
|
|
|------------------------------------------|------------------------------| |
|
|
| [mteb](https://github.com/embeddings-benchmark/mteb) | The implementation of the benchmark. Here you e.g. find the code to run your model on the benchmark. | |
|
|
| [leaderboard](https://huggingface.co/spaces/mteb/leaderboard) | The leaderboard itself, here you can view results of model run on MTEB. | |
|
|
| [results](https://github.com/embeddings-benchmark/results) | The results of MTEB is stored here. Though you can publish them to the leaderboard [adding](https://github.com/embeddings-benchmark/mteb/blob/main/docs/adding_a_model.md) the result to your model card. | |
|
|
|
|
|
## Developer setup |
|
|
|
|
|
To setup the repository: |
|
|
|
|
|
```bash |
|
|
git clone https://github.com/embeddings-benchmark/leaderboard.git |
|
|
cd leaderboard |
|
|
# install requirements |
|
|
pip install -r requirements.txt |
|
|
# fetch new results |
|
|
# python refresh.py |
|
|
# if you'd like to add results to previously cached models, you may have to remove these models in `EXTERNAL_MODEL_RESULTS.json` |
|
|
# you can also directly delete `EXTERNAL_MODEL_RESULTS.json` and it will recreate it (but be much slower) |
|
|
# run the leaderboard |
|
|
python app.py |
|
|
``` |
|
|
|
