Commit
·
008932c
1
Parent(s):
8a8a7c5
Update README.md
Browse files
README.md
CHANGED
@@ -61,7 +61,10 @@ print(scores)
|
|
61 |
## Evaluation
|
62 |
***
|
63 |
|
64 |
-
We evaluated
|
|
|
|
|
|
|
65 |
| | model | RP | MRR@10 | R@10 (↑) | R@20 | R@50 | R@100 |
|
66 |
|---:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------:|---------:|------------:|------------:|------------:|-------------:|
|
67 |
| 1 | **crossencoder-camembert-base-mmarcoFR** | 35.65 | 50.44 | 82.95 | 91.5 | 96.8 | 98.8 |
|
@@ -85,7 +88,7 @@ We trained the model on a single Tesla V100 GPU with 32GBs of memory during 10 e
|
|
85 |
|
86 |
#### Data
|
87 |
|
88 |
-
We used the French version of the [mMARCO](https://huggingface.co/datasets/unicamp-dl/mmarco) dataset to fine-tune
|
89 |
|
90 |
## Citation
|
91 |
***
|
|
|
61 |
## Evaluation
|
62 |
***
|
63 |
|
64 |
+
We evaluated the model on 500 random queries from the mMARCO-fr train set (which were excluded from training). Each of these queries has at least one relevant and up to 200 irrelevant passages.
|
65 |
+
|
66 |
+
Below, we compare the model performance with other cross-encoder models fine-tuned on the same dataset. We report the R-precision (RP), mean reciprocal rank (MRR), and recall at various cut-offs (R@k).
|
67 |
+
|
68 |
| | model | RP | MRR@10 | R@10 (↑) | R@20 | R@50 | R@100 |
|
69 |
|---:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------:|---------:|------------:|------------:|------------:|-------------:|
|
70 |
| 1 | **crossencoder-camembert-base-mmarcoFR** | 35.65 | 50.44 | 82.95 | 91.5 | 96.8 | 98.8 |
|
|
|
88 |
|
89 |
#### Data
|
90 |
|
91 |
+
We used the French version of the [mMARCO](https://huggingface.co/datasets/unicamp-dl/mmarco) dataset to fine-tune the model. mMARCO is a multi-lingual machine-translated version of the MS MARCO dataset, a popular large-scale IR dataset.
|
92 |
|
93 |
## Citation
|
94 |
***
|