--- language: en tags: - medembed - medical-embedding - clinical-embedding - information-retrieval - sentence-transformers - mteb license: apache-2.0 datasets: - MedicalQARetrieval - NFCorpus - PublicHealthQA - TRECCOVID - ArguAna metrics: - nDCG - MAP - Recall - Precision - MRR base_model: - BAAI/bge-large-en-v1.5 model-index: - name: abhinand/MedEmbed-large-v0.1 results: - dataset: config: default name: MTEB CMedQAv2-reranking (default) revision: 23d186750531a14a0357ca22cd92d712fd512ea0 split: test type: C-MTEB/CMedQAv2-reranking metrics: - type: map value: 18.3837 - type: mrr value: 21.1936 - type: nAUC_map_max value: 34.185 - type: nAUC_map_std value: 17.8023 - type: nAUC_map_diff1 value: 29.8803 - type: nAUC_mrr_max value: 33.5715 - type: nAUC_mrr_std value: 18.198700000000002 - type: nAUC_mrr_diff1 value: 28.1289 - type: main_score value: 18.3837 task: type: Reranking - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: all type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 60.775 - type: ndcg_at_3 value: 55.233 - type: ndcg_at_5 value: 53.738 - type: ndcg_at_10 value: 52.898999999999994 - type: ndcg_at_20 value: 53.64 - type: ndcg_at_100 value: 60.18 - type: ndcg_at_1000 value: 68.304 - type: map_at_1 value: 9.185 - type: map_at_3 value: 16.432 - type: map_at_5 value: 20.197000000000003 - type: map_at_10 value: 25.278 - type: map_at_20 value: 29.92 - type: map_at_100 value: 38.107 - type: map_at_1000 value: 41.919000000000004 - type: recall_at_1 value: 9.185 - type: recall_at_3 value: 17.996000000000002 - type: recall_at_5 value: 23.671 - type: recall_at_10 value: 32.817 - type: recall_at_20 value: 43.415 - type: recall_at_100 value: 70.08 - type: recall_at_1000 value: 94.03099999999999 - type: precision_at_1 value: 71.95 - type: precision_at_3 value: 59.567 - type: precision_at_5 value: 53.480000000000004 - type: precision_at_10 value: 44.025 - type: precision_at_20 value: 35.147 - type: precision_at_100 value: 16.997 - type: precision_at_1000 value: 3.399 - type: mrr_at_1 value: 71.95 - type: mrr_at_3 value: 79.4917 - type: mrr_at_5 value: 80.5892 - type: mrr_at_10 value: 80.9957 - type: mrr_at_20 value: 81.1267 - type: mrr_at_100 value: 81.164 - type: mrr_at_1000 value: 81.1657 - type: nauc_ndcg_at_1_max value: 38.9364 - type: nauc_ndcg_at_1_std value: 13.528200000000002 - type: nauc_ndcg_at_1_diff1 value: 40.0184 - type: nauc_ndcg_at_3_max value: 40.0247 - type: nauc_ndcg_at_3_std value: 16.4182 - type: nauc_ndcg_at_3_diff1 value: 27.328000000000003 - type: nauc_ndcg_at_5_max value: 38.8812 - type: nauc_ndcg_at_5_std value: 16.114700000000003 - type: nauc_ndcg_at_5_diff1 value: 26.082300000000004 - type: nauc_ndcg_at_10_max value: 38.0527 - type: nauc_ndcg_at_10_std value: 16.9198 - type: nauc_ndcg_at_10_diff1 value: 25.3401 - type: nauc_ndcg_at_20_max value: 37.3736 - type: nauc_ndcg_at_20_std value: 16.9378 - type: nauc_ndcg_at_20_diff1 value: 26.0711 - type: nauc_ndcg_at_100_max value: 40.096399999999996 - type: nauc_ndcg_at_100_std value: 23.7415 - type: nauc_ndcg_at_100_diff1 value: 26.6824 - type: nauc_ndcg_at_1000_max value: 45.1188 - type: nauc_ndcg_at_1000_std value: 31.591200000000004 - type: nauc_ndcg_at_1000_diff1 value: 25.790200000000002 - type: nauc_map_at_1_max value: 6.683699999999999 - type: nauc_map_at_1_std value: -12.8402 - type: nauc_map_at_1_diff1 value: 42.0279 - type: nauc_map_at_3_max value: 13.3802 - type: nauc_map_at_3_std value: -7.4277999999999995 - type: nauc_map_at_3_diff1 value: 31.4791 - type: nauc_map_at_5_max value: 16.2095 - type: nauc_map_at_5_std value: -4.0392 - type: nauc_map_at_5_diff1 value: 28.5612 - type: nauc_map_at_10_max value: 20.433899999999998 - type: nauc_map_at_10_std value: 0.5703 - type: nauc_map_at_10_diff1 value: 24.7802 - type: nauc_map_at_20_max value: 23.9891 - type: nauc_map_at_20_std value: 5.1846000000000005 - type: nauc_map_at_20_diff1 value: 23.117099999999997 - type: nauc_map_at_100_max value: 30.757299999999997 - type: nauc_map_at_100_std value: 17.927599999999998 - type: nauc_map_at_100_diff1 value: 19.872500000000002 - type: nauc_map_at_1000_max value: 33.2345 - type: nauc_map_at_1000_std value: 21.9999 - type: nauc_map_at_1000_diff1 value: 17.2405 - type: nauc_recall_at_1_max value: 6.683699999999999 - type: nauc_recall_at_1_std value: -12.8402 - type: nauc_recall_at_1_diff1 value: 42.0279 - type: nauc_recall_at_3_max value: 10.1747 - type: nauc_recall_at_3_std value: -8.1576 - type: nauc_recall_at_3_diff1 value: 28.4653 - type: nauc_recall_at_5_max value: 10.7604 - type: nauc_recall_at_5_std value: -5.2583 - type: nauc_recall_at_5_diff1 value: 24.9742 - type: nauc_recall_at_10_max value: 12.6847 - type: nauc_recall_at_10_std value: -2.3404000000000003 - type: nauc_recall_at_10_diff1 value: 20.872 - type: nauc_recall_at_20_max value: 14.197399999999998 - type: nauc_recall_at_20_std value: 1.0598 - type: nauc_recall_at_20_diff1 value: 19.2151 - type: nauc_recall_at_100_max value: 19.700400000000002 - type: nauc_recall_at_100_std value: 18.8582 - type: nauc_recall_at_100_diff1 value: 16.723399999999998 - type: nauc_recall_at_1000_max value: 37.7058 - type: nauc_recall_at_1000_std value: 59.268299999999996 - type: nauc_recall_at_1000_diff1 value: 18.7504 - type: nauc_precision_at_1_max value: 45.151599999999995 - type: nauc_precision_at_1_std value: 19.1001 - type: nauc_precision_at_1_diff1 value: 38.0115 - type: nauc_precision_at_3_max value: 37.2618 - type: nauc_precision_at_3_std value: 24.4577 - type: nauc_precision_at_3_diff1 value: 3.0456 - type: nauc_precision_at_5_max value: 33.4383 - type: nauc_precision_at_5_std value: 26.4563 - type: nauc_precision_at_5_diff1 value: -4.6004000000000005 - type: nauc_precision_at_10_max value: 28.6613 - type: nauc_precision_at_10_std value: 28.3258 - type: nauc_precision_at_10_diff1 value: -11.7935 - type: nauc_precision_at_20_max value: 22.605700000000002 - type: nauc_precision_at_20_std value: 27.8988 - type: nauc_precision_at_20_diff1 value: -14.3591 - type: nauc_precision_at_100_max value: 12.0733 - type: nauc_precision_at_100_std value: 22.561500000000002 - type: nauc_precision_at_100_diff1 value: -17.0289 - type: nauc_precision_at_1000_max value: 3.4652000000000003 - type: nauc_precision_at_1000_std value: 3.819 - type: nauc_precision_at_1000_diff1 value: -19.627200000000002 - type: nauc_mrr_at_1_max value: 45.151599999999995 - type: nauc_mrr_at_1_std value: 19.1001 - type: nauc_mrr_at_1_diff1 value: 38.0115 - type: nauc_mrr_at_3_max value: 48.4709 - type: nauc_mrr_at_3_std value: 25.6162 - type: nauc_mrr_at_3_diff1 value: 37.5255 - type: nauc_mrr_at_5_max value: 48.5705 - type: nauc_mrr_at_5_std value: 24.909 - type: nauc_mrr_at_5_diff1 value: 37.693799999999996 - type: nauc_mrr_at_10_max value: 48.4052 - type: nauc_mrr_at_10_std value: 24.4063 - type: nauc_mrr_at_10_diff1 value: 37.684400000000004 - type: nauc_mrr_at_20_max value: 48.2709 - type: nauc_mrr_at_20_std value: 24.272299999999998 - type: nauc_mrr_at_20_diff1 value: 37.737300000000005 - type: nauc_mrr_at_100_max value: 48.229499999999994 - type: nauc_mrr_at_100_std value: 24.2469 - type: nauc_mrr_at_100_diff1 value: 37.723600000000005 - type: nauc_mrr_at_1000_max value: 48.2309 - type: nauc_mrr_at_1000_std value: 24.2437 - type: nauc_mrr_at_1000_diff1 value: 37.7235 - type: main_score value: 52.898999999999994 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: all type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 15.9 - type: ndcg_at_3 value: 15.412999999999998 - type: ndcg_at_5 value: 15.42 - type: ndcg_at_10 value: 16.145 - type: ndcg_at_20 value: 17.386 - type: ndcg_at_100 value: 22.116 - type: ndcg_at_1000 value: 30.42 - type: map_at_1 value: 2.689 - type: map_at_3 value: 4.891 - type: map_at_5 value: 5.998 - type: map_at_10 value: 7.502000000000001 - type: map_at_20 value: 8.784 - type: map_at_100 value: 11.08 - type: map_at_1000 value: 12.662999999999998 - type: recall_at_1 value: 2.689 - type: recall_at_3 value: 6.002 - type: recall_at_5 value: 8.187 - type: recall_at_10 value: 12.167 - type: recall_at_20 value: 16.824 - type: recall_at_100 value: 31.608999999999998 - type: recall_at_1000 value: 58.023 - type: precision_at_1 value: 20.200000000000003 - type: precision_at_3 value: 17.45 - type: precision_at_5 value: 15.85 - type: precision_at_10 value: 13.645 - type: precision_at_20 value: 11.285 - type: precision_at_100 value: 6.265 - type: precision_at_1000 value: 1.7950000000000002 - type: mrr_at_1 value: 20.200000000000003 - type: mrr_at_3 value: 25.9583 - type: mrr_at_5 value: 27.450799999999997 - type: mrr_at_10 value: 28.6658 - type: mrr_at_20 value: 29.2657 - type: mrr_at_100 value: 29.652 - type: mrr_at_1000 value: 29.724 - type: nauc_ndcg_at_1_max value: 34.4333 - type: nauc_ndcg_at_1_std value: 37.9045 - type: nauc_ndcg_at_1_diff1 value: 13.5962 - type: nauc_ndcg_at_3_max value: 40.9522 - type: nauc_ndcg_at_3_std value: 43.265 - type: nauc_ndcg_at_3_diff1 value: 13.858699999999999 - type: nauc_ndcg_at_5_max value: 42.3408 - type: nauc_ndcg_at_5_std value: 45.1913 - type: nauc_ndcg_at_5_diff1 value: 14.362400000000001 - type: nauc_ndcg_at_10_max value: 43.697399999999995 - type: nauc_ndcg_at_10_std value: 47.9172 - type: nauc_ndcg_at_10_diff1 value: 14.7778 - type: nauc_ndcg_at_20_max value: 45.8538 - type: nauc_ndcg_at_20_std value: 50.7075 - type: nauc_ndcg_at_20_diff1 value: 14.7758 - type: nauc_ndcg_at_100_max value: 48.2383 - type: nauc_ndcg_at_100_std value: 56.974000000000004 - type: nauc_ndcg_at_100_diff1 value: 13.706399999999999 - type: nauc_ndcg_at_1000_max value: 52.815599999999996 - type: nauc_ndcg_at_1000_std value: 61.815200000000004 - type: nauc_ndcg_at_1000_diff1 value: 11.358500000000001 - type: nauc_map_at_1_max value: 32.1329 - type: nauc_map_at_1_std value: 30.017300000000002 - type: nauc_map_at_1_diff1 value: 33.417 - type: nauc_map_at_3_max value: 36.6575 - type: nauc_map_at_3_std value: 34.436099999999996 - type: nauc_map_at_3_diff1 value: 28.370099999999997 - type: nauc_map_at_5_max value: 37.6872 - type: nauc_map_at_5_std value: 37.303599999999996 - type: nauc_map_at_5_diff1 value: 25.6251 - type: nauc_map_at_10_max value: 39.6209 - type: nauc_map_at_10_std value: 41.5154 - type: nauc_map_at_10_diff1 value: 21.7045 - type: nauc_map_at_20_max value: 41.7603 - type: nauc_map_at_20_std value: 45.4034 - type: nauc_map_at_20_diff1 value: 19.9809 - type: nauc_map_at_100_max value: 45.1746 - type: nauc_map_at_100_std value: 52.485899999999994 - type: nauc_map_at_100_diff1 value: 17.2712 - type: nauc_map_at_1000_max value: 47.978 - type: nauc_map_at_1000_std value: 55.4423 - type: nauc_map_at_1000_diff1 value: 15.770999999999999 - type: nauc_recall_at_1_max value: 32.1329 - type: nauc_recall_at_1_std value: 30.017300000000002 - type: nauc_recall_at_1_diff1 value: 33.417 - type: nauc_recall_at_3_max value: 34.0411 - type: nauc_recall_at_3_std value: 32.08 - type: nauc_recall_at_3_diff1 value: 27.1762 - type: nauc_recall_at_5_max value: 33.7353 - type: nauc_recall_at_5_std value: 35.6551 - type: nauc_recall_at_5_diff1 value: 23.2104 - type: nauc_recall_at_10_max value: 34.0794 - type: nauc_recall_at_10_std value: 39.3103 - type: nauc_recall_at_10_diff1 value: 16.700499999999998 - type: nauc_recall_at_20_max value: 36.025 - type: nauc_recall_at_20_std value: 43.8139 - type: nauc_recall_at_20_diff1 value: 14.8745 - type: nauc_recall_at_100_max value: 40.477999999999994 - type: nauc_recall_at_100_std value: 55.299 - type: nauc_recall_at_100_diff1 value: 12.0134 - type: nauc_recall_at_1000_max value: 50.88 - type: nauc_recall_at_1000_std value: 69.1237 - type: nauc_recall_at_1000_diff1 value: 7.1673 - type: nauc_precision_at_1_max value: 36.3741 - type: nauc_precision_at_1_std value: 40.3943 - type: nauc_precision_at_1_diff1 value: 11.9572 - type: nauc_precision_at_3_max value: 44.1485 - type: nauc_precision_at_3_std value: 47.2489 - type: nauc_precision_at_3_diff1 value: 7.3660000000000005 - type: nauc_precision_at_5_max value: 44.5096 - type: nauc_precision_at_5_std value: 49.1423 - type: nauc_precision_at_5_diff1 value: 5.0777 - type: nauc_precision_at_10_max value: 45.307700000000004 - type: nauc_precision_at_10_std value: 51.953700000000005 - type: nauc_precision_at_10_diff1 value: 3.1449 - type: nauc_precision_at_20_max value: 47.2433 - type: nauc_precision_at_20_std value: 53.367200000000004 - type: nauc_precision_at_20_diff1 value: 0.7057 - type: nauc_precision_at_100_max value: 44.3203 - type: nauc_precision_at_100_std value: 49.472 - type: nauc_precision_at_100_diff1 value: -2.0434 - type: nauc_precision_at_1000_max value: 37.7422 - type: nauc_precision_at_1000_std value: 27.3935 - type: nauc_precision_at_1000_diff1 value: -3.1565000000000003 - type: nauc_mrr_at_1_max value: 36.3741 - type: nauc_mrr_at_1_std value: 40.3943 - type: nauc_mrr_at_1_diff1 value: 11.9572 - type: nauc_mrr_at_3_max value: 39.006099999999996 - type: nauc_mrr_at_3_std value: 42.753 - type: nauc_mrr_at_3_diff1 value: 11.0316 - type: nauc_mrr_at_5_max value: 39.2901 - type: nauc_mrr_at_5_std value: 43.4405 - type: nauc_mrr_at_5_diff1 value: 10.519499999999999 - type: nauc_mrr_at_10_max value: 40.2703 - type: nauc_mrr_at_10_std value: 44.372699999999995 - type: nauc_mrr_at_10_diff1 value: 10.6792 - type: nauc_mrr_at_20_max value: 40.605999999999995 - type: nauc_mrr_at_20_std value: 44.630199999999995 - type: nauc_mrr_at_20_diff1 value: 10.5727 - type: nauc_mrr_at_100_max value: 40.6811 - type: nauc_mrr_at_100_std value: 44.646 - type: nauc_mrr_at_100_diff1 value: 10.6126 - type: nauc_mrr_at_1000_max value: 40.6568 - type: nauc_mrr_at_1000_std value: 44.624399999999994 - type: nauc_mrr_at_1000_diff1 value: 10.6075 - type: main_score value: 16.145 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: all type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 24.9 - type: ndcg_at_3 value: 23.435 - type: ndcg_at_5 value: 23.462 - type: ndcg_at_10 value: 24.151 - type: ndcg_at_20 value: 25.171 - type: ndcg_at_100 value: 30.714999999999996 - type: ndcg_at_1000 value: 39.26 - type: map_at_1 value: 3.987 - type: map_at_3 value: 7.101 - type: map_at_5 value: 8.752 - type: map_at_10 value: 11.075 - type: map_at_20 value: 12.964 - type: map_at_100 value: 16.424 - type: map_at_1000 value: 18.472 - type: recall_at_1 value: 3.987 - type: recall_at_3 value: 8.290000000000001 - type: recall_at_5 value: 11.407 - type: recall_at_10 value: 16.906 - type: recall_at_20 value: 22.766000000000002 - type: recall_at_100 value: 40.918 - type: recall_at_1000 value: 67.77 - type: precision_at_1 value: 31.5 - type: precision_at_3 value: 26.533 - type: precision_at_5 value: 24.19 - type: precision_at_10 value: 20.59 - type: precision_at_20 value: 16.54 - type: precision_at_100 value: 8.725 - type: precision_at_1000 value: 2.2159999999999997 - type: mrr_at_1 value: 31.5 - type: mrr_at_3 value: 38.5667 - type: mrr_at_5 value: 40.2692 - type: mrr_at_10 value: 41.4221 - type: mrr_at_20 value: 41.9627 - type: mrr_at_100 value: 42.3045 - type: mrr_at_1000 value: 42.3438 - type: nauc_ndcg_at_1_max value: 41.2346 - type: nauc_ndcg_at_1_std value: 34.096399999999996 - type: nauc_ndcg_at_1_diff1 value: 23.3773 - type: nauc_ndcg_at_3_max value: 48.4194 - type: nauc_ndcg_at_3_std value: 38.5582 - type: nauc_ndcg_at_3_diff1 value: 20.044999999999998 - type: nauc_ndcg_at_5_max value: 50.6073 - type: nauc_ndcg_at_5_std value: 41.7937 - type: nauc_ndcg_at_5_diff1 value: 19.1486 - type: nauc_ndcg_at_10_max value: 52.3725 - type: nauc_ndcg_at_10_std value: 44.8701 - type: nauc_ndcg_at_10_diff1 value: 19.7716 - type: nauc_ndcg_at_20_max value: 52.9589 - type: nauc_ndcg_at_20_std value: 47.6835 - type: nauc_ndcg_at_20_diff1 value: 19.6519 - type: nauc_ndcg_at_100_max value: 57.541799999999995 - type: nauc_ndcg_at_100_std value: 56.922799999999995 - type: nauc_ndcg_at_100_diff1 value: 19.5411 - type: nauc_ndcg_at_1000_max value: 60.6206 - type: nauc_ndcg_at_1000_std value: 61.526 - type: nauc_ndcg_at_1000_diff1 value: 17.1735 - type: nauc_map_at_1_max value: 32.4122 - type: nauc_map_at_1_std value: 21.851499999999998 - type: nauc_map_at_1_diff1 value: 42.2724 - type: nauc_map_at_3_max value: 36.2169 - type: nauc_map_at_3_std value: 26.3249 - type: nauc_map_at_3_diff1 value: 32.3562 - type: nauc_map_at_5_max value: 37.9852 - type: nauc_map_at_5_std value: 29.6092 - type: nauc_map_at_5_diff1 value: 27.9065 - type: nauc_map_at_10_max value: 41.227000000000004 - type: nauc_map_at_10_std value: 34.2656 - type: nauc_map_at_10_diff1 value: 24.9007 - type: nauc_map_at_20_max value: 44.288 - type: nauc_map_at_20_std value: 39.033 - type: nauc_map_at_20_diff1 value: 23.0301 - type: nauc_map_at_100_max value: 50.3709 - type: nauc_map_at_100_std value: 48.2427 - type: nauc_map_at_100_diff1 value: 20.3268 - type: nauc_map_at_1000_max value: 52.983599999999996 - type: nauc_map_at_1000_std value: 51.302899999999994 - type: nauc_map_at_1000_diff1 value: 18.7497 - type: nauc_recall_at_1_max value: 32.4122 - type: nauc_recall_at_1_std value: 21.851499999999998 - type: nauc_recall_at_1_diff1 value: 42.2724 - type: nauc_recall_at_3_max value: 35.6626 - type: nauc_recall_at_3_std value: 26.583000000000002 - type: nauc_recall_at_3_diff1 value: 28.9598 - type: nauc_recall_at_5_max value: 35.6704 - type: nauc_recall_at_5_std value: 29.0846 - type: nauc_recall_at_5_diff1 value: 23.8078 - type: nauc_recall_at_10_max value: 38.3994 - type: nauc_recall_at_10_std value: 34.0908 - type: nauc_recall_at_10_diff1 value: 21.0836 - type: nauc_recall_at_20_max value: 40.962900000000005 - type: nauc_recall_at_20_std value: 40.4136 - type: nauc_recall_at_20_diff1 value: 18.5189 - type: nauc_recall_at_100_max value: 51.4476 - type: nauc_recall_at_100_std value: 58.6901 - type: nauc_recall_at_100_diff1 value: 16.724700000000002 - type: nauc_recall_at_1000_max value: 65.0535 - type: nauc_recall_at_1000_std value: 77.2842 - type: nauc_recall_at_1000_diff1 value: 11.8624 - type: nauc_precision_at_1_max value: 43.3958 - type: nauc_precision_at_1_std value: 37.1612 - type: nauc_precision_at_1_diff1 value: 20.409399999999998 - type: nauc_precision_at_3_max value: 49.9639 - type: nauc_precision_at_3_std value: 41.6999 - type: nauc_precision_at_3_diff1 value: 10.554 - type: nauc_precision_at_5_max value: 51.3906 - type: nauc_precision_at_5_std value: 45.5246 - type: nauc_precision_at_5_diff1 value: 6.8306000000000004 - type: nauc_precision_at_10_max value: 51.8536 - type: nauc_precision_at_10_std value: 47.8267 - type: nauc_precision_at_10_diff1 value: 4.649699999999999 - type: nauc_precision_at_20_max value: 49.1757 - type: nauc_precision_at_20_std value: 48.0193 - type: nauc_precision_at_20_diff1 value: 1.7838 - type: nauc_precision_at_100_max value: 41.7249 - type: nauc_precision_at_100_std value: 42.2626 - type: nauc_precision_at_100_diff1 value: -3.2152 - type: nauc_precision_at_1000_max value: 26.6144 - type: nauc_precision_at_1000_std value: 16.6642 - type: nauc_precision_at_1000_diff1 value: -6.8772 - type: nauc_mrr_at_1_max value: 43.3958 - type: nauc_mrr_at_1_std value: 37.1612 - type: nauc_mrr_at_1_diff1 value: 20.409399999999998 - type: nauc_mrr_at_3_max value: 47.9059 - type: nauc_mrr_at_3_std value: 40.9064 - type: nauc_mrr_at_3_diff1 value: 18.2671 - type: nauc_mrr_at_5_max value: 48.7106 - type: nauc_mrr_at_5_std value: 42.2268 - type: nauc_mrr_at_5_diff1 value: 18.875600000000002 - type: nauc_mrr_at_10_max value: 49.0714 - type: nauc_mrr_at_10_std value: 42.6807 - type: nauc_mrr_at_10_diff1 value: 18.8448 - type: nauc_mrr_at_20_max value: 49.113600000000005 - type: nauc_mrr_at_20_std value: 42.771 - type: nauc_mrr_at_20_diff1 value: 18.6008 - type: nauc_mrr_at_100_max value: 49.1187 - type: nauc_mrr_at_100_std value: 42.795899999999996 - type: nauc_mrr_at_100_diff1 value: 18.7082 - type: nauc_mrr_at_1000_max value: 49.0966 - type: nauc_mrr_at_1000_std value: 42.769600000000004 - type: nauc_mrr_at_1000_diff1 value: 18.7057 - type: main_score value: 24.151 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: dentistry_and_oral_health type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 63.74999999999999 - type: ndcg_at_3 value: 56.45400000000001 - type: ndcg_at_5 value: 53.888999999999996 - type: ndcg_at_10 value: 52.470000000000006 - type: ndcg_at_20 value: 51.661 - type: ndcg_at_100 value: 57.528 - type: ndcg_at_1000 value: 69.085 - type: map_at_1 value: 9.146 - type: map_at_3 value: 15.024999999999999 - type: map_at_5 value: 17.595 - type: map_at_10 value: 21.502 - type: map_at_20 value: 25.112000000000002 - type: map_at_100 value: 33.629999999999995 - type: map_at_1000 value: 38.818000000000005 - type: recall_at_1 value: 9.146 - type: recall_at_3 value: 17.03 - type: recall_at_5 value: 21.325 - type: recall_at_10 value: 28.424 - type: recall_at_20 value: 36.464 - type: recall_at_100 value: 64.31099999999999 - type: recall_at_1000 value: 95.343 - type: precision_at_1 value: 74.0 - type: precision_at_3 value: 60.167 - type: precision_at_5 value: 53.300000000000004 - type: precision_at_10 value: 44.3 - type: precision_at_20 value: 35.25 - type: precision_at_100 value: 18.404999999999998 - type: precision_at_1000 value: 3.721 - type: mrr_at_1 value: 74.0 - type: mrr_at_3 value: 82.08330000000001 - type: mrr_at_5 value: 83.4833 - type: mrr_at_10 value: 83.66029999999999 - type: mrr_at_20 value: 83.66029999999999 - type: mrr_at_100 value: 83.69290000000001 - type: mrr_at_1000 value: 83.69290000000001 - type: nauc_ndcg_at_1_max value: 44.2782 - type: nauc_ndcg_at_1_std value: -5.9135 - type: nauc_ndcg_at_1_diff1 value: 40.12 - type: nauc_ndcg_at_3_max value: 35.1497 - type: nauc_ndcg_at_3_std value: -3.2272000000000003 - type: nauc_ndcg_at_3_diff1 value: 23.9014 - type: nauc_ndcg_at_5_max value: 29.4267 - type: nauc_ndcg_at_5_std value: -9.1058 - type: nauc_ndcg_at_5_diff1 value: 24.3146 - type: nauc_ndcg_at_10_max value: 27.178600000000003 - type: nauc_ndcg_at_10_std value: -11.1484 - type: nauc_ndcg_at_10_diff1 value: 23.947599999999998 - type: nauc_ndcg_at_20_max value: 23.180400000000002 - type: nauc_ndcg_at_20_std value: -11.7635 - type: nauc_ndcg_at_20_diff1 value: 25.630999999999997 - type: nauc_ndcg_at_100_max value: 27.236300000000004 - type: nauc_ndcg_at_100_std value: -11.0596 - type: nauc_ndcg_at_100_diff1 value: 29.851699999999997 - type: nauc_ndcg_at_1000_max value: 38.7358 - type: nauc_ndcg_at_1000_std value: 12.8903 - type: nauc_ndcg_at_1000_diff1 value: 25.6108 - type: nauc_map_at_1_max value: 8.0922 - type: nauc_map_at_1_std value: -31.6799 - type: nauc_map_at_1_diff1 value: 42.1566 - type: nauc_map_at_3_max value: 7.6643 - type: nauc_map_at_3_std value: -32.679399999999994 - type: nauc_map_at_3_diff1 value: 29.1437 - type: nauc_map_at_5_max value: 8.3421 - type: nauc_map_at_5_std value: -32.3957 - type: nauc_map_at_5_diff1 value: 26.3427 - type: nauc_map_at_10_max value: 8.5915 - type: nauc_map_at_10_std value: -31.4049 - type: nauc_map_at_10_diff1 value: 23.3537 - type: nauc_map_at_20_max value: 11.0618 - type: nauc_map_at_20_std value: -28.0167 - type: nauc_map_at_20_diff1 value: 23.278399999999998 - type: nauc_map_at_100_max value: 18.6807 - type: nauc_map_at_100_std value: -10.8842 - type: nauc_map_at_100_diff1 value: 21.370900000000002 - type: nauc_map_at_1000_max value: 20.6202 - type: nauc_map_at_1000_std value: 1.8843999999999999 - type: nauc_map_at_1000_diff1 value: 16.491500000000002 - type: nauc_recall_at_1_max value: 8.0922 - type: nauc_recall_at_1_std value: -31.6799 - type: nauc_recall_at_1_diff1 value: 42.1566 - type: nauc_recall_at_3_max value: 1.7718999999999998 - type: nauc_recall_at_3_std value: -33.3208 - type: nauc_recall_at_3_diff1 value: 24.388 - type: nauc_recall_at_5_max value: -0.1774 - type: nauc_recall_at_5_std value: -34.9405 - type: nauc_recall_at_5_diff1 value: 19.7842 - type: nauc_recall_at_10_max value: -1.7014999999999998 - type: nauc_recall_at_10_std value: -33.9917 - type: nauc_recall_at_10_diff1 value: 15.3697 - type: nauc_recall_at_20_max value: -0.48669999999999997 - type: nauc_recall_at_20_std value: -30.890800000000002 - type: nauc_recall_at_20_diff1 value: 17.1512 - type: nauc_recall_at_100_max value: 7.9208 - type: nauc_recall_at_100_std value: -16.7223 - type: nauc_recall_at_100_diff1 value: 24.4227 - type: nauc_recall_at_1000_max value: 46.3102 - type: nauc_recall_at_1000_std value: 55.03360000000001 - type: nauc_recall_at_1000_diff1 value: 37.6798 - type: nauc_precision_at_1_max value: 55.07379999999999 - type: nauc_precision_at_1_std value: 6.4353 - type: nauc_precision_at_1_diff1 value: 30.604 - type: nauc_precision_at_3_max value: 36.3344 - type: nauc_precision_at_3_std value: 24.7186 - type: nauc_precision_at_3_diff1 value: -5.5176 - type: nauc_precision_at_5_max value: 25.891799999999996 - type: nauc_precision_at_5_std value: 27.0728 - type: nauc_precision_at_5_diff1 value: -8.4442 - type: nauc_precision_at_10_max value: 21.151 - type: nauc_precision_at_10_std value: 33.8452 - type: nauc_precision_at_10_diff1 value: -13.6853 - type: nauc_precision_at_20_max value: 17.8701 - type: nauc_precision_at_20_std value: 39.250600000000006 - type: nauc_precision_at_20_diff1 value: -13.8478 - type: nauc_precision_at_100_max value: 12.2101 - type: nauc_precision_at_100_std value: 45.3089 - type: nauc_precision_at_100_diff1 value: -13.727800000000002 - type: nauc_precision_at_1000_max value: 0.49899999999999994 - type: nauc_precision_at_1000_std value: 34.724199999999996 - type: nauc_precision_at_1000_diff1 value: -19.284399999999998 - type: nauc_mrr_at_1_max value: 55.07379999999999 - type: nauc_mrr_at_1_std value: 6.4353 - type: nauc_mrr_at_1_diff1 value: 30.604 - type: nauc_mrr_at_3_max value: 53.581100000000006 - type: nauc_mrr_at_3_std value: 12.326600000000001 - type: nauc_mrr_at_3_diff1 value: 31.159599999999998 - type: nauc_mrr_at_5_max value: 53.3839 - type: nauc_mrr_at_5_std value: 9.695 - type: nauc_mrr_at_5_diff1 value: 30.5214 - type: nauc_mrr_at_10_max value: 53.8177 - type: nauc_mrr_at_10_std value: 10.4039 - type: nauc_mrr_at_10_diff1 value: 30.158099999999997 - type: nauc_mrr_at_20_max value: 53.8177 - type: nauc_mrr_at_20_std value: 10.4039 - type: nauc_mrr_at_20_diff1 value: 30.158099999999997 - type: nauc_mrr_at_100_max value: 53.7423 - type: nauc_mrr_at_100_std value: 10.235 - type: nauc_mrr_at_100_diff1 value: 30.1529 - type: nauc_mrr_at_1000_max value: 53.7423 - type: nauc_mrr_at_1000_std value: 10.235 - type: nauc_mrr_at_1000_diff1 value: 30.1529 - type: main_score value: 52.470000000000006 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: dentistry_and_oral_health type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 11.5 - type: ndcg_at_3 value: 11.129999999999999 - type: ndcg_at_5 value: 11.216 - type: ndcg_at_10 value: 11.392 - type: ndcg_at_20 value: 12.296999999999999 - type: ndcg_at_100 value: 15.956999999999999 - type: ndcg_at_1000 value: 25.529000000000003 - type: map_at_1 value: 2.275 - type: map_at_3 value: 3.94 - type: map_at_5 value: 4.327 - type: map_at_10 value: 5.04 - type: map_at_20 value: 5.646 - type: map_at_100 value: 7.135 - type: map_at_1000 value: 8.725 - type: recall_at_1 value: 2.275 - type: recall_at_3 value: 5.359 - type: recall_at_5 value: 6.393 - type: recall_at_10 value: 8.35 - type: recall_at_20 value: 11.158 - type: recall_at_100 value: 22.223000000000003 - type: recall_at_1000 value: 50.477000000000004 - type: precision_at_1 value: 15.5 - type: precision_at_3 value: 12.5 - type: precision_at_5 value: 11.4 - type: precision_at_10 value: 9.25 - type: precision_at_20 value: 7.875 - type: precision_at_100 value: 5.04 - type: precision_at_1000 value: 1.7389999999999999 - type: mrr_at_1 value: 15.5 - type: mrr_at_3 value: 20.5 - type: mrr_at_5 value: 21.925 - type: mrr_at_10 value: 22.5444 - type: mrr_at_20 value: 23.2625 - type: mrr_at_100 value: 23.7237 - type: mrr_at_1000 value: 23.8369 - type: nauc_ndcg_at_1_max value: 34.4589 - type: nauc_ndcg_at_1_std value: 42.1292 - type: nauc_ndcg_at_1_diff1 value: 17.9811 - type: nauc_ndcg_at_3_max value: 33.5732 - type: nauc_ndcg_at_3_std value: 41.7443 - type: nauc_ndcg_at_3_diff1 value: 24.6128 - type: nauc_ndcg_at_5_max value: 37.585 - type: nauc_ndcg_at_5_std value: 46.0713 - type: nauc_ndcg_at_5_diff1 value: 27.2211 - type: nauc_ndcg_at_10_max value: 40.0605 - type: nauc_ndcg_at_10_std value: 47.6628 - type: nauc_ndcg_at_10_diff1 value: 29.166700000000002 - type: nauc_ndcg_at_20_max value: 39.6748 - type: nauc_ndcg_at_20_std value: 49.601600000000005 - type: nauc_ndcg_at_20_diff1 value: 28.198 - type: nauc_ndcg_at_100_max value: 38.1407 - type: nauc_ndcg_at_100_std value: 52.9055 - type: nauc_ndcg_at_100_diff1 value: 20.5359 - type: nauc_ndcg_at_1000_max value: 41.737 - type: nauc_ndcg_at_1000_std value: 58.988600000000005 - type: nauc_ndcg_at_1000_diff1 value: 16.1082 - type: nauc_map_at_1_max value: 25.4402 - type: nauc_map_at_1_std value: 25.208599999999997 - type: nauc_map_at_1_diff1 value: 54.664500000000004 - type: nauc_map_at_3_max value: 24.0543 - type: nauc_map_at_3_std value: 28.7833 - type: nauc_map_at_3_diff1 value: 46.6306 - type: nauc_map_at_5_max value: 26.923799999999996 - type: nauc_map_at_5_std value: 32.233000000000004 - type: nauc_map_at_5_diff1 value: 42.9054 - type: nauc_map_at_10_max value: 31.8766 - type: nauc_map_at_10_std value: 36.6634 - type: nauc_map_at_10_diff1 value: 41.185500000000005 - type: nauc_map_at_20_max value: 32.3117 - type: nauc_map_at_20_std value: 39.8134 - type: nauc_map_at_20_diff1 value: 37.7037 - type: nauc_map_at_100_max value: 33.7308 - type: nauc_map_at_100_std value: 47.5419 - type: nauc_map_at_100_diff1 value: 27.683999999999997 - type: nauc_map_at_1000_max value: 37.358000000000004 - type: nauc_map_at_1000_std value: 53.5331 - type: nauc_map_at_1000_diff1 value: 22.4664 - type: nauc_recall_at_1_max value: 25.4402 - type: nauc_recall_at_1_std value: 25.208599999999997 - type: nauc_recall_at_1_diff1 value: 54.664500000000004 - type: nauc_recall_at_3_max value: 26.2666 - type: nauc_recall_at_3_std value: 32.2588 - type: nauc_recall_at_3_diff1 value: 46.1528 - type: nauc_recall_at_5_max value: 32.364 - type: nauc_recall_at_5_std value: 37.816300000000005 - type: nauc_recall_at_5_diff1 value: 38.8915 - type: nauc_recall_at_10_max value: 39.0272 - type: nauc_recall_at_10_std value: 43.8025 - type: nauc_recall_at_10_diff1 value: 37.08 - type: nauc_recall_at_20_max value: 35.629 - type: nauc_recall_at_20_std value: 45.7498 - type: nauc_recall_at_20_diff1 value: 31.1273 - type: nauc_recall_at_100_max value: 31.1328 - type: nauc_recall_at_100_std value: 50.60980000000001 - type: nauc_recall_at_100_diff1 value: 15.974499999999999 - type: nauc_recall_at_1000_max value: 35.4467 - type: nauc_recall_at_1000_std value: 64.4764 - type: nauc_recall_at_1000_diff1 value: 11.8923 - type: nauc_precision_at_1_max value: 34.8024 - type: nauc_precision_at_1_std value: 45.0141 - type: nauc_precision_at_1_diff1 value: 16.5504 - type: nauc_precision_at_3_max value: 34.2196 - type: nauc_precision_at_3_std value: 45.9596 - type: nauc_precision_at_3_diff1 value: 14.921100000000001 - type: nauc_precision_at_5_max value: 40.087 - type: nauc_precision_at_5_std value: 55.0371 - type: nauc_precision_at_5_diff1 value: 13.638 - type: nauc_precision_at_10_max value: 43.2516 - type: nauc_precision_at_10_std value: 55.863 - type: nauc_precision_at_10_diff1 value: 9.0558 - type: nauc_precision_at_20_max value: 42.8832 - type: nauc_precision_at_20_std value: 58.2846 - type: nauc_precision_at_20_diff1 value: -0.9745999999999999 - type: nauc_precision_at_100_max value: 38.4914 - type: nauc_precision_at_100_std value: 58.0033 - type: nauc_precision_at_100_diff1 value: -14.6786 - type: nauc_precision_at_1000_max value: 37.3476 - type: nauc_precision_at_1000_std value: 38.1122 - type: nauc_precision_at_1000_diff1 value: -11.295 - type: nauc_mrr_at_1_max value: 34.8024 - type: nauc_mrr_at_1_std value: 45.0141 - type: nauc_mrr_at_1_diff1 value: 16.5504 - type: nauc_mrr_at_3_max value: 33.0518 - type: nauc_mrr_at_3_std value: 44.4493 - type: nauc_mrr_at_3_diff1 value: 19.974700000000002 - type: nauc_mrr_at_5_max value: 36.3726 - type: nauc_mrr_at_5_std value: 47.3453 - type: nauc_mrr_at_5_diff1 value: 21.0721 - type: nauc_mrr_at_10_max value: 36.788199999999996 - type: nauc_mrr_at_10_std value: 47.2219 - type: nauc_mrr_at_10_diff1 value: 20.68 - type: nauc_mrr_at_20_max value: 36.9338 - type: nauc_mrr_at_20_std value: 46.978500000000004 - type: nauc_mrr_at_20_diff1 value: 20.479400000000002 - type: nauc_mrr_at_100_max value: 36.6516 - type: nauc_mrr_at_100_std value: 46.7474 - type: nauc_mrr_at_100_diff1 value: 20.1232 - type: nauc_mrr_at_1000_max value: 36.647400000000005 - type: nauc_mrr_at_1000_std value: 46.7738 - type: nauc_mrr_at_1000_diff1 value: 20.1326 - type: main_score value: 11.392 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: dentistry_and_oral_health type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 21.0 - type: ndcg_at_3 value: 19.622 - type: ndcg_at_5 value: 19.081 - type: ndcg_at_10 value: 18.684 - type: ndcg_at_20 value: 19.7 - type: ndcg_at_100 value: 24.552 - type: ndcg_at_1000 value: 35.679 - type: map_at_1 value: 3.44 - type: map_at_3 value: 5.6000000000000005 - type: map_at_5 value: 6.300999999999999 - type: map_at_10 value: 7.396 - type: map_at_20 value: 8.601 - type: map_at_100 value: 11.357000000000001 - type: map_at_1000 value: 13.666 - type: recall_at_1 value: 3.44 - type: recall_at_3 value: 6.642 - type: recall_at_5 value: 8.046000000000001 - type: recall_at_10 value: 10.903 - type: recall_at_20 value: 16.244 - type: recall_at_100 value: 31.702 - type: recall_at_1000 value: 64.905 - type: precision_at_1 value: 26.0 - type: precision_at_3 value: 22.0 - type: precision_at_5 value: 19.1 - type: precision_at_10 value: 15.35 - type: precision_at_20 value: 13.0 - type: precision_at_100 value: 7.920000000000001 - type: precision_at_1000 value: 2.2769999999999997 - type: mrr_at_1 value: 26.0 - type: mrr_at_3 value: 33.75 - type: mrr_at_5 value: 34.625 - type: mrr_at_10 value: 35.6232 - type: mrr_at_20 value: 36.229 - type: mrr_at_100 value: 36.717 - type: mrr_at_1000 value: 36.7906 - type: nauc_ndcg_at_1_max value: 45.3643 - type: nauc_ndcg_at_1_std value: 30.419200000000004 - type: nauc_ndcg_at_1_diff1 value: 23.1069 - type: nauc_ndcg_at_3_max value: 54.793800000000005 - type: nauc_ndcg_at_3_std value: 37.5309 - type: nauc_ndcg_at_3_diff1 value: 25.2997 - type: nauc_ndcg_at_5_max value: 56.3971 - type: nauc_ndcg_at_5_std value: 40.1252 - type: nauc_ndcg_at_5_diff1 value: 24.5363 - type: nauc_ndcg_at_10_max value: 54.8812 - type: nauc_ndcg_at_10_std value: 39.813700000000004 - type: nauc_ndcg_at_10_diff1 value: 21.8135 - type: nauc_ndcg_at_20_max value: 53.821 - type: nauc_ndcg_at_20_std value: 41.4119 - type: nauc_ndcg_at_20_diff1 value: 23.8397 - type: nauc_ndcg_at_100_max value: 58.186800000000005 - type: nauc_ndcg_at_100_std value: 47.5131 - type: nauc_ndcg_at_100_diff1 value: 22.016099999999998 - type: nauc_ndcg_at_1000_max value: 63.188599999999994 - type: nauc_ndcg_at_1000_std value: 53.569199999999995 - type: nauc_ndcg_at_1000_diff1 value: 16.2596 - type: nauc_map_at_1_max value: 44.656400000000005 - type: nauc_map_at_1_std value: 18.0396 - type: nauc_map_at_1_diff1 value: 46.7439 - type: nauc_map_at_3_max value: 47.910199999999996 - type: nauc_map_at_3_std value: 22.9629 - type: nauc_map_at_3_diff1 value: 45.7872 - type: nauc_map_at_5_max value: 47.903400000000005 - type: nauc_map_at_5_std value: 25.141799999999996 - type: nauc_map_at_5_diff1 value: 42.8426 - type: nauc_map_at_10_max value: 47.062599999999996 - type: nauc_map_at_10_std value: 27.643600000000003 - type: nauc_map_at_10_diff1 value: 37.3522 - type: nauc_map_at_20_max value: 47.1474 - type: nauc_map_at_20_std value: 31.1367 - type: nauc_map_at_20_diff1 value: 35.6935 - type: nauc_map_at_100_max value: 53.796699999999994 - type: nauc_map_at_100_std value: 42.4636 - type: nauc_map_at_100_diff1 value: 29.5198 - type: nauc_map_at_1000_max value: 58.303799999999995 - type: nauc_map_at_1000_std value: 49.193999999999996 - type: nauc_map_at_1000_diff1 value: 25.9758 - type: nauc_recall_at_1_max value: 44.656400000000005 - type: nauc_recall_at_1_std value: 18.0396 - type: nauc_recall_at_1_diff1 value: 46.7439 - type: nauc_recall_at_3_max value: 47.9712 - type: nauc_recall_at_3_std value: 24.403 - type: nauc_recall_at_3_diff1 value: 43.9697 - type: nauc_recall_at_5_max value: 45.786500000000004 - type: nauc_recall_at_5_std value: 26.670899999999996 - type: nauc_recall_at_5_diff1 value: 38.9454 - type: nauc_recall_at_10_max value: 41.8495 - type: nauc_recall_at_10_std value: 28.158499999999997 - type: nauc_recall_at_10_diff1 value: 29.1754 - type: nauc_recall_at_20_max value: 35.3029 - type: nauc_recall_at_20_std value: 28.608099999999997 - type: nauc_recall_at_20_diff1 value: 30.7132 - type: nauc_recall_at_100_max value: 48.8887 - type: nauc_recall_at_100_std value: 44.1464 - type: nauc_recall_at_100_diff1 value: 22.5521 - type: nauc_recall_at_1000_max value: 59.721599999999995 - type: nauc_recall_at_1000_std value: 62.6649 - type: nauc_recall_at_1000_diff1 value: 11.315999999999999 - type: nauc_precision_at_1_max value: 42.808800000000005 - type: nauc_precision_at_1_std value: 35.0879 - type: nauc_precision_at_1_diff1 value: 16.0605 - type: nauc_precision_at_3_max value: 57.6241 - type: nauc_precision_at_3_std value: 45.3184 - type: nauc_precision_at_3_diff1 value: 13.2253 - type: nauc_precision_at_5_max value: 56.070600000000006 - type: nauc_precision_at_5_std value: 47.815000000000005 - type: nauc_precision_at_5_diff1 value: 7.6166 - type: nauc_precision_at_10_max value: 52.8107 - type: nauc_precision_at_10_std value: 48.8904 - type: nauc_precision_at_10_diff1 value: -0.12160000000000001 - type: nauc_precision_at_20_max value: 49.7597 - type: nauc_precision_at_20_std value: 49.0284 - type: nauc_precision_at_20_diff1 value: -4.8077000000000005 - type: nauc_precision_at_100_max value: 51.4923 - type: nauc_precision_at_100_std value: 54.0702 - type: nauc_precision_at_100_diff1 value: -6.364400000000001 - type: nauc_precision_at_1000_max value: 41.0295 - type: nauc_precision_at_1000_std value: 38.389 - type: nauc_precision_at_1000_diff1 value: -10.9986 - type: nauc_mrr_at_1_max value: 42.808800000000005 - type: nauc_mrr_at_1_std value: 35.0879 - type: nauc_mrr_at_1_diff1 value: 16.0605 - type: nauc_mrr_at_3_max value: 50.1471 - type: nauc_mrr_at_3_std value: 37.419000000000004 - type: nauc_mrr_at_3_diff1 value: 13.4354 - type: nauc_mrr_at_5_max value: 51.6494 - type: nauc_mrr_at_5_std value: 38.4967 - type: nauc_mrr_at_5_diff1 value: 14.396 - type: nauc_mrr_at_10_max value: 51.2505 - type: nauc_mrr_at_10_std value: 38.1019 - type: nauc_mrr_at_10_diff1 value: 13.2467 - type: nauc_mrr_at_20_max value: 51.224000000000004 - type: nauc_mrr_at_20_std value: 38.1196 - type: nauc_mrr_at_20_diff1 value: 13.3101 - type: nauc_mrr_at_100_max value: 51.0104 - type: nauc_mrr_at_100_std value: 38.023 - type: nauc_mrr_at_100_diff1 value: 13.146099999999999 - type: nauc_mrr_at_1000_max value: 50.9907 - type: nauc_mrr_at_1000_std value: 38.0783 - type: nauc_mrr_at_1000_diff1 value: 13.181999999999999 - type: main_score value: 18.684 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: dermatology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 56.99999999999999 - type: ndcg_at_3 value: 53.815999999999995 - type: ndcg_at_5 value: 55.147999999999996 - type: ndcg_at_10 value: 58.379999999999995 - type: ndcg_at_20 value: 62.053000000000004 - type: ndcg_at_100 value: 68.319 - type: ndcg_at_1000 value: 71.417 - type: map_at_1 value: 13.383000000000001 - type: map_at_3 value: 26.349 - type: map_at_5 value: 32.409 - type: map_at_10 value: 40.191 - type: map_at_20 value: 45.568 - type: map_at_100 value: 51.565000000000005 - type: map_at_1000 value: 52.825 - type: recall_at_1 value: 13.383000000000001 - type: recall_at_3 value: 30.053 - type: recall_at_5 value: 39.617999999999995 - type: recall_at_10 value: 53.89000000000001 - type: recall_at_20 value: 67.54599999999999 - type: recall_at_100 value: 89.642 - type: recall_at_1000 value: 99.442 - type: precision_at_1 value: 66.0 - type: precision_at_3 value: 55.167 - type: precision_at_5 value: 49.3 - type: precision_at_10 value: 39.5 - type: precision_at_20 value: 28.825 - type: precision_at_100 value: 10.334999999999999 - type: precision_at_1000 value: 1.3390000000000002 - type: mrr_at_1 value: 66.0 - type: mrr_at_3 value: 75.75 - type: mrr_at_5 value: 76.6 - type: mrr_at_10 value: 77.0468 - type: mrr_at_20 value: 77.28960000000001 - type: mrr_at_100 value: 77.312 - type: mrr_at_1000 value: 77.312 - type: nauc_ndcg_at_1_max value: 42.2784 - type: nauc_ndcg_at_1_std value: 20.6358 - type: nauc_ndcg_at_1_diff1 value: 45.803 - type: nauc_ndcg_at_3_max value: 33.1528 - type: nauc_ndcg_at_3_std value: 5.2341999999999995 - type: nauc_ndcg_at_3_diff1 value: 33.2999 - type: nauc_ndcg_at_5_max value: 33.4856 - type: nauc_ndcg_at_5_std value: -1.797 - type: nauc_ndcg_at_5_diff1 value: 35.65 - type: nauc_ndcg_at_10_max value: 33.5882 - type: nauc_ndcg_at_10_std value: -4.7413 - type: nauc_ndcg_at_10_diff1 value: 32.3708 - type: nauc_ndcg_at_20_max value: 36.698 - type: nauc_ndcg_at_20_std value: -4.7399 - type: nauc_ndcg_at_20_diff1 value: 32.31 - type: nauc_ndcg_at_100_max value: 37.9711 - type: nauc_ndcg_at_100_std value: 0.7188 - type: nauc_ndcg_at_100_diff1 value: 30.981199999999998 - type: nauc_ndcg_at_1000_max value: 39.01 - type: nauc_ndcg_at_1000_std value: 12.5777 - type: nauc_ndcg_at_1000_diff1 value: 31.7875 - type: nauc_map_at_1_max value: 2.2602 - type: nauc_map_at_1_std value: -30.1147 - type: nauc_map_at_1_diff1 value: 42.0209 - type: nauc_map_at_3_max value: 3.7150000000000003 - type: nauc_map_at_3_std value: -34.897 - type: nauc_map_at_3_diff1 value: 33.908 - type: nauc_map_at_5_max value: 8.4404 - type: nauc_map_at_5_std value: -36.6448 - type: nauc_map_at_5_diff1 value: 33.4709 - type: nauc_map_at_10_max value: 16.4324 - type: nauc_map_at_10_std value: -33.3115 - type: nauc_map_at_10_diff1 value: 28.662100000000002 - type: nauc_map_at_20_max value: 21.8025 - type: nauc_map_at_20_std value: -27.3198 - type: nauc_map_at_20_diff1 value: 26.041700000000002 - type: nauc_map_at_100_max value: 26.609500000000004 - type: nauc_map_at_100_std value: -12.804499999999999 - type: nauc_map_at_100_diff1 value: 23.1641 - type: nauc_map_at_1000_max value: 27.4532 - type: nauc_map_at_1000_std value: -7.6803 - type: nauc_map_at_1000_diff1 value: 23.1214 - type: nauc_recall_at_1_max value: 2.2602 - type: nauc_recall_at_1_std value: -30.1147 - type: nauc_recall_at_1_diff1 value: 42.0209 - type: nauc_recall_at_3_max value: -3.0239 - type: nauc_recall_at_3_std value: -40.383599999999994 - type: nauc_recall_at_3_diff1 value: 29.9087 - type: nauc_recall_at_5_max value: 2.141 - type: nauc_recall_at_5_std value: -43.4961 - type: nauc_recall_at_5_diff1 value: 28.571099999999998 - type: nauc_recall_at_10_max value: 6.8745 - type: nauc_recall_at_10_std value: -44.0475 - type: nauc_recall_at_10_diff1 value: 21.3471 - type: nauc_recall_at_20_max value: 13.4685 - type: nauc_recall_at_20_std value: -44.935199999999995 - type: nauc_recall_at_20_diff1 value: 17.552599999999998 - type: nauc_recall_at_100_max value: 19.8098 - type: nauc_recall_at_100_std value: -36.491600000000005 - type: nauc_recall_at_100_diff1 value: 3.5177 - type: nauc_recall_at_1000_max value: 38.0945 - type: nauc_recall_at_1000_std value: 28.4766 - type: nauc_recall_at_1000_diff1 value: -3.2825 - type: nauc_precision_at_1_max value: 43.9045 - type: nauc_precision_at_1_std value: 20.3052 - type: nauc_precision_at_1_diff1 value: 46.9124 - type: nauc_precision_at_3_max value: 32.4069 - type: nauc_precision_at_3_std value: 19.780800000000003 - type: nauc_precision_at_3_diff1 value: 9.3264 - type: nauc_precision_at_5_max value: 35.2436 - type: nauc_precision_at_5_std value: 26.284800000000004 - type: nauc_precision_at_5_diff1 value: 1.4762000000000002 - type: nauc_precision_at_10_max value: 34.0488 - type: nauc_precision_at_10_std value: 43.9403 - type: nauc_precision_at_10_diff1 value: -11.4419 - type: nauc_precision_at_20_max value: 27.6387 - type: nauc_precision_at_20_std value: 54.8187 - type: nauc_precision_at_20_diff1 value: -15.8486 - type: nauc_precision_at_100_max value: 14.2797 - type: nauc_precision_at_100_std value: 62.7694 - type: nauc_precision_at_100_diff1 value: -15.8703 - type: nauc_precision_at_1000_max value: 9.6766 - type: nauc_precision_at_1000_std value: 64.4804 - type: nauc_precision_at_1000_diff1 value: -12.1726 - type: nauc_mrr_at_1_max value: 43.9045 - type: nauc_mrr_at_1_std value: 20.3052 - type: nauc_mrr_at_1_diff1 value: 46.9124 - type: nauc_mrr_at_3_max value: 41.680099999999996 - type: nauc_mrr_at_3_std value: 20.413600000000002 - type: nauc_mrr_at_3_diff1 value: 44.3971 - type: nauc_mrr_at_5_max value: 43.655100000000004 - type: nauc_mrr_at_5_std value: 21.6815 - type: nauc_mrr_at_5_diff1 value: 44.8015 - type: nauc_mrr_at_10_max value: 43.8731 - type: nauc_mrr_at_10_std value: 21.704 - type: nauc_mrr_at_10_diff1 value: 45.651799999999994 - type: nauc_mrr_at_20_max value: 43.6494 - type: nauc_mrr_at_20_std value: 21.4422 - type: nauc_mrr_at_20_diff1 value: 45.4376 - type: nauc_mrr_at_100_max value: 43.5906 - type: nauc_mrr_at_100_std value: 21.4023 - type: nauc_mrr_at_100_diff1 value: 45.4133 - type: nauc_mrr_at_1000_max value: 43.5906 - type: nauc_mrr_at_1000_std value: 21.4023 - type: nauc_mrr_at_1000_diff1 value: 45.4133 - type: main_score value: 58.379999999999995 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: dermatology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 25.25 - type: ndcg_at_3 value: 24.462 - type: ndcg_at_5 value: 24.709999999999997 - type: ndcg_at_10 value: 27.426000000000002 - type: ndcg_at_20 value: 31.517 - type: ndcg_at_100 value: 39.115 - type: ndcg_at_1000 value: 47.007 - type: map_at_1 value: 5.7509999999999994 - type: map_at_3 value: 11.652 - type: map_at_5 value: 14.052999999999999 - type: map_at_10 value: 17.31 - type: map_at_20 value: 20.288 - type: map_at_100 value: 23.880000000000003 - type: map_at_1000 value: 25.47 - type: recall_at_1 value: 5.7509999999999994 - type: recall_at_3 value: 14.343 - type: recall_at_5 value: 19.093 - type: recall_at_10 value: 27.607 - type: recall_at_20 value: 38.940000000000005 - type: recall_at_100 value: 62.632 - type: recall_at_1000 value: 92.482 - type: precision_at_1 value: 31.0 - type: precision_at_3 value: 25.667 - type: precision_at_5 value: 21.9 - type: precision_at_10 value: 18.5 - type: precision_at_20 value: 14.899999999999999 - type: precision_at_100 value: 6.660000000000001 - type: precision_at_1000 value: 1.2510000000000001 - type: mrr_at_1 value: 31.0 - type: mrr_at_3 value: 38.8333 - type: mrr_at_5 value: 40.1833 - type: mrr_at_10 value: 41.700199999999995 - type: mrr_at_20 value: 42.4573 - type: mrr_at_100 value: 42.8185 - type: mrr_at_1000 value: 42.8575 - type: nauc_ndcg_at_1_max value: 30.5552 - type: nauc_ndcg_at_1_std value: 11.4792 - type: nauc_ndcg_at_1_diff1 value: 5.6798 - type: nauc_ndcg_at_3_max value: 42.5877 - type: nauc_ndcg_at_3_std value: 11.561399999999999 - type: nauc_ndcg_at_3_diff1 value: 16.109499999999997 - type: nauc_ndcg_at_5_max value: 41.526 - type: nauc_ndcg_at_5_std value: 8.7759 - type: nauc_ndcg_at_5_diff1 value: 18.9848 - type: nauc_ndcg_at_10_max value: 44.8402 - type: nauc_ndcg_at_10_std value: 8.9034 - type: nauc_ndcg_at_10_diff1 value: 20.1171 - type: nauc_ndcg_at_20_max value: 46.6789 - type: nauc_ndcg_at_20_std value: 11.5659 - type: nauc_ndcg_at_20_diff1 value: 17.324 - type: nauc_ndcg_at_100_max value: 49.778299999999994 - type: nauc_ndcg_at_100_std value: 17.2405 - type: nauc_ndcg_at_100_diff1 value: 12.590399999999999 - type: nauc_ndcg_at_1000_max value: 52.843399999999995 - type: nauc_ndcg_at_1000_std value: 28.4096 - type: nauc_ndcg_at_1000_diff1 value: 11.2003 - type: nauc_map_at_1_max value: 28.37 - type: nauc_map_at_1_std value: -9.6805 - type: nauc_map_at_1_diff1 value: 22.9643 - type: nauc_map_at_3_max value: 29.716900000000003 - type: nauc_map_at_3_std value: -14.0931 - type: nauc_map_at_3_diff1 value: 28.193800000000003 - type: nauc_map_at_5_max value: 29.2528 - type: nauc_map_at_5_std value: -13.189899999999998 - type: nauc_map_at_5_diff1 value: 27.340700000000002 - type: nauc_map_at_10_max value: 33.0456 - type: nauc_map_at_10_std value: -8.3359 - type: nauc_map_at_10_diff1 value: 24.0125 - type: nauc_map_at_20_max value: 38.487 - type: nauc_map_at_20_std value: -1.8057 - type: nauc_map_at_20_diff1 value: 21.898200000000003 - type: nauc_map_at_100_max value: 43.062400000000004 - type: nauc_map_at_100_std value: 7.0085 - type: nauc_map_at_100_diff1 value: 18.4593 - type: nauc_map_at_1000_max value: 44.7972 - type: nauc_map_at_1000_std value: 11.9907 - type: nauc_map_at_1000_diff1 value: 17.4989 - type: nauc_recall_at_1_max value: 28.37 - type: nauc_recall_at_1_std value: -9.6805 - type: nauc_recall_at_1_diff1 value: 22.9643 - type: nauc_recall_at_3_max value: 32.583400000000005 - type: nauc_recall_at_3_std value: -14.077799999999998 - type: nauc_recall_at_3_diff1 value: 34.2738 - type: nauc_recall_at_5_max value: 29.554799999999997 - type: nauc_recall_at_5_std value: -13.7679 - type: nauc_recall_at_5_diff1 value: 34.0684 - type: nauc_recall_at_10_max value: 34.7648 - type: nauc_recall_at_10_std value: -6.702800000000001 - type: nauc_recall_at_10_diff1 value: 27.4069 - type: nauc_recall_at_20_max value: 36.5072 - type: nauc_recall_at_20_std value: -1.8129 - type: nauc_recall_at_20_diff1 value: 20.105500000000003 - type: nauc_recall_at_100_max value: 44.756800000000005 - type: nauc_recall_at_100_std value: 13.087599999999998 - type: nauc_recall_at_100_diff1 value: 8.463099999999999 - type: nauc_recall_at_1000_max value: 71.0217 - type: nauc_recall_at_1000_std value: 66.8821 - type: nauc_recall_at_1000_diff1 value: -5.8082 - type: nauc_precision_at_1_max value: 31.4835 - type: nauc_precision_at_1_std value: 12.057500000000001 - type: nauc_precision_at_1_diff1 value: 4.9834000000000005 - type: nauc_precision_at_3_max value: 41.4188 - type: nauc_precision_at_3_std value: 19.1824 - type: nauc_precision_at_3_diff1 value: 8.4729 - type: nauc_precision_at_5_max value: 39.825 - type: nauc_precision_at_5_std value: 23.592399999999998 - type: nauc_precision_at_5_diff1 value: 4.0221 - type: nauc_precision_at_10_max value: 43.7645 - type: nauc_precision_at_10_std value: 32.7934 - type: nauc_precision_at_10_diff1 value: -1.2515 - type: nauc_precision_at_20_max value: 43.0757 - type: nauc_precision_at_20_std value: 42.4566 - type: nauc_precision_at_20_diff1 value: -7.9428 - type: nauc_precision_at_100_max value: 31.418400000000002 - type: nauc_precision_at_100_std value: 56.1558 - type: nauc_precision_at_100_diff1 value: -17.5248 - type: nauc_precision_at_1000_max value: 18.6971 - type: nauc_precision_at_1000_std value: 50.943000000000005 - type: nauc_precision_at_1000_diff1 value: -12.6889 - type: nauc_mrr_at_1_max value: 31.4835 - type: nauc_mrr_at_1_std value: 12.057500000000001 - type: nauc_mrr_at_1_diff1 value: 4.9834000000000005 - type: nauc_mrr_at_3_max value: 38.426700000000004 - type: nauc_mrr_at_3_std value: 15.7666 - type: nauc_mrr_at_3_diff1 value: 11.0725 - type: nauc_mrr_at_5_max value: 38.1481 - type: nauc_mrr_at_5_std value: 16.1965 - type: nauc_mrr_at_5_diff1 value: 9.7109 - type: nauc_mrr_at_10_max value: 39.7563 - type: nauc_mrr_at_10_std value: 17.0487 - type: nauc_mrr_at_10_diff1 value: 9.6843 - type: nauc_mrr_at_20_max value: 39.2711 - type: nauc_mrr_at_20_std value: 17.1654 - type: nauc_mrr_at_20_diff1 value: 8.7669 - type: nauc_mrr_at_100_max value: 39.2493 - type: nauc_mrr_at_100_std value: 17.198 - type: nauc_mrr_at_100_diff1 value: 9.006699999999999 - type: nauc_mrr_at_1000_max value: 39.2259 - type: nauc_mrr_at_1000_std value: 17.1768 - type: nauc_mrr_at_1000_diff1 value: 9.0104 - type: main_score value: 27.426000000000002 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: dermatology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 31.0 - type: ndcg_at_3 value: 28.693999999999996 - type: ndcg_at_5 value: 30.203000000000003 - type: ndcg_at_10 value: 33.913 - type: ndcg_at_20 value: 38.05 - type: ndcg_at_100 value: 45.211 - type: ndcg_at_1000 value: 51.315 - type: map_at_1 value: 6.069999999999999 - type: map_at_3 value: 12.674 - type: map_at_5 value: 16.296 - type: map_at_10 value: 21.586 - type: map_at_20 value: 25.115 - type: map_at_100 value: 29.037000000000003 - type: map_at_1000 value: 30.389 - type: recall_at_1 value: 6.069999999999999 - type: recall_at_3 value: 15.529000000000002 - type: recall_at_5 value: 22.747 - type: recall_at_10 value: 34.821000000000005 - type: recall_at_20 value: 46.635 - type: recall_at_100 value: 69.726 - type: recall_at_1000 value: 92.412 - type: precision_at_1 value: 36.5 - type: precision_at_3 value: 30.833 - type: precision_at_5 value: 28.499999999999996 - type: precision_at_10 value: 24.349999999999998 - type: precision_at_20 value: 18.75 - type: precision_at_100 value: 7.46 - type: precision_at_1000 value: 1.2309999999999999 - type: mrr_at_1 value: 36.5 - type: mrr_at_3 value: 43.75 - type: mrr_at_5 value: 46.025 - type: mrr_at_10 value: 47.2933 - type: mrr_at_20 value: 47.9269 - type: mrr_at_100 value: 48.2183 - type: mrr_at_1000 value: 48.2385 - type: nauc_ndcg_at_1_max value: 34.7176 - type: nauc_ndcg_at_1_std value: 24.3626 - type: nauc_ndcg_at_1_diff1 value: 14.8799 - type: nauc_ndcg_at_3_max value: 44.1357 - type: nauc_ndcg_at_3_std value: 20.0165 - type: nauc_ndcg_at_3_diff1 value: 7.1797 - type: nauc_ndcg_at_5_max value: 44.8908 - type: nauc_ndcg_at_5_std value: 18.2622 - type: nauc_ndcg_at_5_diff1 value: 8.1525 - type: nauc_ndcg_at_10_max value: 47.6186 - type: nauc_ndcg_at_10_std value: 16.781299999999998 - type: nauc_ndcg_at_10_diff1 value: 9.7705 - type: nauc_ndcg_at_20_max value: 49.9604 - type: nauc_ndcg_at_20_std value: 17.9142 - type: nauc_ndcg_at_20_diff1 value: 8.8726 - type: nauc_ndcg_at_100_max value: 53.24379999999999 - type: nauc_ndcg_at_100_std value: 25.1583 - type: nauc_ndcg_at_100_diff1 value: 8.236400000000001 - type: nauc_ndcg_at_1000_max value: 50.3497 - type: nauc_ndcg_at_1000_std value: 35.7518 - type: nauc_ndcg_at_1000_diff1 value: 3.6366 - type: nauc_map_at_1_max value: 30.600699999999996 - type: nauc_map_at_1_std value: -4.7101 - type: nauc_map_at_1_diff1 value: 27.4254 - type: nauc_map_at_3_max value: 34.5948 - type: nauc_map_at_3_std value: -7.008699999999999 - type: nauc_map_at_3_diff1 value: 17.810699999999997 - type: nauc_map_at_5_max value: 35.4128 - type: nauc_map_at_5_std value: -6.8513 - type: nauc_map_at_5_diff1 value: 14.3932 - type: nauc_map_at_10_max value: 40.1998 - type: nauc_map_at_10_std value: -1.1526 - type: nauc_map_at_10_diff1 value: 11.0807 - type: nauc_map_at_20_max value: 44.6921 - type: nauc_map_at_20_std value: 5.161700000000001 - type: nauc_map_at_20_diff1 value: 9.1443 - type: nauc_map_at_100_max value: 48.127199999999995 - type: nauc_map_at_100_std value: 14.274999999999999 - type: nauc_map_at_100_diff1 value: 7.5502 - type: nauc_map_at_1000_max value: 48.1338 - type: nauc_map_at_1000_std value: 18.9952 - type: nauc_map_at_1000_diff1 value: 6.1902 - type: nauc_recall_at_1_max value: 30.600699999999996 - type: nauc_recall_at_1_std value: -4.7101 - type: nauc_recall_at_1_diff1 value: 27.4254 - type: nauc_recall_at_3_max value: 37.358999999999995 - type: nauc_recall_at_3_std value: -6.412 - type: nauc_recall_at_3_diff1 value: 17.1264 - type: nauc_recall_at_5_max value: 32.5293 - type: nauc_recall_at_5_std value: -10.5716 - type: nauc_recall_at_5_diff1 value: 14.913000000000002 - type: nauc_recall_at_10_max value: 39.690999999999995 - type: nauc_recall_at_10_std value: -5.520300000000001 - type: nauc_recall_at_10_diff1 value: 13.8849 - type: nauc_recall_at_20_max value: 45.4972 - type: nauc_recall_at_20_std value: -2.3068999999999997 - type: nauc_recall_at_20_diff1 value: 11.0097 - type: nauc_recall_at_100_max value: 57.6612 - type: nauc_recall_at_100_std value: 16.7538 - type: nauc_recall_at_100_diff1 value: 10.347000000000001 - type: nauc_recall_at_1000_max value: 70.0951 - type: nauc_recall_at_1000_std value: 72.3359 - type: nauc_recall_at_1000_diff1 value: -16.9591 - type: nauc_precision_at_1_max value: 33.2225 - type: nauc_precision_at_1_std value: 23.122400000000003 - type: nauc_precision_at_1_diff1 value: 16.112199999999998 - type: nauc_precision_at_3_max value: 43.839299999999994 - type: nauc_precision_at_3_std value: 26.492700000000003 - type: nauc_precision_at_3_diff1 value: -2.8691 - type: nauc_precision_at_5_max value: 43.7524 - type: nauc_precision_at_5_std value: 32.3313 - type: nauc_precision_at_5_diff1 value: -4.8185 - type: nauc_precision_at_10_max value: 41.793400000000005 - type: nauc_precision_at_10_std value: 36.8825 - type: nauc_precision_at_10_diff1 value: -8.4039 - type: nauc_precision_at_20_max value: 38.3037 - type: nauc_precision_at_20_std value: 47.396100000000004 - type: nauc_precision_at_20_diff1 value: -8.7708 - type: nauc_precision_at_100_max value: 21.5088 - type: nauc_precision_at_100_std value: 59.5302 - type: nauc_precision_at_100_diff1 value: -11.9556 - type: nauc_precision_at_1000_max value: -0.3614 - type: nauc_precision_at_1000_std value: 53.013600000000004 - type: nauc_precision_at_1000_diff1 value: -16.9022 - type: nauc_mrr_at_1_max value: 33.2225 - type: nauc_mrr_at_1_std value: 23.122400000000003 - type: nauc_mrr_at_1_diff1 value: 16.112199999999998 - type: nauc_mrr_at_3_max value: 41.2421 - type: nauc_mrr_at_3_std value: 24.9103 - type: nauc_mrr_at_3_diff1 value: 10.6623 - type: nauc_mrr_at_5_max value: 40.8046 - type: nauc_mrr_at_5_std value: 26.517699999999998 - type: nauc_mrr_at_5_diff1 value: 12.2735 - type: nauc_mrr_at_10_max value: 40.2669 - type: nauc_mrr_at_10_std value: 26.222800000000003 - type: nauc_mrr_at_10_diff1 value: 11.1785 - type: nauc_mrr_at_20_max value: 40.2118 - type: nauc_mrr_at_20_std value: 25.9945 - type: nauc_mrr_at_20_diff1 value: 11.3524 - type: nauc_mrr_at_100_max value: 40.396300000000004 - type: nauc_mrr_at_100_std value: 26.235799999999998 - type: nauc_mrr_at_100_diff1 value: 11.4847 - type: nauc_mrr_at_1000_max value: 40.3684 - type: nauc_mrr_at_1000_std value: 26.217299999999998 - type: nauc_mrr_at_1000_diff1 value: 11.4967 - type: main_score value: 33.913 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: gastroenterology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 63.24999999999999 - type: ndcg_at_3 value: 58.5 - type: ndcg_at_5 value: 56.14 - type: ndcg_at_10 value: 55.42 - type: ndcg_at_20 value: 56.738 - type: ndcg_at_100 value: 64.412 - type: ndcg_at_1000 value: 72.794 - type: map_at_1 value: 6.805 - type: map_at_3 value: 13.644 - type: map_at_5 value: 17.426 - type: map_at_10 value: 23.127 - type: map_at_20 value: 29.354999999999997 - type: map_at_100 value: 41.601 - type: map_at_1000 value: 46.983999999999995 - type: recall_at_1 value: 6.805 - type: recall_at_3 value: 14.738000000000001 - type: recall_at_5 value: 19.801 - type: recall_at_10 value: 29.599999999999998 - type: recall_at_20 value: 41.77 - type: recall_at_100 value: 72.682 - type: recall_at_1000 value: 97.773 - type: precision_at_1 value: 74.5 - type: precision_at_3 value: 66.667 - type: precision_at_5 value: 60.199999999999996 - type: precision_at_10 value: 52.1 - type: precision_at_20 value: 44.574999999999996 - type: precision_at_100 value: 23.51 - type: precision_at_1000 value: 4.7 - type: mrr_at_1 value: 74.5 - type: mrr_at_3 value: 82.08330000000001 - type: mrr_at_5 value: 82.5583 - type: mrr_at_10 value: 83.19919999999999 - type: mrr_at_20 value: 83.2761 - type: mrr_at_100 value: 83.3136 - type: mrr_at_1000 value: 83.3179 - type: nauc_ndcg_at_1_max value: 30.5093 - type: nauc_ndcg_at_1_std value: 15.548699999999998 - type: nauc_ndcg_at_1_diff1 value: 29.909200000000002 - type: nauc_ndcg_at_3_max value: 37.911 - type: nauc_ndcg_at_3_std value: 21.8751 - type: nauc_ndcg_at_3_diff1 value: 15.7172 - type: nauc_ndcg_at_5_max value: 38.337500000000006 - type: nauc_ndcg_at_5_std value: 18.2302 - type: nauc_ndcg_at_5_diff1 value: 14.9655 - type: nauc_ndcg_at_10_max value: 41.3816 - type: nauc_ndcg_at_10_std value: 17.222 - type: nauc_ndcg_at_10_diff1 value: 17.309 - type: nauc_ndcg_at_20_max value: 45.068799999999996 - type: nauc_ndcg_at_20_std value: 17.0449 - type: nauc_ndcg_at_20_diff1 value: 18.784100000000002 - type: nauc_ndcg_at_100_max value: 46.426899999999996 - type: nauc_ndcg_at_100_std value: 30.9243 - type: nauc_ndcg_at_100_diff1 value: 20.328699999999998 - type: nauc_ndcg_at_1000_max value: 48.744 - type: nauc_ndcg_at_1000_std value: 33.278 - type: nauc_ndcg_at_1000_diff1 value: 14.552100000000001 - type: nauc_map_at_1_max value: 8.1792 - type: nauc_map_at_1_std value: -6.5594 - type: nauc_map_at_1_diff1 value: 49.4009 - type: nauc_map_at_3_max value: 21.1779 - type: nauc_map_at_3_std value: 0.5627 - type: nauc_map_at_3_diff1 value: 33.3576 - type: nauc_map_at_5_max value: 23.1508 - type: nauc_map_at_5_std value: 3.5987 - type: nauc_map_at_5_diff1 value: 26.9839 - type: nauc_map_at_10_max value: 26.486700000000003 - type: nauc_map_at_10_std value: 8.6895 - type: nauc_map_at_10_diff1 value: 21.1984 - type: nauc_map_at_20_max value: 30.484 - type: nauc_map_at_20_std value: 13.815 - type: nauc_map_at_20_diff1 value: 19.4513 - type: nauc_map_at_100_max value: 41.3424 - type: nauc_map_at_100_std value: 29.133300000000002 - type: nauc_map_at_100_diff1 value: 11.1852 - type: nauc_map_at_1000_max value: 47.9787 - type: nauc_map_at_1000_std value: 28.8993 - type: nauc_map_at_1000_diff1 value: 5.1209999999999996 - type: nauc_recall_at_1_max value: 8.1792 - type: nauc_recall_at_1_std value: -6.5594 - type: nauc_recall_at_1_diff1 value: 49.4009 - type: nauc_recall_at_3_max value: 18.0985 - type: nauc_recall_at_3_std value: -1.6192000000000002 - type: nauc_recall_at_3_diff1 value: 30.264200000000002 - type: nauc_recall_at_5_max value: 16.0719 - type: nauc_recall_at_5_std value: -1.7076 - type: nauc_recall_at_5_diff1 value: 27.366200000000003 - type: nauc_recall_at_10_max value: 12.537899999999999 - type: nauc_recall_at_10_std value: 0.514 - type: nauc_recall_at_10_diff1 value: 22.575799999999997 - type: nauc_recall_at_20_max value: 10.0353 - type: nauc_recall_at_20_std value: 2.8885 - type: nauc_recall_at_20_diff1 value: 21.1941 - type: nauc_recall_at_100_max value: 12.2927 - type: nauc_recall_at_100_std value: 24.7839 - type: nauc_recall_at_100_diff1 value: 22.8795 - type: nauc_recall_at_1000_max value: -12.1228 - type: nauc_recall_at_1000_std value: 71.4063 - type: nauc_recall_at_1000_diff1 value: 29.4754 - type: nauc_precision_at_1_max value: 41.396699999999996 - type: nauc_precision_at_1_std value: 32.6591 - type: nauc_precision_at_1_diff1 value: 25.3386 - type: nauc_precision_at_3_max value: 42.5837 - type: nauc_precision_at_3_std value: 30.999399999999998 - type: nauc_precision_at_3_diff1 value: -11.8464 - type: nauc_precision_at_5_max value: 37.371300000000005 - type: nauc_precision_at_5_std value: 22.783800000000003 - type: nauc_precision_at_5_diff1 value: -20.037399999999998 - type: nauc_precision_at_10_max value: 33.7087 - type: nauc_precision_at_10_std value: 22.3454 - type: nauc_precision_at_10_diff1 value: -22.0618 - type: nauc_precision_at_20_max value: 29.107499999999998 - type: nauc_precision_at_20_std value: 17.0055 - type: nauc_precision_at_20_diff1 value: -22.247500000000002 - type: nauc_precision_at_100_max value: 19.4679 - type: nauc_precision_at_100_std value: 7.725600000000001 - type: nauc_precision_at_100_diff1 value: -22.0965 - type: nauc_precision_at_1000_max value: 12.985299999999999 - type: nauc_precision_at_1000_std value: -15.038599999999999 - type: nauc_precision_at_1000_diff1 value: -22.500999999999998 - type: nauc_mrr_at_1_max value: 41.396699999999996 - type: nauc_mrr_at_1_std value: 32.6591 - type: nauc_mrr_at_1_diff1 value: 25.3386 - type: nauc_mrr_at_3_max value: 49.346000000000004 - type: nauc_mrr_at_3_std value: 42.0854 - type: nauc_mrr_at_3_diff1 value: 20.8887 - type: nauc_mrr_at_5_max value: 49.0721 - type: nauc_mrr_at_5_std value: 40.6611 - type: nauc_mrr_at_5_diff1 value: 22.686899999999998 - type: nauc_mrr_at_10_max value: 48.5527 - type: nauc_mrr_at_10_std value: 40.3495 - type: nauc_mrr_at_10_diff1 value: 23.6878 - type: nauc_mrr_at_20_max value: 48.455 - type: nauc_mrr_at_20_std value: 40.0814 - type: nauc_mrr_at_20_diff1 value: 23.9185 - type: nauc_mrr_at_100_max value: 48.3696 - type: nauc_mrr_at_100_std value: 40.065 - type: nauc_mrr_at_100_diff1 value: 23.875 - type: nauc_mrr_at_1000_max value: 48.3586 - type: nauc_mrr_at_1000_std value: 40.0662 - type: nauc_mrr_at_1000_diff1 value: 23.8532 - type: main_score value: 55.42 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: gastroenterology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 24.75 - type: ndcg_at_3 value: 23.608 - type: ndcg_at_5 value: 23.939 - type: ndcg_at_10 value: 23.983999999999998 - type: ndcg_at_20 value: 25.901999999999997 - type: ndcg_at_100 value: 32.934000000000005 - type: ndcg_at_1000 value: 45.14 - type: map_at_1 value: 2.541 - type: map_at_3 value: 5.497 - type: map_at_5 value: 7.659000000000001 - type: map_at_10 value: 9.778 - type: map_at_20 value: 11.962 - type: map_at_100 value: 16.736 - type: map_at_1000 value: 20.135 - type: recall_at_1 value: 2.541 - type: recall_at_3 value: 6.39 - type: recall_at_5 value: 10.511 - type: recall_at_10 value: 14.815999999999999 - type: recall_at_20 value: 22.067 - type: recall_at_100 value: 45.013 - type: recall_at_1000 value: 80.375 - type: precision_at_1 value: 32.0 - type: precision_at_3 value: 28.333000000000002 - type: precision_at_5 value: 27.1 - type: precision_at_10 value: 22.85 - type: precision_at_20 value: 19.625 - type: precision_at_100 value: 11.945 - type: precision_at_1000 value: 3.515 - type: mrr_at_1 value: 32.0 - type: mrr_at_3 value: 38.9167 - type: mrr_at_5 value: 40.9667 - type: mrr_at_10 value: 42.573 - type: mrr_at_20 value: 43.2429 - type: mrr_at_100 value: 43.585 - type: mrr_at_1000 value: 43.6201 - type: nauc_ndcg_at_1_max value: 38.292500000000004 - type: nauc_ndcg_at_1_std value: 31.045499999999997 - type: nauc_ndcg_at_1_diff1 value: 7.539700000000001 - type: nauc_ndcg_at_3_max value: 48.530699999999996 - type: nauc_ndcg_at_3_std value: 38.114 - type: nauc_ndcg_at_3_diff1 value: 13.930000000000001 - type: nauc_ndcg_at_5_max value: 53.109899999999996 - type: nauc_ndcg_at_5_std value: 40.4119 - type: nauc_ndcg_at_5_diff1 value: 16.4794 - type: nauc_ndcg_at_10_max value: 56.45099999999999 - type: nauc_ndcg_at_10_std value: 43.573499999999996 - type: nauc_ndcg_at_10_diff1 value: 17.793300000000002 - type: nauc_ndcg_at_20_max value: 60.3572 - type: nauc_ndcg_at_20_std value: 49.2677 - type: nauc_ndcg_at_20_diff1 value: 19.0431 - type: nauc_ndcg_at_100_max value: 58.897800000000004 - type: nauc_ndcg_at_100_std value: 58.668600000000005 - type: nauc_ndcg_at_100_diff1 value: 20.488799999999998 - type: nauc_ndcg_at_1000_max value: 58.6925 - type: nauc_ndcg_at_1000_std value: 56.21119999999999 - type: nauc_ndcg_at_1000_diff1 value: 15.9784 - type: nauc_map_at_1_max value: 30.462400000000002 - type: nauc_map_at_1_std value: 33.3978 - type: nauc_map_at_1_diff1 value: 38.5237 - type: nauc_map_at_3_max value: 46.6683 - type: nauc_map_at_3_std value: 40.834900000000005 - type: nauc_map_at_3_diff1 value: 35.3553 - type: nauc_map_at_5_max value: 51.3247 - type: nauc_map_at_5_std value: 43.430400000000006 - type: nauc_map_at_5_diff1 value: 32.006299999999996 - type: nauc_map_at_10_max value: 55.0117 - type: nauc_map_at_10_std value: 46.3951 - type: nauc_map_at_10_diff1 value: 28.022999999999996 - type: nauc_map_at_20_max value: 56.61279999999999 - type: nauc_map_at_20_std value: 50.6861 - type: nauc_map_at_20_diff1 value: 25.9926 - type: nauc_map_at_100_max value: 57.3647 - type: nauc_map_at_100_std value: 57.78 - type: nauc_map_at_100_diff1 value: 21.236 - type: nauc_map_at_1000_max value: 61.279700000000005 - type: nauc_map_at_1000_std value: 55.58030000000001 - type: nauc_map_at_1000_diff1 value: 17.4497 - type: nauc_recall_at_1_max value: 30.462400000000002 - type: nauc_recall_at_1_std value: 33.3978 - type: nauc_recall_at_1_diff1 value: 38.5237 - type: nauc_recall_at_3_max value: 46.1706 - type: nauc_recall_at_3_std value: 38.5525 - type: nauc_recall_at_3_diff1 value: 34.3394 - type: nauc_recall_at_5_max value: 43.9982 - type: nauc_recall_at_5_std value: 36.9704 - type: nauc_recall_at_5_diff1 value: 26.4609 - type: nauc_recall_at_10_max value: 48.3495 - type: nauc_recall_at_10_std value: 40.754000000000005 - type: nauc_recall_at_10_diff1 value: 21.409200000000002 - type: nauc_recall_at_20_max value: 49.7858 - type: nauc_recall_at_20_std value: 47.6663 - type: nauc_recall_at_20_diff1 value: 20.288800000000002 - type: nauc_recall_at_100_max value: 43.5882 - type: nauc_recall_at_100_std value: 60.77160000000001 - type: nauc_recall_at_100_diff1 value: 20.619699999999998 - type: nauc_recall_at_1000_max value: 57.527899999999995 - type: nauc_recall_at_1000_std value: 77.9242 - type: nauc_recall_at_1000_diff1 value: 21.8565 - type: nauc_precision_at_1_max value: 40.0045 - type: nauc_precision_at_1_std value: 28.464499999999997 - type: nauc_precision_at_1_diff1 value: 6.1918 - type: nauc_precision_at_3_max value: 50.7363 - type: nauc_precision_at_3_std value: 35.2469 - type: nauc_precision_at_3_diff1 value: 4.5440000000000005 - type: nauc_precision_at_5_max value: 55.355 - type: nauc_precision_at_5_std value: 36.7226 - type: nauc_precision_at_5_diff1 value: 4.345899999999999 - type: nauc_precision_at_10_max value: 55.1894 - type: nauc_precision_at_10_std value: 37.8104 - type: nauc_precision_at_10_diff1 value: 1.3243 - type: nauc_precision_at_20_max value: 53.0123 - type: nauc_precision_at_20_std value: 34.2509 - type: nauc_precision_at_20_diff1 value: -2.3846 - type: nauc_precision_at_100_max value: 39.6665 - type: nauc_precision_at_100_std value: 20.041 - type: nauc_precision_at_100_diff1 value: -9.078700000000001 - type: nauc_precision_at_1000_max value: 29.242099999999997 - type: nauc_precision_at_1000_std value: -17.8247 - type: nauc_precision_at_1000_diff1 value: -11.6906 - type: nauc_mrr_at_1_max value: 40.0045 - type: nauc_mrr_at_1_std value: 28.464499999999997 - type: nauc_mrr_at_1_diff1 value: 6.1918 - type: nauc_mrr_at_3_max value: 44.8685 - type: nauc_mrr_at_3_std value: 31.150299999999998 - type: nauc_mrr_at_3_diff1 value: 7.2494000000000005 - type: nauc_mrr_at_5_max value: 44.649100000000004 - type: nauc_mrr_at_5_std value: 31.2209 - type: nauc_mrr_at_5_diff1 value: 8.2678 - type: nauc_mrr_at_10_max value: 45.3643 - type: nauc_mrr_at_10_std value: 32.7704 - type: nauc_mrr_at_10_diff1 value: 7.9273 - type: nauc_mrr_at_20_max value: 45.6745 - type: nauc_mrr_at_20_std value: 32.7141 - type: nauc_mrr_at_20_diff1 value: 7.692400000000001 - type: nauc_mrr_at_100_max value: 45.4799 - type: nauc_mrr_at_100_std value: 32.4976 - type: nauc_mrr_at_100_diff1 value: 7.861700000000001 - type: nauc_mrr_at_1000_max value: 45.4435 - type: nauc_mrr_at_1000_std value: 32.462 - type: nauc_mrr_at_1000_diff1 value: 7.846 - type: main_score value: 23.983999999999998 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: gastroenterology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 33.0 - type: ndcg_at_3 value: 31.025000000000002 - type: ndcg_at_5 value: 31.290000000000003 - type: ndcg_at_10 value: 31.863999999999997 - type: ndcg_at_20 value: 33.087 - type: ndcg_at_100 value: 40.829 - type: ndcg_at_1000 value: 52.308 - type: map_at_1 value: 3.483 - type: map_at_3 value: 6.7589999999999995 - type: map_at_5 value: 9.339 - type: map_at_10 value: 12.562999999999999 - type: map_at_20 value: 15.661 - type: map_at_100 value: 22.269 - type: map_at_1000 value: 26.366 - type: recall_at_1 value: 3.483 - type: recall_at_3 value: 7.878 - type: recall_at_5 value: 12.071 - type: recall_at_10 value: 18.133 - type: recall_at_20 value: 25.778000000000002 - type: recall_at_100 value: 51.747 - type: recall_at_1000 value: 86.577 - type: precision_at_1 value: 44.5 - type: precision_at_3 value: 37.833 - type: precision_at_5 value: 36.0 - type: precision_at_10 value: 32.0 - type: precision_at_20 value: 27.250000000000004 - type: precision_at_100 value: 15.629999999999999 - type: precision_at_1000 value: 3.962 - type: mrr_at_1 value: 44.5 - type: mrr_at_3 value: 52.5 - type: mrr_at_5 value: 53.75 - type: mrr_at_10 value: 54.7087 - type: mrr_at_20 value: 55.441700000000004 - type: mrr_at_100 value: 55.7371 - type: mrr_at_1000 value: 55.7543 - type: nauc_ndcg_at_1_max value: 43.1991 - type: nauc_ndcg_at_1_std value: 21.0742 - type: nauc_ndcg_at_1_diff1 value: 25.378099999999996 - type: nauc_ndcg_at_3_max value: 48.2356 - type: nauc_ndcg_at_3_std value: 26.9982 - type: nauc_ndcg_at_3_diff1 value: 15.046000000000001 - type: nauc_ndcg_at_5_max value: 50.8549 - type: nauc_ndcg_at_5_std value: 29.8923 - type: nauc_ndcg_at_5_diff1 value: 14.678099999999999 - type: nauc_ndcg_at_10_max value: 52.5725 - type: nauc_ndcg_at_10_std value: 33.7675 - type: nauc_ndcg_at_10_diff1 value: 12.284699999999999 - type: nauc_ndcg_at_20_max value: 54.851499999999994 - type: nauc_ndcg_at_20_std value: 39.1904 - type: nauc_ndcg_at_20_diff1 value: 10.9117 - type: nauc_ndcg_at_100_max value: 58.007299999999994 - type: nauc_ndcg_at_100_std value: 52.051199999999994 - type: nauc_ndcg_at_100_diff1 value: 12.7829 - type: nauc_ndcg_at_1000_max value: 54.610099999999996 - type: nauc_ndcg_at_1000_std value: 48.904199999999996 - type: nauc_ndcg_at_1000_diff1 value: 6.1958 - type: nauc_map_at_1_max value: 12.2458 - type: nauc_map_at_1_std value: 14.7447 - type: nauc_map_at_1_diff1 value: 34.036300000000004 - type: nauc_map_at_3_max value: 23.2805 - type: nauc_map_at_3_std value: 23.075699999999998 - type: nauc_map_at_3_diff1 value: 13.449 - type: nauc_map_at_5_max value: 31.796400000000002 - type: nauc_map_at_5_std value: 28.320800000000002 - type: nauc_map_at_5_diff1 value: 11.802 - type: nauc_map_at_10_max value: 38.4408 - type: nauc_map_at_10_std value: 35.1462 - type: nauc_map_at_10_diff1 value: 11.4836 - type: nauc_map_at_20_max value: 44.432100000000005 - type: nauc_map_at_20_std value: 41.216 - type: nauc_map_at_20_diff1 value: 11.8712 - type: nauc_map_at_100_max value: 52.6408 - type: nauc_map_at_100_std value: 50.6432 - type: nauc_map_at_100_diff1 value: 10.203800000000001 - type: nauc_map_at_1000_max value: 55.83669999999999 - type: nauc_map_at_1000_std value: 50.1 - type: nauc_map_at_1000_diff1 value: 7.138999999999999 - type: nauc_recall_at_1_max value: 12.2458 - type: nauc_recall_at_1_std value: 14.7447 - type: nauc_recall_at_1_diff1 value: 34.036300000000004 - type: nauc_recall_at_3_max value: 21.1536 - type: nauc_recall_at_3_std value: 20.552200000000003 - type: nauc_recall_at_3_diff1 value: 9.8918 - type: nauc_recall_at_5_max value: 27.249000000000002 - type: nauc_recall_at_5_std value: 24.4019 - type: nauc_recall_at_5_diff1 value: 9.941899999999999 - type: nauc_recall_at_10_max value: 33.7524 - type: nauc_recall_at_10_std value: 32.2637 - type: nauc_recall_at_10_diff1 value: 9.5374 - type: nauc_recall_at_20_max value: 39.72 - type: nauc_recall_at_20_std value: 40.192699999999995 - type: nauc_recall_at_20_diff1 value: 10.4948 - type: nauc_recall_at_100_max value: 46.9723 - type: nauc_recall_at_100_std value: 57.1977 - type: nauc_recall_at_100_diff1 value: 13.5774 - type: nauc_recall_at_1000_max value: 49.7254 - type: nauc_recall_at_1000_std value: 77.2495 - type: nauc_recall_at_1000_diff1 value: -2.3819 - type: nauc_precision_at_1_max value: 52.346199999999996 - type: nauc_precision_at_1_std value: 33.2015 - type: nauc_precision_at_1_diff1 value: 23.8076 - type: nauc_precision_at_3_max value: 51.325900000000004 - type: nauc_precision_at_3_std value: 30.794 - type: nauc_precision_at_3_diff1 value: 3.4286 - type: nauc_precision_at_5_max value: 54.672 - type: nauc_precision_at_5_std value: 31.145 - type: nauc_precision_at_5_diff1 value: 3.1246 - type: nauc_precision_at_10_max value: 54.738299999999995 - type: nauc_precision_at_10_std value: 32.656800000000004 - type: nauc_precision_at_10_diff1 value: -0.0848 - type: nauc_precision_at_20_max value: 51.759699999999995 - type: nauc_precision_at_20_std value: 29.5588 - type: nauc_precision_at_20_diff1 value: -2.3040000000000003 - type: nauc_precision_at_100_max value: 36.4731 - type: nauc_precision_at_100_std value: 17.8626 - type: nauc_precision_at_100_diff1 value: -9.2071 - type: nauc_precision_at_1000_max value: 20.854400000000002 - type: nauc_precision_at_1000_std value: -12.570200000000002 - type: nauc_precision_at_1000_diff1 value: -14.3759 - type: nauc_mrr_at_1_max value: 52.346199999999996 - type: nauc_mrr_at_1_std value: 33.2015 - type: nauc_mrr_at_1_diff1 value: 23.8076 - type: nauc_mrr_at_3_max value: 52.0386 - type: nauc_mrr_at_3_std value: 35.7242 - type: nauc_mrr_at_3_diff1 value: 18.7328 - type: nauc_mrr_at_5_max value: 50.8626 - type: nauc_mrr_at_5_std value: 35.3036 - type: nauc_mrr_at_5_diff1 value: 18.8403 - type: nauc_mrr_at_10_max value: 51.0177 - type: nauc_mrr_at_10_std value: 35.2915 - type: nauc_mrr_at_10_diff1 value: 18.1387 - type: nauc_mrr_at_20_max value: 51.4232 - type: nauc_mrr_at_20_std value: 35.1578 - type: nauc_mrr_at_20_diff1 value: 17.8418 - type: nauc_mrr_at_100_max value: 51.5004 - type: nauc_mrr_at_100_std value: 35.3373 - type: nauc_mrr_at_100_diff1 value: 17.9748 - type: nauc_mrr_at_1000_max value: 51.4899 - type: nauc_mrr_at_1000_std value: 35.3187 - type: nauc_mrr_at_1000_diff1 value: 18.0391 - type: main_score value: 31.863999999999997 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: genetics type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 60.25 - type: ndcg_at_3 value: 58.475 - type: ndcg_at_5 value: 57.792 - type: ndcg_at_10 value: 57.830000000000005 - type: ndcg_at_20 value: 58.226 - type: ndcg_at_100 value: 62.902 - type: ndcg_at_1000 value: 71.911 - type: map_at_1 value: 9.786999999999999 - type: map_at_3 value: 18.359 - type: map_at_5 value: 22.716 - type: map_at_10 value: 28.389999999999997 - type: map_at_20 value: 32.991 - type: map_at_100 value: 42.074 - type: map_at_1000 value: 48.083999999999996 - type: recall_at_1 value: 9.786999999999999 - type: recall_at_3 value: 20.128 - type: recall_at_5 value: 26.615 - type: recall_at_10 value: 35.961999999999996 - type: recall_at_20 value: 44.772 - type: recall_at_100 value: 68.51899999999999 - type: recall_at_1000 value: 93.538 - type: precision_at_1 value: 74.0 - type: precision_at_3 value: 65.167 - type: precision_at_5 value: 60.099999999999994 - type: precision_at_10 value: 50.8 - type: precision_at_20 value: 41.25 - type: precision_at_100 value: 22.07 - type: precision_at_1000 value: 5.579 - type: mrr_at_1 value: 74.0 - type: mrr_at_3 value: 82.25 - type: mrr_at_5 value: 83.175 - type: mrr_at_10 value: 83.3589 - type: mrr_at_20 value: 83.52340000000001 - type: mrr_at_100 value: 83.52340000000001 - type: mrr_at_1000 value: 83.52340000000001 - type: nauc_ndcg_at_1_max value: 23.397299999999998 - type: nauc_ndcg_at_1_std value: 24.4939 - type: nauc_ndcg_at_1_diff1 value: 28.0647 - type: nauc_ndcg_at_3_max value: 32.109500000000004 - type: nauc_ndcg_at_3_std value: 24.8319 - type: nauc_ndcg_at_3_diff1 value: 21.9448 - type: nauc_ndcg_at_5_max value: 34.0051 - type: nauc_ndcg_at_5_std value: 27.4148 - type: nauc_ndcg_at_5_diff1 value: 18.6325 - type: nauc_ndcg_at_10_max value: 31.6224 - type: nauc_ndcg_at_10_std value: 31.2952 - type: nauc_ndcg_at_10_diff1 value: 17.426099999999998 - type: nauc_ndcg_at_20_max value: 33.1613 - type: nauc_ndcg_at_20_std value: 33.711400000000005 - type: nauc_ndcg_at_20_diff1 value: 22.2519 - type: nauc_ndcg_at_100_max value: 32.222699999999996 - type: nauc_ndcg_at_100_std value: 40.532000000000004 - type: nauc_ndcg_at_100_diff1 value: 29.092299999999998 - type: nauc_ndcg_at_1000_max value: 34.9461 - type: nauc_ndcg_at_1000_std value: 41.908899999999996 - type: nauc_ndcg_at_1000_diff1 value: 26.2818 - type: nauc_map_at_1_max value: 1.7859 - type: nauc_map_at_1_std value: 9.931099999999999 - type: nauc_map_at_1_diff1 value: 29.546499999999998 - type: nauc_map_at_3_max value: 16.9801 - type: nauc_map_at_3_std value: 17.0517 - type: nauc_map_at_3_diff1 value: 30.8838 - type: nauc_map_at_5_max value: 21.2562 - type: nauc_map_at_5_std value: 24.3807 - type: nauc_map_at_5_diff1 value: 30.370200000000004 - type: nauc_map_at_10_max value: 24.388399999999997 - type: nauc_map_at_10_std value: 30.231599999999997 - type: nauc_map_at_10_diff1 value: 26.3432 - type: nauc_map_at_20_max value: 25.8334 - type: nauc_map_at_20_std value: 33.4471 - type: nauc_map_at_20_diff1 value: 26.217200000000002 - type: nauc_map_at_100_max value: 26.246799999999997 - type: nauc_map_at_100_std value: 40.078399999999995 - type: nauc_map_at_100_diff1 value: 25.5407 - type: nauc_map_at_1000_max value: 29.2257 - type: nauc_map_at_1000_std value: 39.4688 - type: nauc_map_at_1000_diff1 value: 19.9766 - type: nauc_recall_at_1_max value: 1.7859 - type: nauc_recall_at_1_std value: 9.931099999999999 - type: nauc_recall_at_1_diff1 value: 29.546499999999998 - type: nauc_recall_at_3_max value: 16.4105 - type: nauc_recall_at_3_std value: 16.220200000000002 - type: nauc_recall_at_3_diff1 value: 26.8932 - type: nauc_recall_at_5_max value: 18.6335 - type: nauc_recall_at_5_std value: 26.1909 - type: nauc_recall_at_5_diff1 value: 25.570999999999998 - type: nauc_recall_at_10_max value: 19.8384 - type: nauc_recall_at_10_std value: 30.0377 - type: nauc_recall_at_10_diff1 value: 20.7945 - type: nauc_recall_at_20_max value: 20.6784 - type: nauc_recall_at_20_std value: 33.7626 - type: nauc_recall_at_20_diff1 value: 22.6844 - type: nauc_recall_at_100_max value: 16.7405 - type: nauc_recall_at_100_std value: 47.7157 - type: nauc_recall_at_100_diff1 value: 25.9606 - type: nauc_recall_at_1000_max value: 35.3439 - type: nauc_recall_at_1000_std value: 78.67 - type: nauc_recall_at_1000_diff1 value: 27.145799999999998 - type: nauc_precision_at_1_max value: 27.4857 - type: nauc_precision_at_1_std value: 32.4166 - type: nauc_precision_at_1_diff1 value: 35.327 - type: nauc_precision_at_3_max value: 30.1426 - type: nauc_precision_at_3_std value: 22.7352 - type: nauc_precision_at_3_diff1 value: 6.823899999999999 - type: nauc_precision_at_5_max value: 30.236200000000004 - type: nauc_precision_at_5_std value: 20.993100000000002 - type: nauc_precision_at_5_diff1 value: -3.7901 - type: nauc_precision_at_10_max value: 17.6751 - type: nauc_precision_at_10_std value: 13.324300000000001 - type: nauc_precision_at_10_diff1 value: -18.4944 - type: nauc_precision_at_20_max value: 7.290099999999999 - type: nauc_precision_at_20_std value: 3.0109 - type: nauc_precision_at_20_diff1 value: -19.5427 - type: nauc_precision_at_100_max value: -1.3945 - type: nauc_precision_at_100_std value: -11.3057 - type: nauc_precision_at_100_diff1 value: -23.4941 - type: nauc_precision_at_1000_max value: -0.0958 - type: nauc_precision_at_1000_std value: -25.0461 - type: nauc_precision_at_1000_diff1 value: -29.7531 - type: nauc_mrr_at_1_max value: 27.4857 - type: nauc_mrr_at_1_std value: 32.4166 - type: nauc_mrr_at_1_diff1 value: 35.327 - type: nauc_mrr_at_3_max value: 33.575700000000005 - type: nauc_mrr_at_3_std value: 33.7427 - type: nauc_mrr_at_3_diff1 value: 39.4644 - type: nauc_mrr_at_5_max value: 33.331 - type: nauc_mrr_at_5_std value: 33.8148 - type: nauc_mrr_at_5_diff1 value: 38.4811 - type: nauc_mrr_at_10_max value: 33.583800000000004 - type: nauc_mrr_at_10_std value: 33.9609 - type: nauc_mrr_at_10_diff1 value: 38.217 - type: nauc_mrr_at_20_max value: 33.021 - type: nauc_mrr_at_20_std value: 34.0802 - type: nauc_mrr_at_20_diff1 value: 38.281 - type: nauc_mrr_at_100_max value: 33.021 - type: nauc_mrr_at_100_std value: 34.0802 - type: nauc_mrr_at_100_diff1 value: 38.281 - type: nauc_mrr_at_1000_max value: 33.021 - type: nauc_mrr_at_1000_std value: 34.0802 - type: nauc_mrr_at_1000_diff1 value: 38.281 - type: main_score value: 57.830000000000005 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: genetics type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 20.5 - type: ndcg_at_3 value: 20.846 - type: ndcg_at_5 value: 21.898999999999997 - type: ndcg_at_10 value: 23.427999999999997 - type: ndcg_at_20 value: 25.402 - type: ndcg_at_100 value: 29.868 - type: ndcg_at_1000 value: 39.879999999999995 - type: map_at_1 value: 4.573 - type: map_at_3 value: 7.89 - type: map_at_5 value: 9.804 - type: map_at_10 value: 12.147 - type: map_at_20 value: 13.908999999999999 - type: map_at_100 value: 16.784 - type: map_at_1000 value: 19.277 - type: recall_at_1 value: 4.573 - type: recall_at_3 value: 9.396 - type: recall_at_5 value: 13.635 - type: recall_at_10 value: 20.383000000000003 - type: recall_at_20 value: 26.56 - type: recall_at_100 value: 42.141 - type: recall_at_1000 value: 69.513 - type: precision_at_1 value: 26.0 - type: precision_at_3 value: 24.333 - type: precision_at_5 value: 23.0 - type: precision_at_10 value: 18.9 - type: precision_at_20 value: 15.975 - type: precision_at_100 value: 8.790000000000001 - type: precision_at_1000 value: 2.919 - type: mrr_at_1 value: 26.0 - type: mrr_at_3 value: 34.1667 - type: mrr_at_5 value: 36.3917 - type: mrr_at_10 value: 37.5268 - type: mrr_at_20 value: 38.1037 - type: mrr_at_100 value: 38.2909 - type: mrr_at_1000 value: 38.3666 - type: nauc_ndcg_at_1_max value: 30.9651 - type: nauc_ndcg_at_1_std value: 26.2865 - type: nauc_ndcg_at_1_diff1 value: 5.3133 - type: nauc_ndcg_at_3_max value: 34.8346 - type: nauc_ndcg_at_3_std value: 31.572699999999998 - type: nauc_ndcg_at_3_diff1 value: 4.2997 - type: nauc_ndcg_at_5_max value: 38.6848 - type: nauc_ndcg_at_5_std value: 34.4997 - type: nauc_ndcg_at_5_diff1 value: 8.145 - type: nauc_ndcg_at_10_max value: 39.4422 - type: nauc_ndcg_at_10_std value: 42.065000000000005 - type: nauc_ndcg_at_10_diff1 value: 11.3805 - type: nauc_ndcg_at_20_max value: 40.2946 - type: nauc_ndcg_at_20_std value: 45.984199999999994 - type: nauc_ndcg_at_20_diff1 value: 13.3306 - type: nauc_ndcg_at_100_max value: 40.8998 - type: nauc_ndcg_at_100_std value: 52.7489 - type: nauc_ndcg_at_100_diff1 value: 11.8633 - type: nauc_ndcg_at_1000_max value: 46.3694 - type: nauc_ndcg_at_1000_std value: 56.346799999999995 - type: nauc_ndcg_at_1000_diff1 value: 3.6907 - type: nauc_map_at_1_max value: 38.6937 - type: nauc_map_at_1_std value: 30.1883 - type: nauc_map_at_1_diff1 value: 35.4755 - type: nauc_map_at_3_max value: 37.1634 - type: nauc_map_at_3_std value: 39.1229 - type: nauc_map_at_3_diff1 value: 24.429000000000002 - type: nauc_map_at_5_max value: 35.7377 - type: nauc_map_at_5_std value: 41.4592 - type: nauc_map_at_5_diff1 value: 24.508399999999998 - type: nauc_map_at_10_max value: 35.7061 - type: nauc_map_at_10_std value: 44.7305 - type: nauc_map_at_10_diff1 value: 21.0847 - type: nauc_map_at_20_max value: 36.3037 - type: nauc_map_at_20_std value: 47.369299999999996 - type: nauc_map_at_20_diff1 value: 19.8797 - type: nauc_map_at_100_max value: 38.423 - type: nauc_map_at_100_std value: 50.9625 - type: nauc_map_at_100_diff1 value: 17.043400000000002 - type: nauc_map_at_1000_max value: 41.4598 - type: nauc_map_at_1000_std value: 52.303 - type: nauc_map_at_1000_diff1 value: 13.9572 - type: nauc_recall_at_1_max value: 38.6937 - type: nauc_recall_at_1_std value: 30.1883 - type: nauc_recall_at_1_diff1 value: 35.4755 - type: nauc_recall_at_3_max value: 32.7829 - type: nauc_recall_at_3_std value: 37.202200000000005 - type: nauc_recall_at_3_diff1 value: 22.7994 - type: nauc_recall_at_5_max value: 28.983199999999997 - type: nauc_recall_at_5_std value: 39.1822 - type: nauc_recall_at_5_diff1 value: 25.370900000000002 - type: nauc_recall_at_10_max value: 25.569599999999998 - type: nauc_recall_at_10_std value: 44.183 - type: nauc_recall_at_10_diff1 value: 18.7259 - type: nauc_recall_at_20_max value: 24.8139 - type: nauc_recall_at_20_std value: 48.7119 - type: nauc_recall_at_20_diff1 value: 19.9204 - type: nauc_recall_at_100_max value: 28.1375 - type: nauc_recall_at_100_std value: 58.6694 - type: nauc_recall_at_100_diff1 value: 14.3897 - type: nauc_recall_at_1000_max value: 39.3016 - type: nauc_recall_at_1000_std value: 80.6024 - type: nauc_recall_at_1000_diff1 value: -1.5233 - type: nauc_precision_at_1_max value: 33.3634 - type: nauc_precision_at_1_std value: 29.862499999999997 - type: nauc_precision_at_1_diff1 value: 4.5995 - type: nauc_precision_at_3_max value: 34.5883 - type: nauc_precision_at_3_std value: 33.4493 - type: nauc_precision_at_3_diff1 value: -6.5379000000000005 - type: nauc_precision_at_5_max value: 37.2304 - type: nauc_precision_at_5_std value: 34.462399999999995 - type: nauc_precision_at_5_diff1 value: -5.7752 - type: nauc_precision_at_10_max value: 37.0948 - type: nauc_precision_at_10_std value: 35.8302 - type: nauc_precision_at_10_diff1 value: -4.7827 - type: nauc_precision_at_20_max value: 37.642900000000004 - type: nauc_precision_at_20_std value: 30.4822 - type: nauc_precision_at_20_diff1 value: -7.4448 - type: nauc_precision_at_100_max value: 32.3888 - type: nauc_precision_at_100_std value: 16.284100000000002 - type: nauc_precision_at_100_diff1 value: -13.534299999999998 - type: nauc_precision_at_1000_max value: 29.5665 - type: nauc_precision_at_1000_std value: -4.736 - type: nauc_precision_at_1000_diff1 value: -14.4178 - type: nauc_mrr_at_1_max value: 33.3634 - type: nauc_mrr_at_1_std value: 29.862499999999997 - type: nauc_mrr_at_1_diff1 value: 4.5995 - type: nauc_mrr_at_3_max value: 35.8166 - type: nauc_mrr_at_3_std value: 30.7058 - type: nauc_mrr_at_3_diff1 value: 2.931 - type: nauc_mrr_at_5_max value: 37.0172 - type: nauc_mrr_at_5_std value: 30.959999999999997 - type: nauc_mrr_at_5_diff1 value: 3.8329000000000004 - type: nauc_mrr_at_10_max value: 38.2552 - type: nauc_mrr_at_10_std value: 32.524300000000004 - type: nauc_mrr_at_10_diff1 value: 3.6144999999999996 - type: nauc_mrr_at_20_max value: 38.2202 - type: nauc_mrr_at_20_std value: 32.5916 - type: nauc_mrr_at_20_diff1 value: 3.0888999999999998 - type: nauc_mrr_at_100_max value: 38.2367 - type: nauc_mrr_at_100_std value: 32.5874 - type: nauc_mrr_at_100_diff1 value: 3.0557999999999996 - type: nauc_mrr_at_1000_max value: 38.2161 - type: nauc_mrr_at_1000_std value: 32.5373 - type: nauc_mrr_at_1000_diff1 value: 3.0961 - type: main_score value: 23.427999999999997 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: genetics type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 34.75 - type: ndcg_at_3 value: 33.451 - type: ndcg_at_5 value: 33.062000000000005 - type: ndcg_at_10 value: 34.393 - type: ndcg_at_20 value: 34.725 - type: ndcg_at_100 value: 39.509 - type: ndcg_at_1000 value: 49.708999999999996 - type: map_at_1 value: 6.478000000000001 - type: map_at_3 value: 11.562 - type: map_at_5 value: 13.883999999999999 - type: map_at_10 value: 16.830000000000002 - type: map_at_20 value: 18.719 - type: map_at_100 value: 22.726 - type: map_at_1000 value: 26.284999999999997 - type: recall_at_1 value: 6.478000000000001 - type: recall_at_3 value: 13.05 - type: recall_at_5 value: 17.217 - type: recall_at_10 value: 24.532999999999998 - type: recall_at_20 value: 29.921999999999997 - type: recall_at_100 value: 48.243 - type: recall_at_1000 value: 77.143 - type: precision_at_1 value: 44.0 - type: precision_at_3 value: 37.5 - type: precision_at_5 value: 32.6 - type: precision_at_10 value: 27.800000000000004 - type: precision_at_20 value: 22.15 - type: precision_at_100 value: 13.020000000000001 - type: precision_at_1000 value: 4.009 - type: mrr_at_1 value: 44.0 - type: mrr_at_3 value: 52.416700000000006 - type: mrr_at_5 value: 53.8917 - type: mrr_at_10 value: 55.232099999999996 - type: mrr_at_20 value: 55.5794 - type: mrr_at_100 value: 55.7582 - type: mrr_at_1000 value: 55.7889 - type: nauc_ndcg_at_1_max value: 36.268 - type: nauc_ndcg_at_1_std value: 27.228099999999998 - type: nauc_ndcg_at_1_diff1 value: 24.5667 - type: nauc_ndcg_at_3_max value: 33.9523 - type: nauc_ndcg_at_3_std value: 30.5914 - type: nauc_ndcg_at_3_diff1 value: 22.4058 - type: nauc_ndcg_at_5_max value: 36.106500000000004 - type: nauc_ndcg_at_5_std value: 36.521300000000004 - type: nauc_ndcg_at_5_diff1 value: 23.0627 - type: nauc_ndcg_at_10_max value: 39.3899 - type: nauc_ndcg_at_10_std value: 45.009100000000004 - type: nauc_ndcg_at_10_diff1 value: 30.0523 - type: nauc_ndcg_at_20_max value: 39.0796 - type: nauc_ndcg_at_20_std value: 48.718 - type: nauc_ndcg_at_20_diff1 value: 31.4393 - type: nauc_ndcg_at_100_max value: 41.5535 - type: nauc_ndcg_at_100_std value: 59.6568 - type: nauc_ndcg_at_100_diff1 value: 31.8814 - type: nauc_ndcg_at_1000_max value: 48.430299999999995 - type: nauc_ndcg_at_1000_std value: 60.9746 - type: nauc_ndcg_at_1000_diff1 value: 30.778499999999998 - type: nauc_map_at_1_max value: 21.8143 - type: nauc_map_at_1_std value: 39.863 - type: nauc_map_at_1_diff1 value: 38.9446 - type: nauc_map_at_3_max value: 23.871000000000002 - type: nauc_map_at_3_std value: 42.3217 - type: nauc_map_at_3_diff1 value: 34.5713 - type: nauc_map_at_5_max value: 22.4625 - type: nauc_map_at_5_std value: 44.1124 - type: nauc_map_at_5_diff1 value: 31.431199999999997 - type: nauc_map_at_10_max value: 24.9392 - type: nauc_map_at_10_std value: 46.123999999999995 - type: nauc_map_at_10_diff1 value: 32.6507 - type: nauc_map_at_20_max value: 26.084400000000002 - type: nauc_map_at_20_std value: 47.902 - type: nauc_map_at_20_diff1 value: 32.002900000000004 - type: nauc_map_at_100_max value: 30.115399999999998 - type: nauc_map_at_100_std value: 52.4305 - type: nauc_map_at_100_diff1 value: 32.242 - type: nauc_map_at_1000_max value: 35.355199999999996 - type: nauc_map_at_1000_std value: 52.38270000000001 - type: nauc_map_at_1000_diff1 value: 29.8275 - type: nauc_recall_at_1_max value: 21.8143 - type: nauc_recall_at_1_std value: 39.863 - type: nauc_recall_at_1_diff1 value: 38.9446 - type: nauc_recall_at_3_max value: 22.8151 - type: nauc_recall_at_3_std value: 43.5837 - type: nauc_recall_at_3_diff1 value: 31.1201 - type: nauc_recall_at_5_max value: 19.276699999999998 - type: nauc_recall_at_5_std value: 46.0849 - type: nauc_recall_at_5_diff1 value: 26.518900000000002 - type: nauc_recall_at_10_max value: 25.4214 - type: nauc_recall_at_10_std value: 52.4193 - type: nauc_recall_at_10_diff1 value: 32.5876 - type: nauc_recall_at_20_max value: 26.7202 - type: nauc_recall_at_20_std value: 57.550000000000004 - type: nauc_recall_at_20_diff1 value: 30.7318 - type: nauc_recall_at_100_max value: 32.1892 - type: nauc_recall_at_100_std value: 71.679 - type: nauc_recall_at_100_diff1 value: 28.494500000000002 - type: nauc_recall_at_1000_max value: 53.0749 - type: nauc_recall_at_1000_std value: 86.1305 - type: nauc_recall_at_1000_diff1 value: 26.4679 - type: nauc_precision_at_1_max value: 36.2642 - type: nauc_precision_at_1_std value: 29.8246 - type: nauc_precision_at_1_diff1 value: 24.369699999999998 - type: nauc_precision_at_3_max value: 33.6592 - type: nauc_precision_at_3_std value: 24.313000000000002 - type: nauc_precision_at_3_diff1 value: 11.6768 - type: nauc_precision_at_5_max value: 35.1515 - type: nauc_precision_at_5_std value: 23.6793 - type: nauc_precision_at_5_diff1 value: 10.2316 - type: nauc_precision_at_10_max value: 34.3044 - type: nauc_precision_at_10_std value: 20.2514 - type: nauc_precision_at_10_diff1 value: 12.906 - type: nauc_precision_at_20_max value: 31.694699999999997 - type: nauc_precision_at_20_std value: 12.6698 - type: nauc_precision_at_20_diff1 value: 7.758800000000001 - type: nauc_precision_at_100_max value: 24.8188 - type: nauc_precision_at_100_std value: -1.4749999999999999 - type: nauc_precision_at_100_diff1 value: -4.7867 - type: nauc_precision_at_1000_max value: 18.5152 - type: nauc_precision_at_1000_std value: -21.6261 - type: nauc_precision_at_1000_diff1 value: -13.8974 - type: nauc_mrr_at_1_max value: 36.2642 - type: nauc_mrr_at_1_std value: 29.8246 - type: nauc_mrr_at_1_diff1 value: 24.369699999999998 - type: nauc_mrr_at_3_max value: 38.6868 - type: nauc_mrr_at_3_std value: 34.336 - type: nauc_mrr_at_3_diff1 value: 20.794999999999998 - type: nauc_mrr_at_5_max value: 39.9258 - type: nauc_mrr_at_5_std value: 36.4033 - type: nauc_mrr_at_5_diff1 value: 21.2071 - type: nauc_mrr_at_10_max value: 41.1203 - type: nauc_mrr_at_10_std value: 38.0702 - type: nauc_mrr_at_10_diff1 value: 22.213 - type: nauc_mrr_at_20_max value: 41.1365 - type: nauc_mrr_at_20_std value: 37.7203 - type: nauc_mrr_at_20_diff1 value: 22.2009 - type: nauc_mrr_at_100_max value: 41.0795 - type: nauc_mrr_at_100_std value: 37.574400000000004 - type: nauc_mrr_at_100_diff1 value: 22.298000000000002 - type: nauc_mrr_at_1000_max value: 41.0355 - type: nauc_mrr_at_1000_std value: 37.528800000000004 - type: nauc_mrr_at_1000_diff1 value: 22.2988 - type: main_score value: 34.393 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: neuroscience_and_neurology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 54.50000000000001 - type: ndcg_at_3 value: 51.083999999999996 - type: ndcg_at_5 value: 50.954 - type: ndcg_at_10 value: 48.884 - type: ndcg_at_20 value: 49.376 - type: ndcg_at_100 value: 57.452999999999996 - type: ndcg_at_1000 value: 67.477 - type: map_at_1 value: 5.471 - type: map_at_3 value: 10.197000000000001 - type: map_at_5 value: 13.496 - type: map_at_10 value: 18.206 - type: map_at_20 value: 23.843 - type: map_at_100 value: 35.476 - type: map_at_1000 value: 40.632000000000005 - type: recall_at_1 value: 5.471 - type: recall_at_3 value: 11.132 - type: recall_at_5 value: 16.394000000000002 - type: recall_at_10 value: 24.582 - type: recall_at_20 value: 36.967 - type: recall_at_100 value: 70.206 - type: recall_at_1000 value: 96.883 - type: precision_at_1 value: 69.5 - type: precision_at_3 value: 61.833000000000006 - type: precision_at_5 value: 58.3 - type: precision_at_10 value: 49.35 - type: precision_at_20 value: 41.525 - type: precision_at_100 value: 22.0 - type: precision_at_1000 value: 3.977 - type: mrr_at_1 value: 69.5 - type: mrr_at_3 value: 76.9167 - type: mrr_at_5 value: 78.1167 - type: mrr_at_10 value: 78.5228 - type: mrr_at_20 value: 78.6442 - type: mrr_at_100 value: 78.7209 - type: mrr_at_1000 value: 78.7261 - type: nauc_ndcg_at_1_max value: 39.2904 - type: nauc_ndcg_at_1_std value: 26.1804 - type: nauc_ndcg_at_1_diff1 value: 15.9502 - type: nauc_ndcg_at_3_max value: 45.1527 - type: nauc_ndcg_at_3_std value: 29.781000000000002 - type: nauc_ndcg_at_3_diff1 value: 12.642999999999999 - type: nauc_ndcg_at_5_max value: 44.426700000000004 - type: nauc_ndcg_at_5_std value: 28.3707 - type: nauc_ndcg_at_5_diff1 value: 12.6422 - type: nauc_ndcg_at_10_max value: 46.7505 - type: nauc_ndcg_at_10_std value: 28.2544 - type: nauc_ndcg_at_10_diff1 value: 17.0276 - type: nauc_ndcg_at_20_max value: 47.1505 - type: nauc_ndcg_at_20_std value: 24.3433 - type: nauc_ndcg_at_20_diff1 value: 16.1224 - type: nauc_ndcg_at_100_max value: 52.66309999999999 - type: nauc_ndcg_at_100_std value: 27.4478 - type: nauc_ndcg_at_100_diff1 value: 16.6075 - type: nauc_ndcg_at_1000_max value: 56.2764 - type: nauc_ndcg_at_1000_std value: 40.754000000000005 - type: nauc_ndcg_at_1000_diff1 value: 15.6926 - type: nauc_map_at_1_max value: 3.0124999999999997 - type: nauc_map_at_1_std value: -8.2124 - type: nauc_map_at_1_diff1 value: 10.987 - type: nauc_map_at_3_max value: 19.0176 - type: nauc_map_at_3_std value: -10.0504 - type: nauc_map_at_3_diff1 value: 13.661200000000001 - type: nauc_map_at_5_max value: 20.0561 - type: nauc_map_at_5_std value: -8.7265 - type: nauc_map_at_5_diff1 value: 13.9652 - type: nauc_map_at_10_max value: 26.4123 - type: nauc_map_at_10_std value: -3.6491000000000002 - type: nauc_map_at_10_diff1 value: 16.9533 - type: nauc_map_at_20_max value: 33.7953 - type: nauc_map_at_20_std value: 1.6022999999999998 - type: nauc_map_at_20_diff1 value: 16.6287 - type: nauc_map_at_100_max value: 48.194900000000004 - type: nauc_map_at_100_std value: 20.2344 - type: nauc_map_at_100_diff1 value: 16.5675 - type: nauc_map_at_1000_max value: 51.1486 - type: nauc_map_at_1000_std value: 27.487000000000002 - type: nauc_map_at_1000_diff1 value: 12.745000000000001 - type: nauc_recall_at_1_max value: 3.0124999999999997 - type: nauc_recall_at_1_std value: -8.2124 - type: nauc_recall_at_1_diff1 value: 10.987 - type: nauc_recall_at_3_max value: 14.3101 - type: nauc_recall_at_3_std value: -10.6097 - type: nauc_recall_at_3_diff1 value: 9.5236 - type: nauc_recall_at_5_max value: 7.350700000000001 - type: nauc_recall_at_5_std value: -13.6181 - type: nauc_recall_at_5_diff1 value: 7.914599999999999 - type: nauc_recall_at_10_max value: 10.7259 - type: nauc_recall_at_10_std value: -11.0079 - type: nauc_recall_at_10_diff1 value: 15.920599999999999 - type: nauc_recall_at_20_max value: 13.755899999999999 - type: nauc_recall_at_20_std value: -10.1572 - type: nauc_recall_at_20_diff1 value: 14.7261 - type: nauc_recall_at_100_max value: 25.160700000000002 - type: nauc_recall_at_100_std value: 4.3699 - type: nauc_recall_at_100_diff1 value: 12.0403 - type: nauc_recall_at_1000_max value: 47.7286 - type: nauc_recall_at_1000_std value: 51.61750000000001 - type: nauc_recall_at_1000_diff1 value: 9.2401 - type: nauc_precision_at_1_max value: 47.9257 - type: nauc_precision_at_1_std value: 23.233 - type: nauc_precision_at_1_diff1 value: 25.5884 - type: nauc_precision_at_3_max value: 46.5061 - type: nauc_precision_at_3_std value: 27.938800000000004 - type: nauc_precision_at_3_diff1 value: 14.1398 - type: nauc_precision_at_5_max value: 40.203 - type: nauc_precision_at_5_std value: 28.471400000000003 - type: nauc_precision_at_5_diff1 value: 10.7772 - type: nauc_precision_at_10_max value: 39.1491 - type: nauc_precision_at_10_std value: 31.4945 - type: nauc_precision_at_10_diff1 value: 12.3007 - type: nauc_precision_at_20_max value: 34.4246 - type: nauc_precision_at_20_std value: 29.8628 - type: nauc_precision_at_20_diff1 value: 2.8098 - type: nauc_precision_at_100_max value: 21.1322 - type: nauc_precision_at_100_std value: 29.6973 - type: nauc_precision_at_100_diff1 value: -6.4503 - type: nauc_precision_at_1000_max value: 5.7992 - type: nauc_precision_at_1000_std value: 13.969599999999998 - type: nauc_precision_at_1000_diff1 value: -11.9987 - type: nauc_mrr_at_1_max value: 47.9257 - type: nauc_mrr_at_1_std value: 23.233 - type: nauc_mrr_at_1_diff1 value: 25.5884 - type: nauc_mrr_at_3_max value: 49.356899999999996 - type: nauc_mrr_at_3_std value: 31.1402 - type: nauc_mrr_at_3_diff1 value: 24.6431 - type: nauc_mrr_at_5_max value: 49.8243 - type: nauc_mrr_at_5_std value: 29.1812 - type: nauc_mrr_at_5_diff1 value: 26.9064 - type: nauc_mrr_at_10_max value: 49.7113 - type: nauc_mrr_at_10_std value: 29.296 - type: nauc_mrr_at_10_diff1 value: 27.2803 - type: nauc_mrr_at_20_max value: 49.613099999999996 - type: nauc_mrr_at_20_std value: 29.137200000000004 - type: nauc_mrr_at_20_diff1 value: 27.3154 - type: nauc_mrr_at_100_max value: 49.613099999999996 - type: nauc_mrr_at_100_std value: 29.0941 - type: nauc_mrr_at_100_diff1 value: 27.1667 - type: nauc_mrr_at_1000_max value: 49.606899999999996 - type: nauc_mrr_at_1000_std value: 29.082900000000002 - type: nauc_mrr_at_1000_diff1 value: 27.151999999999997 - type: main_score value: 48.884 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: neuroscience_and_neurology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 18.5 - type: ndcg_at_3 value: 15.914 - type: ndcg_at_5 value: 15.717 - type: ndcg_at_10 value: 15.190000000000001 - type: ndcg_at_20 value: 15.901000000000002 - type: ndcg_at_100 value: 21.93 - type: ndcg_at_1000 value: 35.305 - type: map_at_1 value: 1.764 - type: map_at_3 value: 3.2620000000000005 - type: map_at_5 value: 4.166 - type: map_at_10 value: 5.16 - type: map_at_20 value: 6.539000000000001 - type: map_at_100 value: 10.008000000000001 - type: map_at_1000 value: 12.806999999999999 - type: recall_at_1 value: 1.764 - type: recall_at_3 value: 3.585 - type: recall_at_5 value: 5.311 - type: recall_at_10 value: 7.692 - type: recall_at_20 value: 12.034 - type: recall_at_100 value: 30.930999999999997 - type: recall_at_1000 value: 69.574 - type: precision_at_1 value: 26.5 - type: precision_at_3 value: 20.333000000000002 - type: precision_at_5 value: 19.3 - type: precision_at_10 value: 16.35 - type: precision_at_20 value: 14.000000000000002 - type: precision_at_100 value: 8.985 - type: precision_at_1000 value: 2.617 - type: mrr_at_1 value: 26.5 - type: mrr_at_3 value: 30.8333 - type: mrr_at_5 value: 32.2833 - type: mrr_at_10 value: 33.2004 - type: mrr_at_20 value: 34.037800000000004 - type: mrr_at_100 value: 34.5533 - type: mrr_at_1000 value: 34.6277 - type: nauc_ndcg_at_1_max value: 20.1184 - type: nauc_ndcg_at_1_std value: 38.5704 - type: nauc_ndcg_at_1_diff1 value: 5.0864 - type: nauc_ndcg_at_3_max value: 36.4827 - type: nauc_ndcg_at_3_std value: 48.3948 - type: nauc_ndcg_at_3_diff1 value: 5.2124999999999995 - type: nauc_ndcg_at_5_max value: 37.5982 - type: nauc_ndcg_at_5_std value: 50.1478 - type: nauc_ndcg_at_5_diff1 value: 4.7903 - type: nauc_ndcg_at_10_max value: 38.5586 - type: nauc_ndcg_at_10_std value: 49.945299999999996 - type: nauc_ndcg_at_10_diff1 value: 3.0567 - type: nauc_ndcg_at_20_max value: 40.3095 - type: nauc_ndcg_at_20_std value: 50.8079 - type: nauc_ndcg_at_20_diff1 value: 2.9066 - type: nauc_ndcg_at_100_max value: 44.5802 - type: nauc_ndcg_at_100_std value: 56.85059999999999 - type: nauc_ndcg_at_100_diff1 value: 0.6169 - type: nauc_ndcg_at_1000_max value: 50.469500000000004 - type: nauc_ndcg_at_1000_std value: 64.4092 - type: nauc_ndcg_at_1000_diff1 value: 4.1708 - type: nauc_map_at_1_max value: 30.6966 - type: nauc_map_at_1_std value: 24.0367 - type: nauc_map_at_1_diff1 value: 5.1290000000000004 - type: nauc_map_at_3_max value: 49.699 - type: nauc_map_at_3_std value: 39.0809 - type: nauc_map_at_3_diff1 value: 17.871000000000002 - type: nauc_map_at_5_max value: 48.4367 - type: nauc_map_at_5_std value: 38.408 - type: nauc_map_at_5_diff1 value: 15.0561 - type: nauc_map_at_10_max value: 47.6762 - type: nauc_map_at_10_std value: 41.3061 - type: nauc_map_at_10_diff1 value: 11.0321 - type: nauc_map_at_20_max value: 47.9184 - type: nauc_map_at_20_std value: 45.1168 - type: nauc_map_at_20_diff1 value: 7.9868 - type: nauc_map_at_100_max value: 46.889199999999995 - type: nauc_map_at_100_std value: 53.928200000000004 - type: nauc_map_at_100_diff1 value: 3.3692 - type: nauc_map_at_1000_max value: 49.5583 - type: nauc_map_at_1000_std value: 58.64470000000001 - type: nauc_map_at_1000_diff1 value: 3.9248 - type: nauc_recall_at_1_max value: 30.6966 - type: nauc_recall_at_1_std value: 24.0367 - type: nauc_recall_at_1_diff1 value: 5.1290000000000004 - type: nauc_recall_at_3_max value: 50.294000000000004 - type: nauc_recall_at_3_std value: 38.821600000000004 - type: nauc_recall_at_3_diff1 value: 14.833499999999999 - type: nauc_recall_at_5_max value: 43.410199999999996 - type: nauc_recall_at_5_std value: 33.9841 - type: nauc_recall_at_5_diff1 value: 8.3385 - type: nauc_recall_at_10_max value: 37.335499999999996 - type: nauc_recall_at_10_std value: 31.113200000000003 - type: nauc_recall_at_10_diff1 value: 3.4785999999999997 - type: nauc_recall_at_20_max value: 35.8026 - type: nauc_recall_at_20_std value: 32.4445 - type: nauc_recall_at_20_diff1 value: -0.8593 - type: nauc_recall_at_100_max value: 36.2594 - type: nauc_recall_at_100_std value: 47.6576 - type: nauc_recall_at_100_diff1 value: -1.8901999999999999 - type: nauc_recall_at_1000_max value: 50.1605 - type: nauc_recall_at_1000_std value: 68.881 - type: nauc_recall_at_1000_diff1 value: 1.444 - type: nauc_precision_at_1_max value: 25.905099999999997 - type: nauc_precision_at_1_std value: 43.729 - type: nauc_precision_at_1_diff1 value: 5.370699999999999 - type: nauc_precision_at_3_max value: 41.4022 - type: nauc_precision_at_3_std value: 51.6336 - type: nauc_precision_at_3_diff1 value: 5.1273 - type: nauc_precision_at_5_max value: 39.137899999999995 - type: nauc_precision_at_5_std value: 52.505599999999994 - type: nauc_precision_at_5_diff1 value: 2.8716 - type: nauc_precision_at_10_max value: 37.201699999999995 - type: nauc_precision_at_10_std value: 51.8003 - type: nauc_precision_at_10_diff1 value: 1.3533 - type: nauc_precision_at_20_max value: 37.0372 - type: nauc_precision_at_20_std value: 55.199 - type: nauc_precision_at_20_diff1 value: -1.5115 - type: nauc_precision_at_100_max value: 33.7239 - type: nauc_precision_at_100_std value: 53.2366 - type: nauc_precision_at_100_diff1 value: -0.1888 - type: nauc_precision_at_1000_max value: 35.5969 - type: nauc_precision_at_1000_std value: 34.8242 - type: nauc_precision_at_1000_diff1 value: 12.4941 - type: nauc_mrr_at_1_max value: 25.905099999999997 - type: nauc_mrr_at_1_std value: 43.729 - type: nauc_mrr_at_1_diff1 value: 5.370699999999999 - type: nauc_mrr_at_3_max value: 32.7879 - type: nauc_mrr_at_3_std value: 47.7489 - type: nauc_mrr_at_3_diff1 value: 3.7714 - type: nauc_mrr_at_5_max value: 32.2069 - type: nauc_mrr_at_5_std value: 47.4263 - type: nauc_mrr_at_5_diff1 value: 2.5936 - type: nauc_mrr_at_10_max value: 31.827499999999997 - type: nauc_mrr_at_10_std value: 46.3717 - type: nauc_mrr_at_10_diff1 value: 3.6629 - type: nauc_mrr_at_20_max value: 32.4509 - type: nauc_mrr_at_20_std value: 47.153 - type: nauc_mrr_at_20_diff1 value: 3.5404999999999998 - type: nauc_mrr_at_100_max value: 32.5451 - type: nauc_mrr_at_100_std value: 47.1451 - type: nauc_mrr_at_100_diff1 value: 3.6397 - type: nauc_mrr_at_1000_max value: 32.5079 - type: nauc_mrr_at_1000_std value: 47.160999999999994 - type: nauc_mrr_at_1000_diff1 value: 3.6235000000000004 - type: main_score value: 15.190000000000001 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: neuroscience_and_neurology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 21.5 - type: ndcg_at_3 value: 21.453 - type: ndcg_at_5 value: 22.07 - type: ndcg_at_10 value: 22.159000000000002 - type: ndcg_at_20 value: 23.23 - type: ndcg_at_100 value: 30.018 - type: ndcg_at_1000 value: 42.709 - type: map_at_1 value: 1.891 - type: map_at_3 value: 4.484 - type: map_at_5 value: 5.753 - type: map_at_10 value: 7.648000000000001 - type: map_at_20 value: 9.916 - type: map_at_100 value: 15.229999999999999 - type: map_at_1000 value: 18.569 - type: recall_at_1 value: 1.891 - type: recall_at_3 value: 5.295 - type: recall_at_5 value: 8.021 - type: recall_at_10 value: 12.124 - type: recall_at_20 value: 18.348 - type: recall_at_100 value: 40.699999999999996 - type: recall_at_1000 value: 76.359 - type: precision_at_1 value: 29.5 - type: precision_at_3 value: 28.166999999999998 - type: precision_at_5 value: 27.500000000000004 - type: precision_at_10 value: 24.15 - type: precision_at_20 value: 20.549999999999997 - type: precision_at_100 value: 12.195 - type: precision_at_1000 value: 3.044 - type: mrr_at_1 value: 29.5 - type: mrr_at_3 value: 37.166700000000006 - type: mrr_at_5 value: 39.6667 - type: mrr_at_10 value: 41.2244 - type: mrr_at_20 value: 41.8155 - type: mrr_at_100 value: 42.0516 - type: mrr_at_1000 value: 42.0902 - type: nauc_ndcg_at_1_max value: 43.5626 - type: nauc_ndcg_at_1_std value: 33.057700000000004 - type: nauc_ndcg_at_1_diff1 value: 20.1071 - type: nauc_ndcg_at_3_max value: 52.8819 - type: nauc_ndcg_at_3_std value: 37.508399999999995 - type: nauc_ndcg_at_3_diff1 value: 11.0716 - type: nauc_ndcg_at_5_max value: 56.7662 - type: nauc_ndcg_at_5_std value: 44.0626 - type: nauc_ndcg_at_5_diff1 value: 13.7661 - type: nauc_ndcg_at_10_max value: 59.0881 - type: nauc_ndcg_at_10_std value: 47.4331 - type: nauc_ndcg_at_10_diff1 value: 12.359 - type: nauc_ndcg_at_20_max value: 59.967000000000006 - type: nauc_ndcg_at_20_std value: 48.2673 - type: nauc_ndcg_at_20_diff1 value: 9.870800000000001 - type: nauc_ndcg_at_100_max value: 67.0856 - type: nauc_ndcg_at_100_std value: 57.9806 - type: nauc_ndcg_at_100_diff1 value: 10.313799999999999 - type: nauc_ndcg_at_1000_max value: 65.5311 - type: nauc_ndcg_at_1000_std value: 63.056 - type: nauc_ndcg_at_1000_diff1 value: 10.4369 - type: nauc_map_at_1_max value: 51.8776 - type: nauc_map_at_1_std value: 16.841 - type: nauc_map_at_1_diff1 value: 28.8536 - type: nauc_map_at_3_max value: 59.908300000000004 - type: nauc_map_at_3_std value: 15.8167 - type: nauc_map_at_3_diff1 value: 22.8331 - type: nauc_map_at_5_max value: 58.5464 - type: nauc_map_at_5_std value: 23.1686 - type: nauc_map_at_5_diff1 value: 22.9876 - type: nauc_map_at_10_max value: 60.4082 - type: nauc_map_at_10_std value: 29.4774 - type: nauc_map_at_10_diff1 value: 18.101 - type: nauc_map_at_20_max value: 62.37350000000001 - type: nauc_map_at_20_std value: 34.9525 - type: nauc_map_at_20_diff1 value: 13.4316 - type: nauc_map_at_100_max value: 67.3622 - type: nauc_map_at_100_std value: 49.4945 - type: nauc_map_at_100_diff1 value: 12.040099999999999 - type: nauc_map_at_1000_max value: 68.682 - type: nauc_map_at_1000_std value: 54.789 - type: nauc_map_at_1000_diff1 value: 12.1231 - type: nauc_recall_at_1_max value: 51.8776 - type: nauc_recall_at_1_std value: 16.841 - type: nauc_recall_at_1_diff1 value: 28.8536 - type: nauc_recall_at_3_max value: 54.79579999999999 - type: nauc_recall_at_3_std value: 14.1491 - type: nauc_recall_at_3_diff1 value: 17.1418 - type: nauc_recall_at_5_max value: 46.6136 - type: nauc_recall_at_5_std value: 26.7224 - type: nauc_recall_at_5_diff1 value: 20.1685 - type: nauc_recall_at_10_max value: 47.1669 - type: nauc_recall_at_10_std value: 29.942800000000002 - type: nauc_recall_at_10_diff1 value: 11.7095 - type: nauc_recall_at_20_max value: 47.9761 - type: nauc_recall_at_20_std value: 32.1175 - type: nauc_recall_at_20_diff1 value: 2.7359 - type: nauc_recall_at_100_max value: 61.0279 - type: nauc_recall_at_100_std value: 54.3599 - type: nauc_recall_at_100_diff1 value: 1.3716000000000002 - type: nauc_recall_at_1000_max value: 72.0291 - type: nauc_recall_at_1000_std value: 80.9602 - type: nauc_recall_at_1000_diff1 value: -0.41869999999999996 - type: nauc_precision_at_1_max value: 47.8929 - type: nauc_precision_at_1_std value: 37.763000000000005 - type: nauc_precision_at_1_diff1 value: 18.953999999999997 - type: nauc_precision_at_3_max value: 52.2589 - type: nauc_precision_at_3_std value: 39.362399999999994 - type: nauc_precision_at_3_diff1 value: 9.073 - type: nauc_precision_at_5_max value: 54.110800000000005 - type: nauc_precision_at_5_std value: 46.6259 - type: nauc_precision_at_5_diff1 value: 10.1231 - type: nauc_precision_at_10_max value: 54.676199999999994 - type: nauc_precision_at_10_std value: 51.1227 - type: nauc_precision_at_10_diff1 value: 7.298699999999999 - type: nauc_precision_at_20_max value: 51.2621 - type: nauc_precision_at_20_std value: 51.6519 - type: nauc_precision_at_20_diff1 value: 6.9037 - type: nauc_precision_at_100_max value: 44.887100000000004 - type: nauc_precision_at_100_std value: 53.1115 - type: nauc_precision_at_100_diff1 value: 5.6667000000000005 - type: nauc_precision_at_1000_max value: 22.2381 - type: nauc_precision_at_1000_std value: 26.4376 - type: nauc_precision_at_1000_diff1 value: 6.6614 - type: nauc_mrr_at_1_max value: 47.8929 - type: nauc_mrr_at_1_std value: 37.763000000000005 - type: nauc_mrr_at_1_diff1 value: 18.953999999999997 - type: nauc_mrr_at_3_max value: 43.709199999999996 - type: nauc_mrr_at_3_std value: 36.9888 - type: nauc_mrr_at_3_diff1 value: 13.1974 - type: nauc_mrr_at_5_max value: 44.7596 - type: nauc_mrr_at_5_std value: 38.9198 - type: nauc_mrr_at_5_diff1 value: 14.5207 - type: nauc_mrr_at_10_max value: 44.952 - type: nauc_mrr_at_10_std value: 39.1837 - type: nauc_mrr_at_10_diff1 value: 14.904100000000001 - type: nauc_mrr_at_20_max value: 45.0789 - type: nauc_mrr_at_20_std value: 39.0244 - type: nauc_mrr_at_20_diff1 value: 14.617099999999999 - type: nauc_mrr_at_100_max value: 45.2951 - type: nauc_mrr_at_100_std value: 39.2859 - type: nauc_mrr_at_100_diff1 value: 14.5809 - type: nauc_mrr_at_1000_max value: 45.2772 - type: nauc_mrr_at_1000_std value: 39.2712 - type: nauc_mrr_at_1000_diff1 value: 14.6173 - type: main_score value: 22.159000000000002 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: orthopedic_surgery type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 60.75000000000001 - type: ndcg_at_3 value: 54.434000000000005 - type: ndcg_at_5 value: 51.845 - type: ndcg_at_10 value: 51.17 - type: ndcg_at_20 value: 53.299 - type: ndcg_at_100 value: 61.650000000000006 - type: ndcg_at_1000 value: 67.209 - type: map_at_1 value: 13.605999999999998 - type: map_at_3 value: 21.435000000000002 - type: map_at_5 value: 25.217 - type: map_at_10 value: 29.909000000000002 - type: map_at_20 value: 34.355000000000004 - type: map_at_100 value: 40.714 - type: map_at_1000 value: 42.296 - type: recall_at_1 value: 13.605999999999998 - type: recall_at_3 value: 22.628999999999998 - type: recall_at_5 value: 28.365000000000002 - type: recall_at_10 value: 37.228 - type: recall_at_20 value: 48.777 - type: recall_at_100 value: 77.02 - type: recall_at_1000 value: 97.235 - type: precision_at_1 value: 69.5 - type: precision_at_3 value: 53.5 - type: precision_at_5 value: 44.7 - type: precision_at_10 value: 34.35 - type: precision_at_20 value: 26.375 - type: precision_at_100 value: 10.514999999999999 - type: precision_at_1000 value: 1.44 - type: mrr_at_1 value: 69.5 - type: mrr_at_3 value: 76.25 - type: mrr_at_5 value: 77.325 - type: mrr_at_10 value: 77.9123 - type: mrr_at_20 value: 78.0774 - type: mrr_at_100 value: 78.1866 - type: mrr_at_1000 value: 78.19369999999999 - type: nauc_ndcg_at_1_max value: 54.398500000000006 - type: nauc_ndcg_at_1_std value: 17.9876 - type: nauc_ndcg_at_1_diff1 value: 45.747 - type: nauc_ndcg_at_3_max value: 52.347 - type: nauc_ndcg_at_3_std value: 20.6541 - type: nauc_ndcg_at_3_diff1 value: 33.0152 - type: nauc_ndcg_at_5_max value: 48.4688 - type: nauc_ndcg_at_5_std value: 15.216099999999999 - type: nauc_ndcg_at_5_diff1 value: 30.163 - type: nauc_ndcg_at_10_max value: 47.938399999999994 - type: nauc_ndcg_at_10_std value: 12.8266 - type: nauc_ndcg_at_10_diff1 value: 31.5183 - type: nauc_ndcg_at_20_max value: 47.1248 - type: nauc_ndcg_at_20_std value: 10.111 - type: nauc_ndcg_at_20_diff1 value: 32.941500000000005 - type: nauc_ndcg_at_100_max value: 49.1599 - type: nauc_ndcg_at_100_std value: 14.7579 - type: nauc_ndcg_at_100_diff1 value: 30.8168 - type: nauc_ndcg_at_1000_max value: 52.4098 - type: nauc_ndcg_at_1000_std value: 21.5182 - type: nauc_ndcg_at_1000_diff1 value: 31.093500000000002 - type: nauc_map_at_1_max value: 17.618000000000002 - type: nauc_map_at_1_std value: -13.4546 - type: nauc_map_at_1_diff1 value: 41.566399999999994 - type: nauc_map_at_3_max value: 23.082 - type: nauc_map_at_3_std value: -10.227 - type: nauc_map_at_3_diff1 value: 29.129500000000004 - type: nauc_map_at_5_max value: 26.845000000000002 - type: nauc_map_at_5_std value: -9.3116 - type: nauc_map_at_5_diff1 value: 27.1069 - type: nauc_map_at_10_max value: 33.0877 - type: nauc_map_at_10_std value: -5.7204999999999995 - type: nauc_map_at_10_diff1 value: 26.088 - type: nauc_map_at_20_max value: 34.9799 - type: nauc_map_at_20_std value: -2.9575 - type: nauc_map_at_20_diff1 value: 24.5261 - type: nauc_map_at_100_max value: 37.684400000000004 - type: nauc_map_at_100_std value: 6.4834000000000005 - type: nauc_map_at_100_diff1 value: 21.1685 - type: nauc_map_at_1000_max value: 38.1583 - type: nauc_map_at_1000_std value: 9.4362 - type: nauc_map_at_1000_diff1 value: 20.1685 - type: nauc_recall_at_1_max value: 17.618000000000002 - type: nauc_recall_at_1_std value: -13.4546 - type: nauc_recall_at_1_diff1 value: 41.566399999999994 - type: nauc_recall_at_3_max value: 21.3422 - type: nauc_recall_at_3_std value: -11.7132 - type: nauc_recall_at_3_diff1 value: 28.5136 - type: nauc_recall_at_5_max value: 23.0503 - type: nauc_recall_at_5_std value: -12.0308 - type: nauc_recall_at_5_diff1 value: 27.3613 - type: nauc_recall_at_10_max value: 30.965700000000002 - type: nauc_recall_at_10_std value: -7.4334 - type: nauc_recall_at_10_diff1 value: 28.5328 - type: nauc_recall_at_20_max value: 30.848399999999998 - type: nauc_recall_at_20_std value: -6.601 - type: nauc_recall_at_20_diff1 value: 27.900399999999998 - type: nauc_recall_at_100_max value: 29.0117 - type: nauc_recall_at_100_std value: 2.0551 - type: nauc_recall_at_100_diff1 value: 16.6381 - type: nauc_recall_at_1000_max value: 38.0238 - type: nauc_recall_at_1000_std value: 46.2507 - type: nauc_recall_at_1000_diff1 value: 23.321 - type: nauc_precision_at_1_max value: 59.151399999999995 - type: nauc_precision_at_1_std value: 20.9755 - type: nauc_precision_at_1_diff1 value: 39.576899999999995 - type: nauc_precision_at_3_max value: 41.3475 - type: nauc_precision_at_3_std value: 28.674899999999997 - type: nauc_precision_at_3_diff1 value: 3.8309 - type: nauc_precision_at_5_max value: 35.9683 - type: nauc_precision_at_5_std value: 26.0424 - type: nauc_precision_at_5_diff1 value: -4.8175 - type: nauc_precision_at_10_max value: 31.426199999999998 - type: nauc_precision_at_10_std value: 32.1259 - type: nauc_precision_at_10_diff1 value: -10.7677 - type: nauc_precision_at_20_max value: 18.674599999999998 - type: nauc_precision_at_20_std value: 33.768100000000004 - type: nauc_precision_at_20_diff1 value: -14.386299999999999 - type: nauc_precision_at_100_max value: 6.2094000000000005 - type: nauc_precision_at_100_std value: 37.741400000000006 - type: nauc_precision_at_100_diff1 value: -18.4227 - type: nauc_precision_at_1000_max value: 3.1195 - type: nauc_precision_at_1000_std value: 37.286500000000004 - type: nauc_precision_at_1000_diff1 value: -18.4375 - type: nauc_mrr_at_1_max value: 59.151399999999995 - type: nauc_mrr_at_1_std value: 20.9755 - type: nauc_mrr_at_1_diff1 value: 39.576899999999995 - type: nauc_mrr_at_3_max value: 63.780800000000006 - type: nauc_mrr_at_3_std value: 23.3993 - type: nauc_mrr_at_3_diff1 value: 39.055299999999995 - type: nauc_mrr_at_5_max value: 62.9519 - type: nauc_mrr_at_5_std value: 22.863500000000002 - type: nauc_mrr_at_5_diff1 value: 39.056200000000004 - type: nauc_mrr_at_10_max value: 63.10829999999999 - type: nauc_mrr_at_10_std value: 22.8684 - type: nauc_mrr_at_10_diff1 value: 38.8828 - type: nauc_mrr_at_20_max value: 63.3607 - type: nauc_mrr_at_20_std value: 22.9754 - type: nauc_mrr_at_20_diff1 value: 39.0582 - type: nauc_mrr_at_100_max value: 63.25229999999999 - type: nauc_mrr_at_100_std value: 22.922600000000003 - type: nauc_mrr_at_100_diff1 value: 39.1318 - type: nauc_mrr_at_1000_max value: 63.239000000000004 - type: nauc_mrr_at_1000_std value: 22.8994 - type: nauc_mrr_at_1000_diff1 value: 39.137499999999996 - type: main_score value: 51.17 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: orthopedic_surgery type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 12.25 - type: ndcg_at_3 value: 12.534999999999998 - type: ndcg_at_5 value: 12.592 - type: ndcg_at_10 value: 13.916 - type: ndcg_at_20 value: 15.325 - type: ndcg_at_100 value: 20.25 - type: ndcg_at_1000 value: 28.003 - type: map_at_1 value: 3.7769999999999997 - type: map_at_3 value: 5.4719999999999995 - type: map_at_5 value: 6.366 - type: map_at_10 value: 7.836 - type: map_at_20 value: 8.731 - type: map_at_100 value: 10.223 - type: map_at_1000 value: 10.972 - type: recall_at_1 value: 3.7769999999999997 - type: recall_at_3 value: 6.697 - type: recall_at_5 value: 8.474 - type: recall_at_10 value: 13.136000000000001 - type: recall_at_20 value: 17.427 - type: recall_at_100 value: 31.661 - type: recall_at_1000 value: 62.053999999999995 - type: precision_at_1 value: 14.000000000000002 - type: precision_at_3 value: 11.667 - type: precision_at_5 value: 10.100000000000001 - type: precision_at_10 value: 8.799999999999999 - type: precision_at_20 value: 7.025 - type: precision_at_100 value: 3.53 - type: precision_at_1000 value: 0.8500000000000001 - type: mrr_at_1 value: 14.000000000000002 - type: mrr_at_3 value: 19.416700000000002 - type: mrr_at_5 value: 20.041700000000002 - type: mrr_at_10 value: 21.4329 - type: mrr_at_20 value: 22.1672 - type: mrr_at_100 value: 22.6437 - type: mrr_at_1000 value: 22.750799999999998 - type: nauc_ndcg_at_1_max value: 32.8059 - type: nauc_ndcg_at_1_std value: 23.0151 - type: nauc_ndcg_at_1_diff1 value: 17.6054 - type: nauc_ndcg_at_3_max value: 32.2789 - type: nauc_ndcg_at_3_std value: 30.056300000000004 - type: nauc_ndcg_at_3_diff1 value: 14.2457 - type: nauc_ndcg_at_5_max value: 33.2192 - type: nauc_ndcg_at_5_std value: 28.3219 - type: nauc_ndcg_at_5_diff1 value: 13.1414 - type: nauc_ndcg_at_10_max value: 36.961 - type: nauc_ndcg_at_10_std value: 29.5117 - type: nauc_ndcg_at_10_diff1 value: 11.3792 - type: nauc_ndcg_at_20_max value: 36.1903 - type: nauc_ndcg_at_20_std value: 28.485300000000002 - type: nauc_ndcg_at_20_diff1 value: 11.7044 - type: nauc_ndcg_at_100_max value: 41.8182 - type: nauc_ndcg_at_100_std value: 34.623799999999996 - type: nauc_ndcg_at_100_diff1 value: 7.639500000000001 - type: nauc_ndcg_at_1000_max value: 42.3133 - type: nauc_ndcg_at_1000_std value: 32.9471 - type: nauc_ndcg_at_1000_diff1 value: 6.7987 - type: nauc_map_at_1_max value: 20.5729 - type: nauc_map_at_1_std value: 18.8459 - type: nauc_map_at_1_diff1 value: 18.7146 - type: nauc_map_at_3_max value: 28.4936 - type: nauc_map_at_3_std value: 24.8274 - type: nauc_map_at_3_diff1 value: 17.458199999999998 - type: nauc_map_at_5_max value: 29.9757 - type: nauc_map_at_5_std value: 21.243100000000002 - type: nauc_map_at_5_diff1 value: 16.2896 - type: nauc_map_at_10_max value: 34.1833 - type: nauc_map_at_10_std value: 22.8358 - type: nauc_map_at_10_diff1 value: 13.134199999999998 - type: nauc_map_at_20_max value: 35.067 - type: nauc_map_at_20_std value: 24.9975 - type: nauc_map_at_20_diff1 value: 13.1876 - type: nauc_map_at_100_max value: 37.1922 - type: nauc_map_at_100_std value: 31.132900000000003 - type: nauc_map_at_100_diff1 value: 12.1831 - type: nauc_map_at_1000_max value: 37.4097 - type: nauc_map_at_1000_std value: 31.6388 - type: nauc_map_at_1000_diff1 value: 12.045 - type: nauc_recall_at_1_max value: 20.5729 - type: nauc_recall_at_1_std value: 18.8459 - type: nauc_recall_at_1_diff1 value: 18.7146 - type: nauc_recall_at_3_max value: 28.170299999999997 - type: nauc_recall_at_3_std value: 26.860400000000002 - type: nauc_recall_at_3_diff1 value: 13.3677 - type: nauc_recall_at_5_max value: 28.220299999999998 - type: nauc_recall_at_5_std value: 19.519000000000002 - type: nauc_recall_at_5_diff1 value: 12.501999999999999 - type: nauc_recall_at_10_max value: 36.1699 - type: nauc_recall_at_10_std value: 22.950699999999998 - type: nauc_recall_at_10_diff1 value: 8.3368 - type: nauc_recall_at_20_max value: 34.869 - type: nauc_recall_at_20_std value: 23.3199 - type: nauc_recall_at_20_diff1 value: 8.0855 - type: nauc_recall_at_100_max value: 48.916399999999996 - type: nauc_recall_at_100_std value: 39.6427 - type: nauc_recall_at_100_diff1 value: 0.1031 - type: nauc_recall_at_1000_max value: 56.2504 - type: nauc_recall_at_1000_std value: 42.9539 - type: nauc_recall_at_1000_diff1 value: -3.3978 - type: nauc_precision_at_1_max value: 34.6833 - type: nauc_precision_at_1_std value: 22.2443 - type: nauc_precision_at_1_diff1 value: 18.937 - type: nauc_precision_at_3_max value: 39.2643 - type: nauc_precision_at_3_std value: 36.851099999999995 - type: nauc_precision_at_3_diff1 value: 12.923699999999998 - type: nauc_precision_at_5_max value: 37.3389 - type: nauc_precision_at_5_std value: 30.6489 - type: nauc_precision_at_5_diff1 value: 8.0897 - type: nauc_precision_at_10_max value: 41.051 - type: nauc_precision_at_10_std value: 36.8489 - type: nauc_precision_at_10_diff1 value: 3.2661999999999995 - type: nauc_precision_at_20_max value: 37.676700000000004 - type: nauc_precision_at_20_std value: 35.885 - type: nauc_precision_at_20_diff1 value: 6.8808 - type: nauc_precision_at_100_max value: 31.550800000000002 - type: nauc_precision_at_100_std value: 35.3254 - type: nauc_precision_at_100_diff1 value: -0.0628 - type: nauc_precision_at_1000_max value: 12.159699999999999 - type: nauc_precision_at_1000_std value: 8.086699999999999 - type: nauc_precision_at_1000_diff1 value: 1.609 - type: nauc_mrr_at_1_max value: 34.6833 - type: nauc_mrr_at_1_std value: 22.2443 - type: nauc_mrr_at_1_diff1 value: 18.937 - type: nauc_mrr_at_3_max value: 34.6155 - type: nauc_mrr_at_3_std value: 26.871299999999998 - type: nauc_mrr_at_3_diff1 value: 13.0607 - type: nauc_mrr_at_5_max value: 34.1124 - type: nauc_mrr_at_5_std value: 26.673000000000002 - type: nauc_mrr_at_5_diff1 value: 12.9241 - type: nauc_mrr_at_10_max value: 35.2871 - type: nauc_mrr_at_10_std value: 27.596799999999998 - type: nauc_mrr_at_10_diff1 value: 11.6235 - type: nauc_mrr_at_20_max value: 35.1618 - type: nauc_mrr_at_20_std value: 26.937 - type: nauc_mrr_at_20_diff1 value: 11.5846 - type: nauc_mrr_at_100_max value: 35.4628 - type: nauc_mrr_at_100_std value: 26.630599999999998 - type: nauc_mrr_at_100_diff1 value: 11.338099999999999 - type: nauc_mrr_at_1000_max value: 35.4548 - type: nauc_mrr_at_1000_std value: 26.617600000000003 - type: nauc_mrr_at_1000_diff1 value: 11.4321 - type: main_score value: 13.916 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: orthopedic_surgery type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 24.5 - type: ndcg_at_3 value: 24.687 - type: ndcg_at_5 value: 24.352999999999998 - type: ndcg_at_10 value: 24.891 - type: ndcg_at_20 value: 26.287 - type: ndcg_at_100 value: 33.079 - type: ndcg_at_1000 value: 41.436 - type: map_at_1 value: 6.834999999999999 - type: map_at_3 value: 10.338 - type: map_at_5 value: 11.949 - type: map_at_10 value: 13.785 - type: map_at_20 value: 15.445999999999998 - type: map_at_100 value: 18.32 - type: map_at_1000 value: 19.709 - type: recall_at_1 value: 6.834999999999999 - type: recall_at_3 value: 11.558 - type: recall_at_5 value: 15.079999999999998 - type: recall_at_10 value: 20.226 - type: recall_at_20 value: 26.022000000000002 - type: recall_at_100 value: 46.611999999999995 - type: recall_at_1000 value: 78.901 - type: precision_at_1 value: 29.5 - type: precision_at_3 value: 24.833 - type: precision_at_5 value: 20.599999999999998 - type: precision_at_10 value: 16.150000000000002 - type: precision_at_20 value: 12.575 - type: precision_at_100 value: 5.94 - type: precision_at_1000 value: 1.143 - type: mrr_at_1 value: 29.5 - type: mrr_at_3 value: 36.916700000000006 - type: mrr_at_5 value: 38.9917 - type: mrr_at_10 value: 40.5738 - type: mrr_at_20 value: 41.2425 - type: mrr_at_100 value: 41.5069 - type: mrr_at_1000 value: 41.5546 - type: nauc_ndcg_at_1_max value: 39.3236 - type: nauc_ndcg_at_1_std value: 21.2847 - type: nauc_ndcg_at_1_diff1 value: 31.586399999999998 - type: nauc_ndcg_at_3_max value: 48.9355 - type: nauc_ndcg_at_3_std value: 22.438299999999998 - type: nauc_ndcg_at_3_diff1 value: 27.0578 - type: nauc_ndcg_at_5_max value: 52.169 - type: nauc_ndcg_at_5_std value: 23.6694 - type: nauc_ndcg_at_5_diff1 value: 23.735300000000002 - type: nauc_ndcg_at_10_max value: 51.781 - type: nauc_ndcg_at_10_std value: 24.0064 - type: nauc_ndcg_at_10_diff1 value: 20.6463 - type: nauc_ndcg_at_20_max value: 51.7658 - type: nauc_ndcg_at_20_std value: 24.0852 - type: nauc_ndcg_at_20_diff1 value: 19.7103 - type: nauc_ndcg_at_100_max value: 54.4474 - type: nauc_ndcg_at_100_std value: 31.6873 - type: nauc_ndcg_at_100_diff1 value: 17.1214 - type: nauc_ndcg_at_1000_max value: 56.9241 - type: nauc_ndcg_at_1000_std value: 38.0309 - type: nauc_ndcg_at_1000_diff1 value: 16.6583 - type: nauc_map_at_1_max value: 40.3816 - type: nauc_map_at_1_std value: 5.3704 - type: nauc_map_at_1_diff1 value: 53.0836 - type: nauc_map_at_3_max value: 42.9518 - type: nauc_map_at_3_std value: 8.5508 - type: nauc_map_at_3_diff1 value: 41.4236 - type: nauc_map_at_5_max value: 44.039699999999996 - type: nauc_map_at_5_std value: 9.459299999999999 - type: nauc_map_at_5_diff1 value: 34.3919 - type: nauc_map_at_10_max value: 43.518699999999995 - type: nauc_map_at_10_std value: 10.4546 - type: nauc_map_at_10_diff1 value: 28.8859 - type: nauc_map_at_20_max value: 45.3241 - type: nauc_map_at_20_std value: 13.3366 - type: nauc_map_at_20_diff1 value: 26.4547 - type: nauc_map_at_100_max value: 47.9247 - type: nauc_map_at_100_std value: 20.048 - type: nauc_map_at_100_diff1 value: 21.915499999999998 - type: nauc_map_at_1000_max value: 48.8745 - type: nauc_map_at_1000_std value: 23.2159 - type: nauc_map_at_1000_diff1 value: 21.1535 - type: nauc_recall_at_1_max value: 40.3816 - type: nauc_recall_at_1_std value: 5.3704 - type: nauc_recall_at_1_diff1 value: 53.0836 - type: nauc_recall_at_3_max value: 43.917699999999996 - type: nauc_recall_at_3_std value: 8.602 - type: nauc_recall_at_3_diff1 value: 36.0712 - type: nauc_recall_at_5_max value: 43.6882 - type: nauc_recall_at_5_std value: 9.4163 - type: nauc_recall_at_5_diff1 value: 21.9081 - type: nauc_recall_at_10_max value: 40.0266 - type: nauc_recall_at_10_std value: 11.4113 - type: nauc_recall_at_10_diff1 value: 12.4679 - type: nauc_recall_at_20_max value: 40.718199999999996 - type: nauc_recall_at_20_std value: 13.3378 - type: nauc_recall_at_20_diff1 value: 8.6095 - type: nauc_recall_at_100_max value: 48.9917 - type: nauc_recall_at_100_std value: 34.8589 - type: nauc_recall_at_100_diff1 value: 2.7233 - type: nauc_recall_at_1000_max value: 63.3372 - type: nauc_recall_at_1000_std value: 63.046899999999994 - type: nauc_recall_at_1000_diff1 value: -9.3282 - type: nauc_precision_at_1_max value: 44.1351 - type: nauc_precision_at_1_std value: 25.255300000000002 - type: nauc_precision_at_1_diff1 value: 28.532200000000003 - type: nauc_precision_at_3_max value: 46.5017 - type: nauc_precision_at_3_std value: 29.053 - type: nauc_precision_at_3_diff1 value: 12.581500000000002 - type: nauc_precision_at_5_max value: 50.9217 - type: nauc_precision_at_5_std value: 35.749900000000004 - type: nauc_precision_at_5_diff1 value: 2.922 - type: nauc_precision_at_10_max value: 45.213 - type: nauc_precision_at_10_std value: 38.149100000000004 - type: nauc_precision_at_10_diff1 value: -7.9922 - type: nauc_precision_at_20_max value: 40.5852 - type: nauc_precision_at_20_std value: 40.3656 - type: nauc_precision_at_20_diff1 value: -11.210199999999999 - type: nauc_precision_at_100_max value: 28.71 - type: nauc_precision_at_100_std value: 43.1581 - type: nauc_precision_at_100_diff1 value: -15.4147 - type: nauc_precision_at_1000_max value: 17.1742 - type: nauc_precision_at_1000_std value: 35.162 - type: nauc_precision_at_1000_diff1 value: -8.7069 - type: nauc_mrr_at_1_max value: 44.1351 - type: nauc_mrr_at_1_std value: 25.255300000000002 - type: nauc_mrr_at_1_diff1 value: 28.532200000000003 - type: nauc_mrr_at_3_max value: 51.83069999999999 - type: nauc_mrr_at_3_std value: 30.6181 - type: nauc_mrr_at_3_diff1 value: 26.160600000000002 - type: nauc_mrr_at_5_max value: 52.36110000000001 - type: nauc_mrr_at_5_std value: 30.839100000000002 - type: nauc_mrr_at_5_diff1 value: 24.5763 - type: nauc_mrr_at_10_max value: 52.1246 - type: nauc_mrr_at_10_std value: 30.9131 - type: nauc_mrr_at_10_diff1 value: 23.5647 - type: nauc_mrr_at_20_max value: 51.9513 - type: nauc_mrr_at_20_std value: 30.7942 - type: nauc_mrr_at_20_diff1 value: 23.8432 - type: nauc_mrr_at_100_max value: 52.123200000000004 - type: nauc_mrr_at_100_std value: 30.9398 - type: nauc_mrr_at_100_diff1 value: 23.9534 - type: nauc_mrr_at_1000_max value: 52.076299999999996 - type: nauc_mrr_at_1000_std value: 30.8819 - type: nauc_mrr_at_1000_diff1 value: 23.9329 - type: main_score value: 24.891 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: otorhinolaryngology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 58.75 - type: ndcg_at_3 value: 51.113 - type: ndcg_at_5 value: 48.946 - type: ndcg_at_10 value: 47.447 - type: ndcg_at_20 value: 47.425 - type: ndcg_at_100 value: 54.642 - type: ndcg_at_1000 value: 65.155 - type: map_at_1 value: 6.365 - type: map_at_3 value: 12.296999999999999 - type: map_at_5 value: 14.866999999999999 - type: map_at_10 value: 18.975 - type: map_at_20 value: 22.985 - type: map_at_100 value: 30.8 - type: map_at_1000 value: 35.141 - type: recall_at_1 value: 6.365 - type: recall_at_3 value: 13.253 - type: recall_at_5 value: 17.333000000000002 - type: recall_at_10 value: 25.029 - type: recall_at_20 value: 34.623 - type: recall_at_100 value: 62.605 - type: recall_at_1000 value: 94.306 - type: precision_at_1 value: 70.0 - type: precision_at_3 value: 56.333 - type: precision_at_5 value: 50.6 - type: precision_at_10 value: 41.75 - type: precision_at_20 value: 33.800000000000004 - type: precision_at_100 value: 16.845 - type: precision_at_1000 value: 3.3939999999999997 - type: mrr_at_1 value: 70.0 - type: mrr_at_3 value: 77.0833 - type: mrr_at_5 value: 78.7333 - type: mrr_at_10 value: 78.9942 - type: mrr_at_20 value: 79.0994 - type: mrr_at_100 value: 79.1791 - type: mrr_at_1000 value: 79.183 - type: nauc_ndcg_at_1_max value: 23.0541 - type: nauc_ndcg_at_1_std value: -6.7547999999999995 - type: nauc_ndcg_at_1_diff1 value: 34.7899 - type: nauc_ndcg_at_3_max value: 31.9251 - type: nauc_ndcg_at_3_std value: 5.774900000000001 - type: nauc_ndcg_at_3_diff1 value: 29.1505 - type: nauc_ndcg_at_5_max value: 31.531599999999997 - type: nauc_ndcg_at_5_std value: 6.903099999999999 - type: nauc_ndcg_at_5_diff1 value: 25.2152 - type: nauc_ndcg_at_10_max value: 32.771 - type: nauc_ndcg_at_10_std value: 4.9274 - type: nauc_ndcg_at_10_diff1 value: 23.6843 - type: nauc_ndcg_at_20_max value: 29.857400000000002 - type: nauc_ndcg_at_20_std value: 2.3416 - type: nauc_ndcg_at_20_diff1 value: 25.3388 - type: nauc_ndcg_at_100_max value: 29.4487 - type: nauc_ndcg_at_100_std value: 8.363199999999999 - type: nauc_ndcg_at_100_diff1 value: 24.3061 - type: nauc_ndcg_at_1000_max value: 32.656800000000004 - type: nauc_ndcg_at_1000_std value: 15.3001 - type: nauc_ndcg_at_1000_diff1 value: 21.607599999999998 - type: nauc_map_at_1_max value: 2.984 - type: nauc_map_at_1_std value: -16.3357 - type: nauc_map_at_1_diff1 value: 39.8679 - type: nauc_map_at_3_max value: 13.1494 - type: nauc_map_at_3_std value: -9.7997 - type: nauc_map_at_3_diff1 value: 32.3868 - type: nauc_map_at_5_max value: 15.5304 - type: nauc_map_at_5_std value: -6.934899999999999 - type: nauc_map_at_5_diff1 value: 29.6433 - type: nauc_map_at_10_max value: 20.6028 - type: nauc_map_at_10_std value: -4.782 - type: nauc_map_at_10_diff1 value: 26.6735 - type: nauc_map_at_20_max value: 23.7623 - type: nauc_map_at_20_std value: -1.7728000000000002 - type: nauc_map_at_20_diff1 value: 24.4019 - type: nauc_map_at_100_max value: 27.324900000000003 - type: nauc_map_at_100_std value: 7.7754 - type: nauc_map_at_100_diff1 value: 22.1984 - type: nauc_map_at_1000_max value: 27.751900000000003 - type: nauc_map_at_1000_std value: 13.422899999999998 - type: nauc_map_at_1000_diff1 value: 20.6692 - type: nauc_recall_at_1_max value: 2.984 - type: nauc_recall_at_1_std value: -16.3357 - type: nauc_recall_at_1_diff1 value: 39.8679 - type: nauc_recall_at_3_max value: 11.4488 - type: nauc_recall_at_3_std value: -9.1606 - type: nauc_recall_at_3_diff1 value: 32.008199999999995 - type: nauc_recall_at_5_max value: 11.7309 - type: nauc_recall_at_5_std value: -7.855099999999999 - type: nauc_recall_at_5_diff1 value: 26.928 - type: nauc_recall_at_10_max value: 14.1124 - type: nauc_recall_at_10_std value: -8.2747 - type: nauc_recall_at_10_diff1 value: 21.8328 - type: nauc_recall_at_20_max value: 13.697999999999999 - type: nauc_recall_at_20_std value: -5.6466 - type: nauc_recall_at_20_diff1 value: 17.2139 - type: nauc_recall_at_100_max value: 12.7504 - type: nauc_recall_at_100_std value: 9.058399999999999 - type: nauc_recall_at_100_diff1 value: 8.5206 - type: nauc_recall_at_1000_max value: 37.2254 - type: nauc_recall_at_1000_std value: 49.817699999999995 - type: nauc_recall_at_1000_diff1 value: -4.812799999999999 - type: nauc_precision_at_1_max value: 27.128400000000003 - type: nauc_precision_at_1_std value: -6.3169 - type: nauc_precision_at_1_diff1 value: 37.2347 - type: nauc_precision_at_3_max value: 40.02 - type: nauc_precision_at_3_std value: 17.0871 - type: nauc_precision_at_3_diff1 value: 16.1509 - type: nauc_precision_at_5_max value: 35.7648 - type: nauc_precision_at_5_std value: 22.4046 - type: nauc_precision_at_5_diff1 value: 3.5429000000000004 - type: nauc_precision_at_10_max value: 32.5383 - type: nauc_precision_at_10_std value: 19.8196 - type: nauc_precision_at_10_diff1 value: -2.6108 - type: nauc_precision_at_20_max value: 24.7033 - type: nauc_precision_at_20_std value: 20.3249 - type: nauc_precision_at_20_diff1 value: -5.3223 - type: nauc_precision_at_100_max value: 4.4075999999999995 - type: nauc_precision_at_100_std value: 20.3847 - type: nauc_precision_at_100_diff1 value: -7.7542 - type: nauc_precision_at_1000_max value: -0.1031 - type: nauc_precision_at_1000_std value: 13.970699999999999 - type: nauc_precision_at_1000_diff1 value: -8.6302 - type: nauc_mrr_at_1_max value: 27.128400000000003 - type: nauc_mrr_at_1_std value: -6.3169 - type: nauc_mrr_at_1_diff1 value: 37.2347 - type: nauc_mrr_at_3_max value: 32.1131 - type: nauc_mrr_at_3_std value: 2.7546 - type: nauc_mrr_at_3_diff1 value: 37.2989 - type: nauc_mrr_at_5_max value: 30.8927 - type: nauc_mrr_at_5_std value: 0.422 - type: nauc_mrr_at_5_diff1 value: 36.0035 - type: nauc_mrr_at_10_max value: 30.3494 - type: nauc_mrr_at_10_std value: -0.2848 - type: nauc_mrr_at_10_diff1 value: 35.881600000000006 - type: nauc_mrr_at_20_max value: 30.2487 - type: nauc_mrr_at_20_std value: -0.4132 - type: nauc_mrr_at_20_diff1 value: 35.9397 - type: nauc_mrr_at_100_max value: 30.206 - type: nauc_mrr_at_100_std value: -0.39039999999999997 - type: nauc_mrr_at_100_diff1 value: 36.020799999999994 - type: nauc_mrr_at_1000_max value: 30.2487 - type: nauc_mrr_at_1000_std value: -0.3869 - type: nauc_mrr_at_1000_diff1 value: 36.0524 - type: main_score value: 47.447 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: otorhinolaryngology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 9.0 - type: ndcg_at_3 value: 8.487 - type: ndcg_at_5 value: 8.622 - type: ndcg_at_10 value: 9.475999999999999 - type: ndcg_at_20 value: 10.620000000000001 - type: ndcg_at_100 value: 16.346 - type: ndcg_at_1000 value: 27.265 - type: map_at_1 value: 0.9900000000000001 - type: map_at_3 value: 1.892 - type: map_at_5 value: 2.261 - type: map_at_10 value: 2.94 - type: map_at_20 value: 3.6990000000000003 - type: map_at_100 value: 5.8229999999999995 - type: map_at_1000 value: 7.664999999999999 - type: recall_at_1 value: 0.9900000000000001 - type: recall_at_3 value: 2.637 - type: recall_at_5 value: 3.6189999999999998 - type: recall_at_10 value: 5.88 - type: recall_at_20 value: 9.259 - type: recall_at_100 value: 25.215 - type: recall_at_1000 value: 58.672999999999995 - type: precision_at_1 value: 12.0 - type: precision_at_3 value: 10.333 - type: precision_at_5 value: 10.0 - type: precision_at_10 value: 9.65 - type: precision_at_20 value: 8.825 - type: precision_at_100 value: 6.175 - type: precision_at_1000 value: 1.916 - type: mrr_at_1 value: 12.0 - type: mrr_at_3 value: 16.5833 - type: mrr_at_5 value: 18.9583 - type: mrr_at_10 value: 20.194000000000003 - type: mrr_at_20 value: 21.0313 - type: mrr_at_100 value: 21.6374 - type: mrr_at_1000 value: 21.7083 - type: nauc_ndcg_at_1_max value: 42.2033 - type: nauc_ndcg_at_1_std value: 33.1304 - type: nauc_ndcg_at_1_diff1 value: 3.8263 - type: nauc_ndcg_at_3_max value: 36.240899999999996 - type: nauc_ndcg_at_3_std value: 46.0242 - type: nauc_ndcg_at_3_diff1 value: 0.638 - type: nauc_ndcg_at_5_max value: 35.0781 - type: nauc_ndcg_at_5_std value: 44.2813 - type: nauc_ndcg_at_5_diff1 value: -0.772 - type: nauc_ndcg_at_10_max value: 34.6139 - type: nauc_ndcg_at_10_std value: 47.0856 - type: nauc_ndcg_at_10_diff1 value: 2.5038 - type: nauc_ndcg_at_20_max value: 38.8003 - type: nauc_ndcg_at_20_std value: 49.3951 - type: nauc_ndcg_at_20_diff1 value: 6.1885 - type: nauc_ndcg_at_100_max value: 43.7993 - type: nauc_ndcg_at_100_std value: 57.052 - type: nauc_ndcg_at_100_diff1 value: 6.4491000000000005 - type: nauc_ndcg_at_1000_max value: 50.7017 - type: nauc_ndcg_at_1000_std value: 58.455999999999996 - type: nauc_ndcg_at_1000_diff1 value: 3.6403 - type: nauc_map_at_1_max value: 46.286899999999996 - type: nauc_map_at_1_std value: 26.722600000000003 - type: nauc_map_at_1_diff1 value: 23.3318 - type: nauc_map_at_3_max value: 34.0861 - type: nauc_map_at_3_std value: 46.7414 - type: nauc_map_at_3_diff1 value: 8.5018 - type: nauc_map_at_5_max value: 34.7325 - type: nauc_map_at_5_std value: 43.0688 - type: nauc_map_at_5_diff1 value: 9.9683 - type: nauc_map_at_10_max value: 35.2354 - type: nauc_map_at_10_std value: 46.2276 - type: nauc_map_at_10_diff1 value: 11.2777 - type: nauc_map_at_20_max value: 39.5891 - type: nauc_map_at_20_std value: 48.1422 - type: nauc_map_at_20_diff1 value: 10.886999999999999 - type: nauc_map_at_100_max value: 42.2714 - type: nauc_map_at_100_std value: 57.924699999999994 - type: nauc_map_at_100_diff1 value: 7.001799999999999 - type: nauc_map_at_1000_max value: 44.162800000000004 - type: nauc_map_at_1000_std value: 59.3092 - type: nauc_map_at_1000_diff1 value: 3.3226 - type: nauc_recall_at_1_max value: 46.286899999999996 - type: nauc_recall_at_1_std value: 26.722600000000003 - type: nauc_recall_at_1_diff1 value: 23.3318 - type: nauc_recall_at_3_max value: 26.035399999999996 - type: nauc_recall_at_3_std value: 53.1263 - type: nauc_recall_at_3_diff1 value: 0.42129999999999995 - type: nauc_recall_at_5_max value: 25.4674 - type: nauc_recall_at_5_std value: 44.1066 - type: nauc_recall_at_5_diff1 value: 4.22 - type: nauc_recall_at_10_max value: 24.2043 - type: nauc_recall_at_10_std value: 43.383 - type: nauc_recall_at_10_diff1 value: 10.5937 - type: nauc_recall_at_20_max value: 36.433 - type: nauc_recall_at_20_std value: 46.3323 - type: nauc_recall_at_20_diff1 value: 13.9961 - type: nauc_recall_at_100_max value: 37.576100000000004 - type: nauc_recall_at_100_std value: 54.7615 - type: nauc_recall_at_100_diff1 value: 11.209900000000001 - type: nauc_recall_at_1000_max value: 55.1674 - type: nauc_recall_at_1000_std value: 59.4684 - type: nauc_recall_at_1000_diff1 value: 11.3697 - type: nauc_precision_at_1_max value: 42.180800000000005 - type: nauc_precision_at_1_std value: 32.8073 - type: nauc_precision_at_1_diff1 value: 0.4348 - type: nauc_precision_at_3_max value: 38.1217 - type: nauc_precision_at_3_std value: 46.0086 - type: nauc_precision_at_3_diff1 value: -2.2963 - type: nauc_precision_at_5_max value: 35.9551 - type: nauc_precision_at_5_std value: 42.0543 - type: nauc_precision_at_5_diff1 value: -3.1207 - type: nauc_precision_at_10_max value: 34.2246 - type: nauc_precision_at_10_std value: 46.5836 - type: nauc_precision_at_10_diff1 value: -2.844 - type: nauc_precision_at_20_max value: 39.338899999999995 - type: nauc_precision_at_20_std value: 50.256299999999996 - type: nauc_precision_at_20_diff1 value: -3.9847 - type: nauc_precision_at_100_max value: 35.5887 - type: nauc_precision_at_100_std value: 49.170700000000004 - type: nauc_precision_at_100_diff1 value: -6.653499999999999 - type: nauc_precision_at_1000_max value: 22.698999999999998 - type: nauc_precision_at_1000_std value: 23.137900000000002 - type: nauc_precision_at_1000_diff1 value: -5.8536 - type: nauc_mrr_at_1_max value: 42.180800000000005 - type: nauc_mrr_at_1_std value: 32.8073 - type: nauc_mrr_at_1_diff1 value: 0.4348 - type: nauc_mrr_at_3_max value: 39.9656 - type: nauc_mrr_at_3_std value: 38.564 - type: nauc_mrr_at_3_diff1 value: -0.4943 - type: nauc_mrr_at_5_max value: 39.152300000000004 - type: nauc_mrr_at_5_std value: 38.2446 - type: nauc_mrr_at_5_diff1 value: -1.0491000000000001 - type: nauc_mrr_at_10_max value: 39.4098 - type: nauc_mrr_at_10_std value: 38.4484 - type: nauc_mrr_at_10_diff1 value: -0.134 - type: nauc_mrr_at_20_max value: 40.426899999999996 - type: nauc_mrr_at_20_std value: 38.9061 - type: nauc_mrr_at_20_diff1 value: 0.20010000000000003 - type: nauc_mrr_at_100_max value: 40.4788 - type: nauc_mrr_at_100_std value: 38.8631 - type: nauc_mrr_at_100_diff1 value: -0.013300000000000001 - type: nauc_mrr_at_1000_max value: 40.4469 - type: nauc_mrr_at_1000_std value: 38.8602 - type: nauc_mrr_at_1000_diff1 value: 0.0619 - type: main_score value: 9.475999999999999 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: otorhinolaryngology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 19.5 - type: ndcg_at_3 value: 17.663 - type: ndcg_at_5 value: 17.574 - type: ndcg_at_10 value: 17.660999999999998 - type: ndcg_at_20 value: 18.372 - type: ndcg_at_100 value: 24.366 - type: ndcg_at_1000 value: 35.075 - type: map_at_1 value: 2.095 - type: map_at_3 value: 4.43 - type: map_at_5 value: 5.437 - type: map_at_10 value: 6.736000000000001 - type: map_at_20 value: 8.116 - type: map_at_100 value: 11.183 - type: map_at_1000 value: 13.373 - type: recall_at_1 value: 2.095 - type: recall_at_3 value: 5.135 - type: recall_at_5 value: 7.002999999999999 - type: recall_at_10 value: 10.571 - type: recall_at_20 value: 14.809 - type: recall_at_100 value: 32.126 - type: recall_at_1000 value: 64.834 - type: precision_at_1 value: 25.5 - type: precision_at_3 value: 21.5 - type: precision_at_5 value: 19.7 - type: precision_at_10 value: 16.6 - type: precision_at_20 value: 13.8 - type: precision_at_100 value: 8.075000000000001 - type: precision_at_1000 value: 2.1399999999999997 - type: mrr_at_1 value: 25.5 - type: mrr_at_3 value: 31.9167 - type: mrr_at_5 value: 33.216699999999996 - type: mrr_at_10 value: 34.9563 - type: mrr_at_20 value: 35.6321 - type: mrr_at_100 value: 36.0473 - type: mrr_at_1000 value: 36.106500000000004 - type: nauc_ndcg_at_1_max value: 38.752900000000004 - type: nauc_ndcg_at_1_std value: 22.9816 - type: nauc_ndcg_at_1_diff1 value: 14.973 - type: nauc_ndcg_at_3_max value: 45.1268 - type: nauc_ndcg_at_3_std value: 27.2744 - type: nauc_ndcg_at_3_diff1 value: 17.1938 - type: nauc_ndcg_at_5_max value: 50.1247 - type: nauc_ndcg_at_5_std value: 32.867000000000004 - type: nauc_ndcg_at_5_diff1 value: 17.5213 - type: nauc_ndcg_at_10_max value: 54.09080000000001 - type: nauc_ndcg_at_10_std value: 35.4424 - type: nauc_ndcg_at_10_diff1 value: 19.7834 - type: nauc_ndcg_at_20_max value: 55.9825 - type: nauc_ndcg_at_20_std value: 36.863600000000005 - type: nauc_ndcg_at_20_diff1 value: 21.8584 - type: nauc_ndcg_at_100_max value: 60.00529999999999 - type: nauc_ndcg_at_100_std value: 45.7262 - type: nauc_ndcg_at_100_diff1 value: 20.6427 - type: nauc_ndcg_at_1000_max value: 59.0706 - type: nauc_ndcg_at_1000_std value: 50.505100000000006 - type: nauc_ndcg_at_1000_diff1 value: 15.5525 - type: nauc_map_at_1_max value: 33.4795 - type: nauc_map_at_1_std value: 14.426400000000001 - type: nauc_map_at_1_diff1 value: 16.761699999999998 - type: nauc_map_at_3_max value: 37.6433 - type: nauc_map_at_3_std value: 16.2529 - type: nauc_map_at_3_diff1 value: 12.893099999999999 - type: nauc_map_at_5_max value: 41.5271 - type: nauc_map_at_5_std value: 20.3954 - type: nauc_map_at_5_diff1 value: 17.0506 - type: nauc_map_at_10_max value: 47.210499999999996 - type: nauc_map_at_10_std value: 25.4344 - type: nauc_map_at_10_diff1 value: 19.8433 - type: nauc_map_at_20_max value: 51.430299999999995 - type: nauc_map_at_20_std value: 31.0075 - type: nauc_map_at_20_diff1 value: 21.435000000000002 - type: nauc_map_at_100_max value: 55.723299999999995 - type: nauc_map_at_100_std value: 41.9041 - type: nauc_map_at_100_diff1 value: 20.6049 - type: nauc_map_at_1000_max value: 54.0405 - type: nauc_map_at_1000_std value: 45.088 - type: nauc_map_at_1000_diff1 value: 17.8828 - type: nauc_recall_at_1_max value: 33.4795 - type: nauc_recall_at_1_std value: 14.426400000000001 - type: nauc_recall_at_1_diff1 value: 16.761699999999998 - type: nauc_recall_at_3_max value: 41.3467 - type: nauc_recall_at_3_std value: 19.2823 - type: nauc_recall_at_3_diff1 value: 13.3268 - type: nauc_recall_at_5_max value: 46.3683 - type: nauc_recall_at_5_std value: 23.827 - type: nauc_recall_at_5_diff1 value: 19.297 - type: nauc_recall_at_10_max value: 50.0236 - type: nauc_recall_at_10_std value: 25.537 - type: nauc_recall_at_10_diff1 value: 23.9121 - type: nauc_recall_at_20_max value: 52.249500000000005 - type: nauc_recall_at_20_std value: 30.9311 - type: nauc_recall_at_20_diff1 value: 23.5544 - type: nauc_recall_at_100_max value: 60.0984 - type: nauc_recall_at_100_std value: 46.9398 - type: nauc_recall_at_100_diff1 value: 20.657600000000002 - type: nauc_recall_at_1000_max value: 71.0111 - type: nauc_recall_at_1000_std value: 63.178999999999995 - type: nauc_recall_at_1000_diff1 value: 14.643900000000002 - type: nauc_precision_at_1_max value: 38.3342 - type: nauc_precision_at_1_std value: 24.3116 - type: nauc_precision_at_1_diff1 value: 12.347199999999999 - type: nauc_precision_at_3_max value: 49.369099999999996 - type: nauc_precision_at_3_std value: 32.3882 - type: nauc_precision_at_3_diff1 value: 16.189500000000002 - type: nauc_precision_at_5_max value: 55.2461 - type: nauc_precision_at_5_std value: 39.0339 - type: nauc_precision_at_5_diff1 value: 17.2901 - type: nauc_precision_at_10_max value: 57.067800000000005 - type: nauc_precision_at_10_std value: 43.318 - type: nauc_precision_at_10_diff1 value: 17.4133 - type: nauc_precision_at_20_max value: 52.382600000000004 - type: nauc_precision_at_20_std value: 45.1982 - type: nauc_precision_at_20_diff1 value: 16.1817 - type: nauc_precision_at_100_max value: 33.0242 - type: nauc_precision_at_100_std value: 43.7429 - type: nauc_precision_at_100_diff1 value: 6.1582 - type: nauc_precision_at_1000_max value: 6.4754000000000005 - type: nauc_precision_at_1000_std value: 16.081500000000002 - type: nauc_precision_at_1000_diff1 value: -2.5237 - type: nauc_mrr_at_1_max value: 38.3342 - type: nauc_mrr_at_1_std value: 24.3116 - type: nauc_mrr_at_1_diff1 value: 12.347199999999999 - type: nauc_mrr_at_3_max value: 44.7413 - type: nauc_mrr_at_3_std value: 28.607100000000003 - type: nauc_mrr_at_3_diff1 value: 15.176 - type: nauc_mrr_at_5_max value: 46.9016 - type: nauc_mrr_at_5_std value: 29.964000000000002 - type: nauc_mrr_at_5_diff1 value: 15.0206 - type: nauc_mrr_at_10_max value: 46.6374 - type: nauc_mrr_at_10_std value: 29.4985 - type: nauc_mrr_at_10_diff1 value: 15.9884 - type: nauc_mrr_at_20_max value: 46.2181 - type: nauc_mrr_at_20_std value: 29.538999999999998 - type: nauc_mrr_at_20_diff1 value: 15.964999999999998 - type: nauc_mrr_at_100_max value: 46.1424 - type: nauc_mrr_at_100_std value: 29.5665 - type: nauc_mrr_at_100_diff1 value: 15.7968 - type: nauc_mrr_at_1000_max value: 46.1536 - type: nauc_mrr_at_1000_std value: 29.5835 - type: nauc_mrr_at_1000_diff1 value: 15.7713 - type: main_score value: 17.660999999999998 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: plastic_surgery type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 67.75 - type: ndcg_at_3 value: 59.958 - type: ndcg_at_5 value: 56.188 - type: ndcg_at_10 value: 54.081999999999994 - type: ndcg_at_20 value: 54.352000000000004 - type: ndcg_at_100 value: 63.461 - type: ndcg_at_1000 value: 71.052 - type: map_at_1 value: 8.769 - type: map_at_3 value: 16.821 - type: map_at_5 value: 20.674999999999997 - type: map_at_10 value: 25.861 - type: map_at_20 value: 30.334 - type: map_at_100 value: 39.324 - type: map_at_1000 value: 42.05 - type: recall_at_1 value: 8.769 - type: recall_at_3 value: 17.97 - type: recall_at_5 value: 23.372999999999998 - type: recall_at_10 value: 32.952 - type: recall_at_20 value: 43.49 - type: recall_at_100 value: 74.634 - type: recall_at_1000 value: 97.513 - type: precision_at_1 value: 78.0 - type: precision_at_3 value: 62.333000000000006 - type: precision_at_5 value: 53.7 - type: precision_at_10 value: 42.75 - type: precision_at_20 value: 33.125 - type: precision_at_100 value: 15.129999999999999 - type: precision_at_1000 value: 2.39 - type: mrr_at_1 value: 78.0 - type: mrr_at_3 value: 85.25 - type: mrr_at_5 value: 86.05000000000001 - type: mrr_at_10 value: 86.3069 - type: mrr_at_20 value: 86.3364 - type: mrr_at_100 value: 86.3549 - type: mrr_at_1000 value: 86.3549 - type: nauc_ndcg_at_1_max value: 39.9723 - type: nauc_ndcg_at_1_std value: 1.3915 - type: nauc_ndcg_at_1_diff1 value: 56.5282 - type: nauc_ndcg_at_3_max value: 33.899 - type: nauc_ndcg_at_3_std value: 4.6108 - type: nauc_ndcg_at_3_diff1 value: 28.0094 - type: nauc_ndcg_at_5_max value: 32.277699999999996 - type: nauc_ndcg_at_5_std value: 9.3641 - type: nauc_ndcg_at_5_diff1 value: 25.5435 - type: nauc_ndcg_at_10_max value: 29.232999999999997 - type: nauc_ndcg_at_10_std value: 11.2034 - type: nauc_ndcg_at_10_diff1 value: 23.6675 - type: nauc_ndcg_at_20_max value: 27.5872 - type: nauc_ndcg_at_20_std value: 11.1633 - type: nauc_ndcg_at_20_diff1 value: 21.0519 - type: nauc_ndcg_at_100_max value: 31.857999999999997 - type: nauc_ndcg_at_100_std value: 26.0807 - type: nauc_ndcg_at_100_diff1 value: 19.3271 - type: nauc_ndcg_at_1000_max value: 31.2605 - type: nauc_ndcg_at_1000_std value: 21.9248 - type: nauc_ndcg_at_1000_diff1 value: 19.2753 - type: nauc_map_at_1_max value: 11.0953 - type: nauc_map_at_1_std value: -16.169 - type: nauc_map_at_1_diff1 value: 43.9757 - type: nauc_map_at_3_max value: 15.6448 - type: nauc_map_at_3_std value: -11.5982 - type: nauc_map_at_3_diff1 value: 24.3589 - type: nauc_map_at_5_max value: 17.2011 - type: nauc_map_at_5_std value: -7.3374 - type: nauc_map_at_5_diff1 value: 21.685399999999998 - type: nauc_map_at_10_max value: 19.043499999999998 - type: nauc_map_at_10_std value: -3.2707 - type: nauc_map_at_10_diff1 value: 18.8752 - type: nauc_map_at_20_max value: 20.0765 - type: nauc_map_at_20_std value: 2.5831 - type: nauc_map_at_20_diff1 value: 17.7315 - type: nauc_map_at_100_max value: 20.599899999999998 - type: nauc_map_at_100_std value: 18.8259 - type: nauc_map_at_100_diff1 value: 10.5128 - type: nauc_map_at_1000_max value: 19.1446 - type: nauc_map_at_1000_std value: 19.122700000000002 - type: nauc_map_at_1000_diff1 value: 7.8306000000000004 - type: nauc_recall_at_1_max value: 11.0953 - type: nauc_recall_at_1_std value: -16.169 - type: nauc_recall_at_1_diff1 value: 43.9757 - type: nauc_recall_at_3_max value: 15.4575 - type: nauc_recall_at_3_std value: -10.2927 - type: nauc_recall_at_3_diff1 value: 21.893 - type: nauc_recall_at_5_max value: 14.186599999999999 - type: nauc_recall_at_5_std value: -6.930600000000001 - type: nauc_recall_at_5_diff1 value: 17.793400000000002 - type: nauc_recall_at_10_max value: 13.0639 - type: nauc_recall_at_10_std value: -4.063400000000001 - type: nauc_recall_at_10_diff1 value: 16.2193 - type: nauc_recall_at_20_max value: 13.801 - type: nauc_recall_at_20_std value: 2.5117000000000003 - type: nauc_recall_at_20_diff1 value: 13.7599 - type: nauc_recall_at_100_max value: 18.8061 - type: nauc_recall_at_100_std value: 34.467999999999996 - type: nauc_recall_at_100_diff1 value: 4.1441 - type: nauc_recall_at_1000_max value: 20.785899999999998 - type: nauc_recall_at_1000_std value: 63.2456 - type: nauc_recall_at_1000_diff1 value: -11.3226 - type: nauc_precision_at_1_max value: 38.5172 - type: nauc_precision_at_1_std value: 3.4462 - type: nauc_precision_at_1_diff1 value: 50.5645 - type: nauc_precision_at_3_max value: 19.0546 - type: nauc_precision_at_3_std value: 8.3792 - type: nauc_precision_at_3_diff1 value: -1.7430999999999999 - type: nauc_precision_at_5_max value: 16.231499999999997 - type: nauc_precision_at_5_std value: 18.2838 - type: nauc_precision_at_5_diff1 value: -5.2056 - type: nauc_precision_at_10_max value: 9.3026 - type: nauc_precision_at_10_std value: 23.149800000000003 - type: nauc_precision_at_10_diff1 value: -12.2439 - type: nauc_precision_at_20_max value: 1.4767000000000001 - type: nauc_precision_at_20_std value: 26.437300000000004 - type: nauc_precision_at_20_diff1 value: -16.6781 - type: nauc_precision_at_100_max value: -7.756200000000001 - type: nauc_precision_at_100_std value: 22.622999999999998 - type: nauc_precision_at_100_diff1 value: -21.8454 - type: nauc_precision_at_1000_max value: -17.3046 - type: nauc_precision_at_1000_std value: -0.4687 - type: nauc_precision_at_1000_diff1 value: -21.2201 - type: nauc_mrr_at_1_max value: 38.5172 - type: nauc_mrr_at_1_std value: 3.4462 - type: nauc_mrr_at_1_diff1 value: 50.5645 - type: nauc_mrr_at_3_max value: 48.4513 - type: nauc_mrr_at_3_std value: 10.053700000000001 - type: nauc_mrr_at_3_diff1 value: 46.6829 - type: nauc_mrr_at_5_max value: 46.7355 - type: nauc_mrr_at_5_std value: 8.5268 - type: nauc_mrr_at_5_diff1 value: 46.7111 - type: nauc_mrr_at_10_max value: 45.942499999999995 - type: nauc_mrr_at_10_std value: 7.4193 - type: nauc_mrr_at_10_diff1 value: 47.857699999999994 - type: nauc_mrr_at_20_max value: 45.8818 - type: nauc_mrr_at_20_std value: 7.475999999999999 - type: nauc_mrr_at_20_diff1 value: 47.7373 - type: nauc_mrr_at_100_max value: 45.822 - type: nauc_mrr_at_100_std value: 7.469 - type: nauc_mrr_at_100_diff1 value: 47.7016 - type: nauc_mrr_at_1000_max value: 45.822 - type: nauc_mrr_at_1000_std value: 7.469 - type: nauc_mrr_at_1000_diff1 value: 47.7016 - type: main_score value: 54.081999999999994 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: plastic_surgery type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 21.75 - type: ndcg_at_3 value: 18.837 - type: ndcg_at_5 value: 18.215999999999998 - type: ndcg_at_10 value: 18.512 - type: ndcg_at_20 value: 19.611 - type: ndcg_at_100 value: 26.711000000000002 - type: ndcg_at_1000 value: 36.405 - type: map_at_1 value: 2.932 - type: map_at_3 value: 4.791 - type: map_at_5 value: 5.998 - type: map_at_10 value: 7.713 - type: map_at_20 value: 9.075 - type: map_at_100 value: 12.352 - type: map_at_1000 value: 13.861 - type: recall_at_1 value: 2.932 - type: recall_at_3 value: 5.4510000000000005 - type: recall_at_5 value: 8.051 - type: recall_at_10 value: 12.638 - type: recall_at_20 value: 18.009 - type: recall_at_100 value: 38.15 - type: recall_at_1000 value: 71.577 - type: precision_at_1 value: 27.0 - type: precision_at_3 value: 19.833000000000002 - type: precision_at_5 value: 18.099999999999998 - type: precision_at_10 value: 15.75 - type: precision_at_20 value: 12.5 - type: precision_at_100 value: 6.52 - type: precision_at_1000 value: 1.498 - type: mrr_at_1 value: 27.0 - type: mrr_at_3 value: 31.833299999999998 - type: mrr_at_5 value: 33.7333 - type: mrr_at_10 value: 35.3665 - type: mrr_at_20 value: 35.9375 - type: mrr_at_100 value: 36.2769 - type: mrr_at_1000 value: 36.3373 - type: nauc_ndcg_at_1_max value: 43.717800000000004 - type: nauc_ndcg_at_1_std value: 35.2105 - type: nauc_ndcg_at_1_diff1 value: 28.829500000000003 - type: nauc_ndcg_at_3_max value: 43.349399999999996 - type: nauc_ndcg_at_3_std value: 39.1872 - type: nauc_ndcg_at_3_diff1 value: 21.1047 - type: nauc_ndcg_at_5_max value: 47.8451 - type: nauc_ndcg_at_5_std value: 42.5181 - type: nauc_ndcg_at_5_diff1 value: 19.078300000000002 - type: nauc_ndcg_at_10_max value: 49.788199999999996 - type: nauc_ndcg_at_10_std value: 46.5601 - type: nauc_ndcg_at_10_diff1 value: 16.556 - type: nauc_ndcg_at_20_max value: 51.60209999999999 - type: nauc_ndcg_at_20_std value: 47.5537 - type: nauc_ndcg_at_20_diff1 value: 16.5479 - type: nauc_ndcg_at_100_max value: 53.5291 - type: nauc_ndcg_at_100_std value: 59.6699 - type: nauc_ndcg_at_100_diff1 value: 13.365499999999999 - type: nauc_ndcg_at_1000_max value: 53.889900000000004 - type: nauc_ndcg_at_1000_std value: 62.61449999999999 - type: nauc_ndcg_at_1000_diff1 value: 11.686399999999999 - type: nauc_map_at_1_max value: 36.228 - type: nauc_map_at_1_std value: 16.9316 - type: nauc_map_at_1_diff1 value: 36.4257 - type: nauc_map_at_3_max value: 37.6485 - type: nauc_map_at_3_std value: 20.9514 - type: nauc_map_at_3_diff1 value: 24.4027 - type: nauc_map_at_5_max value: 41.510999999999996 - type: nauc_map_at_5_std value: 25.6374 - type: nauc_map_at_5_diff1 value: 21.309800000000003 - type: nauc_map_at_10_max value: 44.3985 - type: nauc_map_at_10_std value: 33.6395 - type: nauc_map_at_10_diff1 value: 18.4125 - type: nauc_map_at_20_max value: 46.2344 - type: nauc_map_at_20_std value: 39.099000000000004 - type: nauc_map_at_20_diff1 value: 17.7258 - type: nauc_map_at_100_max value: 48.254599999999996 - type: nauc_map_at_100_std value: 53.358000000000004 - type: nauc_map_at_100_diff1 value: 15.387999999999998 - type: nauc_map_at_1000_max value: 48.4559 - type: nauc_map_at_1000_std value: 56.648900000000005 - type: nauc_map_at_1000_diff1 value: 14.731 - type: nauc_recall_at_1_max value: 36.228 - type: nauc_recall_at_1_std value: 16.9316 - type: nauc_recall_at_1_diff1 value: 36.4257 - type: nauc_recall_at_3_max value: 35.6758 - type: nauc_recall_at_3_std value: 20.495 - type: nauc_recall_at_3_diff1 value: 20.2299 - type: nauc_recall_at_5_max value: 42.1255 - type: nauc_recall_at_5_std value: 26.890399999999996 - type: nauc_recall_at_5_diff1 value: 16.5719 - type: nauc_recall_at_10_max value: 41.8796 - type: nauc_recall_at_10_std value: 35.886 - type: nauc_recall_at_10_diff1 value: 9.186900000000001 - type: nauc_recall_at_20_max value: 42.714 - type: nauc_recall_at_20_std value: 38.8114 - type: nauc_recall_at_20_diff1 value: 8.1773 - type: nauc_recall_at_100_max value: 46.6919 - type: nauc_recall_at_100_std value: 61.7482 - type: nauc_recall_at_100_diff1 value: 3.8259000000000003 - type: nauc_recall_at_1000_max value: 55.9762 - type: nauc_recall_at_1000_std value: 71.4429 - type: nauc_recall_at_1000_diff1 value: 1.5741 - type: nauc_precision_at_1_max value: 45.0774 - type: nauc_precision_at_1_std value: 41.1016 - type: nauc_precision_at_1_diff1 value: 25.1666 - type: nauc_precision_at_3_max value: 43.3536 - type: nauc_precision_at_3_std value: 45.083400000000005 - type: nauc_precision_at_3_diff1 value: 14.815900000000001 - type: nauc_precision_at_5_max value: 48.3736 - type: nauc_precision_at_5_std value: 52.174600000000005 - type: nauc_precision_at_5_diff1 value: 12.2844 - type: nauc_precision_at_10_max value: 46.069700000000005 - type: nauc_precision_at_10_std value: 58.0045 - type: nauc_precision_at_10_diff1 value: 9.6372 - type: nauc_precision_at_20_max value: 44.2562 - type: nauc_precision_at_20_std value: 61.706799999999994 - type: nauc_precision_at_20_diff1 value: 11.1688 - type: nauc_precision_at_100_max value: 34.7099 - type: nauc_precision_at_100_std value: 66.8047 - type: nauc_precision_at_100_diff1 value: 5.9025 - type: nauc_precision_at_1000_max value: 11.375300000000001 - type: nauc_precision_at_1000_std value: 40.9784 - type: nauc_precision_at_1000_diff1 value: -5.2021 - type: nauc_mrr_at_1_max value: 45.0774 - type: nauc_mrr_at_1_std value: 41.1016 - type: nauc_mrr_at_1_diff1 value: 25.1666 - type: nauc_mrr_at_3_max value: 43.3192 - type: nauc_mrr_at_3_std value: 42.6595 - type: nauc_mrr_at_3_diff1 value: 19.1777 - type: nauc_mrr_at_5_max value: 46.1263 - type: nauc_mrr_at_5_std value: 45.2286 - type: nauc_mrr_at_5_diff1 value: 19.4694 - type: nauc_mrr_at_10_max value: 46.421600000000005 - type: nauc_mrr_at_10_std value: 46.0056 - type: nauc_mrr_at_10_diff1 value: 18.924599999999998 - type: nauc_mrr_at_20_max value: 46.7969 - type: nauc_mrr_at_20_std value: 45.9656 - type: nauc_mrr_at_20_diff1 value: 19.1375 - type: nauc_mrr_at_100_max value: 46.8001 - type: nauc_mrr_at_100_std value: 45.9121 - type: nauc_mrr_at_100_diff1 value: 19.1912 - type: nauc_mrr_at_1000_max value: 46.751 - type: nauc_mrr_at_1000_std value: 45.886500000000005 - type: nauc_mrr_at_1000_diff1 value: 19.189500000000002 - type: main_score value: 18.512 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: plastic_surgery type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 26.5 - type: ndcg_at_3 value: 23.357 - type: ndcg_at_5 value: 23.148 - type: ndcg_at_10 value: 24.154 - type: ndcg_at_20 value: 25.901000000000003 - type: ndcg_at_100 value: 34.075 - type: ndcg_at_1000 value: 43.693 - type: map_at_1 value: 3.202 - type: map_at_3 value: 6.101999999999999 - type: map_at_5 value: 7.611999999999999 - type: map_at_10 value: 10.245999999999999 - type: map_at_20 value: 12.4 - type: map_at_100 value: 16.489 - type: map_at_1000 value: 18.276 - type: recall_at_1 value: 3.202 - type: recall_at_3 value: 6.97 - type: recall_at_5 value: 9.805 - type: recall_at_10 value: 16.884 - type: recall_at_20 value: 24.099 - type: recall_at_100 value: 48.03 - type: recall_at_1000 value: 80.098 - type: precision_at_1 value: 32.0 - type: precision_at_3 value: 25.833000000000002 - type: precision_at_5 value: 23.9 - type: precision_at_10 value: 20.599999999999998 - type: precision_at_20 value: 16.575 - type: precision_at_100 value: 8.290000000000001 - type: precision_at_1000 value: 1.73 - type: mrr_at_1 value: 32.0 - type: mrr_at_3 value: 38.5 - type: mrr_at_5 value: 41.025 - type: mrr_at_10 value: 42.7242 - type: mrr_at_20 value: 43.2621 - type: mrr_at_100 value: 43.6464 - type: mrr_at_1000 value: 43.684 - type: nauc_ndcg_at_1_max value: 33.564899999999994 - type: nauc_ndcg_at_1_std value: 18.848200000000002 - type: nauc_ndcg_at_1_diff1 value: 20.8665 - type: nauc_ndcg_at_3_max value: 38.5293 - type: nauc_ndcg_at_3_std value: 30.883399999999998 - type: nauc_ndcg_at_3_diff1 value: 17.0333 - type: nauc_ndcg_at_5_max value: 43.662 - type: nauc_ndcg_at_5_std value: 34.9714 - type: nauc_ndcg_at_5_diff1 value: 15.587200000000001 - type: nauc_ndcg_at_10_max value: 49.431999999999995 - type: nauc_ndcg_at_10_std value: 41.0383 - type: nauc_ndcg_at_10_diff1 value: 19.7453 - type: nauc_ndcg_at_20_max value: 50.2147 - type: nauc_ndcg_at_20_std value: 45.2975 - type: nauc_ndcg_at_20_diff1 value: 21.377499999999998 - type: nauc_ndcg_at_100_max value: 56.59889999999999 - type: nauc_ndcg_at_100_std value: 56.2936 - type: nauc_ndcg_at_100_diff1 value: 19.9873 - type: nauc_ndcg_at_1000_max value: 55.2342 - type: nauc_ndcg_at_1000_std value: 57.22390000000001 - type: nauc_ndcg_at_1000_diff1 value: 18.0735 - type: nauc_map_at_1_max value: 34.0341 - type: nauc_map_at_1_std value: 5.3872 - type: nauc_map_at_1_diff1 value: 38.5789 - type: nauc_map_at_3_max value: 36.638 - type: nauc_map_at_3_std value: 21.882199999999997 - type: nauc_map_at_3_diff1 value: 29.220000000000002 - type: nauc_map_at_5_max value: 40.0682 - type: nauc_map_at_5_std value: 25.2585 - type: nauc_map_at_5_diff1 value: 25.2411 - type: nauc_map_at_10_max value: 45.246700000000004 - type: nauc_map_at_10_std value: 31.0429 - type: nauc_map_at_10_diff1 value: 25.9568 - type: nauc_map_at_20_max value: 45.726 - type: nauc_map_at_20_std value: 35.6744 - type: nauc_map_at_20_diff1 value: 24.348200000000002 - type: nauc_map_at_100_max value: 49.9123 - type: nauc_map_at_100_std value: 46.9594 - type: nauc_map_at_100_diff1 value: 19.500600000000002 - type: nauc_map_at_1000_max value: 49.6205 - type: nauc_map_at_1000_std value: 49.8661 - type: nauc_map_at_1000_diff1 value: 17.8803 - type: nauc_recall_at_1_max value: 34.0341 - type: nauc_recall_at_1_std value: 5.3872 - type: nauc_recall_at_1_diff1 value: 38.5789 - type: nauc_recall_at_3_max value: 39.1413 - type: nauc_recall_at_3_std value: 27.670699999999997 - type: nauc_recall_at_3_diff1 value: 27.185 - type: nauc_recall_at_5_max value: 41.5029 - type: nauc_recall_at_5_std value: 29.4584 - type: nauc_recall_at_5_diff1 value: 23.2386 - type: nauc_recall_at_10_max value: 45.006299999999996 - type: nauc_recall_at_10_std value: 33.4615 - type: nauc_recall_at_10_diff1 value: 25.6115 - type: nauc_recall_at_20_max value: 45.466699999999996 - type: nauc_recall_at_20_std value: 41.477000000000004 - type: nauc_recall_at_20_diff1 value: 22.309 - type: nauc_recall_at_100_max value: 51.815900000000006 - type: nauc_recall_at_100_std value: 61.633 - type: nauc_recall_at_100_diff1 value: 15.5623 - type: nauc_recall_at_1000_max value: 69.5645 - type: nauc_recall_at_1000_std value: 76.03370000000001 - type: nauc_recall_at_1000_diff1 value: 18.5411 - type: nauc_precision_at_1_max value: 33.3284 - type: nauc_precision_at_1_std value: 20.809 - type: nauc_precision_at_1_diff1 value: 17.3773 - type: nauc_precision_at_3_max value: 38.0662 - type: nauc_precision_at_3_std value: 37.0294 - type: nauc_precision_at_3_diff1 value: 10.726700000000001 - type: nauc_precision_at_5_max value: 42.142 - type: nauc_precision_at_5_std value: 40.913199999999996 - type: nauc_precision_at_5_diff1 value: 6.334 - type: nauc_precision_at_10_max value: 45.536 - type: nauc_precision_at_10_std value: 46.061800000000005 - type: nauc_precision_at_10_diff1 value: 9.435 - type: nauc_precision_at_20_max value: 39.5048 - type: nauc_precision_at_20_std value: 49.370799999999996 - type: nauc_precision_at_20_diff1 value: 4.9357999999999995 - type: nauc_precision_at_100_max value: 29.158800000000003 - type: nauc_precision_at_100_std value: 50.5371 - type: nauc_precision_at_100_diff1 value: -5.7985 - type: nauc_precision_at_1000_max value: -3.2552 - type: nauc_precision_at_1000_std value: 21.9283 - type: nauc_precision_at_1000_diff1 value: -13.078699999999998 - type: nauc_mrr_at_1_max value: 33.3284 - type: nauc_mrr_at_1_std value: 20.809 - type: nauc_mrr_at_1_diff1 value: 17.3773 - type: nauc_mrr_at_3_max value: 39.1492 - type: nauc_mrr_at_3_std value: 31.512400000000003 - type: nauc_mrr_at_3_diff1 value: 15.7237 - type: nauc_mrr_at_5_max value: 40.6122 - type: nauc_mrr_at_5_std value: 33.4723 - type: nauc_mrr_at_5_diff1 value: 16.361600000000003 - type: nauc_mrr_at_10_max value: 41.0804 - type: nauc_mrr_at_10_std value: 33.3934 - type: nauc_mrr_at_10_diff1 value: 17.163800000000002 - type: nauc_mrr_at_20_max value: 41.3014 - type: nauc_mrr_at_20_std value: 33.826299999999996 - type: nauc_mrr_at_20_diff1 value: 17.3245 - type: nauc_mrr_at_100_max value: 40.9816 - type: nauc_mrr_at_100_std value: 33.4721 - type: nauc_mrr_at_100_diff1 value: 17.1153 - type: nauc_mrr_at_1000_max value: 40.941 - type: nauc_mrr_at_1000_std value: 33.4166 - type: nauc_mrr_at_1000_diff1 value: 17.0834 - type: main_score value: 24.154 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: psychiatry_and_psychology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 62.25000000000001 - type: ndcg_at_3 value: 56.178 - type: ndcg_at_5 value: 55.35600000000001 - type: ndcg_at_10 value: 53.787 - type: ndcg_at_20 value: 54.425999999999995 - type: ndcg_at_100 value: 61.51 - type: ndcg_at_1000 value: 69.59100000000001 - type: map_at_1 value: 6.508 - type: map_at_3 value: 11.125 - type: map_at_5 value: 14.84 - type: map_at_10 value: 20.191 - type: map_at_20 value: 26.150000000000002 - type: map_at_100 value: 37.021 - type: map_at_1000 value: 42.846000000000004 - type: recall_at_1 value: 6.508 - type: recall_at_3 value: 12.055 - type: recall_at_5 value: 17.66 - type: recall_at_10 value: 26.485999999999997 - type: recall_at_20 value: 38.799 - type: recall_at_100 value: 68.774 - type: recall_at_1000 value: 93.62899999999999 - type: precision_at_1 value: 75.5 - type: precision_at_3 value: 64.667 - type: precision_at_5 value: 61.1 - type: precision_at_10 value: 52.6 - type: precision_at_20 value: 44.3 - type: precision_at_100 value: 23.735 - type: precision_at_1000 value: 5.314 - type: mrr_at_1 value: 75.5 - type: mrr_at_3 value: 80.8333 - type: mrr_at_5 value: 82.1833 - type: mrr_at_10 value: 82.6575 - type: mrr_at_20 value: 82.7818 - type: mrr_at_100 value: 82.8156 - type: mrr_at_1000 value: 82.8179 - type: nauc_ndcg_at_1_max value: 29.204200000000004 - type: nauc_ndcg_at_1_std value: 5.5276 - type: nauc_ndcg_at_1_diff1 value: 44.7735 - type: nauc_ndcg_at_3_max value: 39.7652 - type: nauc_ndcg_at_3_std value: 12.5479 - type: nauc_ndcg_at_3_diff1 value: 37.257200000000005 - type: nauc_ndcg_at_5_max value: 38.7411 - type: nauc_ndcg_at_5_std value: 17.310200000000002 - type: nauc_ndcg_at_5_diff1 value: 38.327600000000004 - type: nauc_ndcg_at_10_max value: 39.135799999999996 - type: nauc_ndcg_at_10_std value: 22.9689 - type: nauc_ndcg_at_10_diff1 value: 36.9699 - type: nauc_ndcg_at_20_max value: 39.9846 - type: nauc_ndcg_at_20_std value: 25.7726 - type: nauc_ndcg_at_20_diff1 value: 35.3069 - type: nauc_ndcg_at_100_max value: 46.7474 - type: nauc_ndcg_at_100_std value: 40.069700000000005 - type: nauc_ndcg_at_100_diff1 value: 29.802 - type: nauc_ndcg_at_1000_max value: 52.6498 - type: nauc_ndcg_at_1000_std value: 42.8565 - type: nauc_ndcg_at_1000_diff1 value: 35.4601 - type: nauc_map_at_1_max value: -16.932 - type: nauc_map_at_1_std value: -17.021 - type: nauc_map_at_1_diff1 value: 52.6778 - type: nauc_map_at_3_max value: -1.1856 - type: nauc_map_at_3_std value: -6.725299999999999 - type: nauc_map_at_3_diff1 value: 39.343 - type: nauc_map_at_5_max value: 4.2418 - type: nauc_map_at_5_std value: -0.39569999999999994 - type: nauc_map_at_5_diff1 value: 32.5041 - type: nauc_map_at_10_max value: 13.7455 - type: nauc_map_at_10_std value: 7.9563999999999995 - type: nauc_map_at_10_diff1 value: 27.549699999999998 - type: nauc_map_at_20_max value: 20.1616 - type: nauc_map_at_20_std value: 15.3559 - type: nauc_map_at_20_diff1 value: 24.5985 - type: nauc_map_at_100_max value: 35.7109 - type: nauc_map_at_100_std value: 35.8448 - type: nauc_map_at_100_diff1 value: 22.9314 - type: nauc_map_at_1000_max value: 40.5827 - type: nauc_map_at_1000_std value: 40.516999999999996 - type: nauc_map_at_1000_diff1 value: 21.8728 - type: nauc_recall_at_1_max value: -16.932 - type: nauc_recall_at_1_std value: -17.021 - type: nauc_recall_at_1_diff1 value: 52.6778 - type: nauc_recall_at_3_max value: -1.6465 - type: nauc_recall_at_3_std value: -4.1869000000000005 - type: nauc_recall_at_3_diff1 value: 36.6417 - type: nauc_recall_at_5_max value: -1.897 - type: nauc_recall_at_5_std value: -1.6251000000000002 - type: nauc_recall_at_5_diff1 value: 26.605 - type: nauc_recall_at_10_max value: 3.3542 - type: nauc_recall_at_10_std value: 2.3941 - type: nauc_recall_at_10_diff1 value: 21.5494 - type: nauc_recall_at_20_max value: 6.8939 - type: nauc_recall_at_20_std value: 6.0322000000000005 - type: nauc_recall_at_20_diff1 value: 15.322700000000001 - type: nauc_recall_at_100_max value: 22.0212 - type: nauc_recall_at_100_std value: 35.032000000000004 - type: nauc_recall_at_100_diff1 value: 12.5475 - type: nauc_recall_at_1000_max value: 41.3636 - type: nauc_recall_at_1000_std value: 61.2891 - type: nauc_recall_at_1000_diff1 value: 25.154700000000002 - type: nauc_precision_at_1_max value: 39.4009 - type: nauc_precision_at_1_std value: 13.0772 - type: nauc_precision_at_1_diff1 value: 44.709199999999996 - type: nauc_precision_at_3_max value: 45.9236 - type: nauc_precision_at_3_std value: 23.3125 - type: nauc_precision_at_3_diff1 value: 12.0883 - type: nauc_precision_at_5_max value: 39.2342 - type: nauc_precision_at_5_std value: 27.9706 - type: nauc_precision_at_5_diff1 value: 6.1063 - type: nauc_precision_at_10_max value: 36.578500000000005 - type: nauc_precision_at_10_std value: 30.2286 - type: nauc_precision_at_10_diff1 value: 0.212 - type: nauc_precision_at_20_max value: 31.939899999999998 - type: nauc_precision_at_20_std value: 29.0114 - type: nauc_precision_at_20_diff1 value: -1.5977000000000001 - type: nauc_precision_at_100_max value: 17.9772 - type: nauc_precision_at_100_std value: 19.254199999999997 - type: nauc_precision_at_100_diff1 value: -8.297400000000001 - type: nauc_precision_at_1000_max value: 1.124 - type: nauc_precision_at_1000_std value: -3.2368 - type: nauc_precision_at_1000_diff1 value: -11.5367 - type: nauc_mrr_at_1_max value: 39.4009 - type: nauc_mrr_at_1_std value: 13.0772 - type: nauc_mrr_at_1_diff1 value: 44.709199999999996 - type: nauc_mrr_at_3_max value: 47.9986 - type: nauc_mrr_at_3_std value: 22.703200000000002 - type: nauc_mrr_at_3_diff1 value: 48.8416 - type: nauc_mrr_at_5_max value: 46.2575 - type: nauc_mrr_at_5_std value: 21.3868 - type: nauc_mrr_at_5_diff1 value: 48.9122 - type: nauc_mrr_at_10_max value: 46.3047 - type: nauc_mrr_at_10_std value: 21.0973 - type: nauc_mrr_at_10_diff1 value: 48.5351 - type: nauc_mrr_at_20_max value: 46.056000000000004 - type: nauc_mrr_at_20_std value: 20.9263 - type: nauc_mrr_at_20_diff1 value: 48.5802 - type: nauc_mrr_at_100_max value: 46.0327 - type: nauc_mrr_at_100_std value: 20.9821 - type: nauc_mrr_at_100_diff1 value: 48.5011 - type: nauc_mrr_at_1000_max value: 46.0251 - type: nauc_mrr_at_1000_std value: 20.9709 - type: nauc_mrr_at_1000_diff1 value: 48.4956 - type: main_score value: 53.787 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: psychiatry_and_psychology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 15.25 - type: ndcg_at_3 value: 16.503999999999998 - type: ndcg_at_5 value: 17.229 - type: ndcg_at_10 value: 18.285 - type: ndcg_at_20 value: 20.498 - type: ndcg_at_100 value: 28.155 - type: ndcg_at_1000 value: 39.995999999999995 - type: map_at_1 value: 1.691 - type: map_at_3 value: 3.596 - type: map_at_5 value: 4.546 - type: map_at_10 value: 6.461 - type: map_at_20 value: 8.507000000000001 - type: map_at_100 value: 12.629999999999999 - type: map_at_1000 value: 16.005 - type: recall_at_1 value: 1.691 - type: recall_at_3 value: 4.661 - type: recall_at_5 value: 6.413 - type: recall_at_10 value: 11.055 - type: recall_at_20 value: 17.264 - type: recall_at_100 value: 39.257 - type: recall_at_1000 value: 73.616 - type: precision_at_1 value: 19.5 - type: precision_at_3 value: 21.5 - type: precision_at_5 value: 21.2 - type: precision_at_10 value: 19.8 - type: precision_at_20 value: 18.7 - type: precision_at_100 value: 11.97 - type: precision_at_1000 value: 3.6639999999999997 - type: mrr_at_1 value: 19.5 - type: mrr_at_3 value: 28.4167 - type: mrr_at_5 value: 30.3667 - type: mrr_at_10 value: 32.1278 - type: mrr_at_20 value: 33.194 - type: mrr_at_100 value: 33.679500000000004 - type: mrr_at_1000 value: 33.7017 - type: nauc_ndcg_at_1_max value: 8.3704 - type: nauc_ndcg_at_1_std value: 35.2143 - type: nauc_ndcg_at_1_diff1 value: 3.4162999999999997 - type: nauc_ndcg_at_3_max value: 10.3469 - type: nauc_ndcg_at_3_std value: 35.4444 - type: nauc_ndcg_at_3_diff1 value: -1.7632999999999999 - type: nauc_ndcg_at_5_max value: 9.8734 - type: nauc_ndcg_at_5_std value: 34.4609 - type: nauc_ndcg_at_5_diff1 value: -3.7905 - type: nauc_ndcg_at_10_max value: 10.8117 - type: nauc_ndcg_at_10_std value: 38.156400000000005 - type: nauc_ndcg_at_10_diff1 value: -2.3385 - type: nauc_ndcg_at_20_max value: 14.477100000000002 - type: nauc_ndcg_at_20_std value: 41.1031 - type: nauc_ndcg_at_20_diff1 value: -4.7229 - type: nauc_ndcg_at_100_max value: 23.9752 - type: nauc_ndcg_at_100_std value: 54.198299999999996 - type: nauc_ndcg_at_100_diff1 value: 0.47939999999999994 - type: nauc_ndcg_at_1000_max value: 31.2555 - type: nauc_ndcg_at_1000_std value: 56.81420000000001 - type: nauc_ndcg_at_1000_diff1 value: 0.7486999999999999 - type: nauc_map_at_1_max value: 5.1724 - type: nauc_map_at_1_std value: 37.5486 - type: nauc_map_at_1_diff1 value: 28.0577 - type: nauc_map_at_3_max value: 6.870900000000001 - type: nauc_map_at_3_std value: 35.1499 - type: nauc_map_at_3_diff1 value: 16.5024 - type: nauc_map_at_5_max value: 5.1497 - type: nauc_map_at_5_std value: 33.9484 - type: nauc_map_at_5_diff1 value: 9.6245 - type: nauc_map_at_10_max value: 2.4238 - type: nauc_map_at_10_std value: 34.7536 - type: nauc_map_at_10_diff1 value: 2.4971 - type: nauc_map_at_20_max value: 4.6439 - type: nauc_map_at_20_std value: 36.771300000000004 - type: nauc_map_at_20_diff1 value: -1.6913 - type: nauc_map_at_100_max value: 13.689200000000001 - type: nauc_map_at_100_std value: 46.6883 - type: nauc_map_at_100_diff1 value: -0.9819 - type: nauc_map_at_1000_max value: 19.6067 - type: nauc_map_at_1000_std value: 50.14809999999999 - type: nauc_map_at_1000_diff1 value: 0.21389999999999998 - type: nauc_recall_at_1_max value: 5.1724 - type: nauc_recall_at_1_std value: 37.5486 - type: nauc_recall_at_1_diff1 value: 28.0577 - type: nauc_recall_at_3_max value: 4.0584 - type: nauc_recall_at_3_std value: 32.493100000000005 - type: nauc_recall_at_3_diff1 value: 13.8324 - type: nauc_recall_at_5_max value: 0.9519 - type: nauc_recall_at_5_std value: 29.073999999999998 - type: nauc_recall_at_5_diff1 value: 4.2552 - type: nauc_recall_at_10_max value: -1.1522 - type: nauc_recall_at_10_std value: 30.8215 - type: nauc_recall_at_10_diff1 value: -0.7575000000000001 - type: nauc_recall_at_20_max value: 4.0141 - type: nauc_recall_at_20_std value: 34.6209 - type: nauc_recall_at_20_diff1 value: -4.9465 - type: nauc_recall_at_100_max value: 19.0596 - type: nauc_recall_at_100_std value: 53.043099999999995 - type: nauc_recall_at_100_diff1 value: -0.4443 - type: nauc_recall_at_1000_max value: 40.3603 - type: nauc_recall_at_1000_std value: 66.3722 - type: nauc_recall_at_1000_diff1 value: -5.2899 - type: nauc_precision_at_1_max value: 15.3147 - type: nauc_precision_at_1_std value: 38.4547 - type: nauc_precision_at_1_diff1 value: 5.2894 - type: nauc_precision_at_3_max value: 13.0357 - type: nauc_precision_at_3_std value: 34.6165 - type: nauc_precision_at_3_diff1 value: -6.7761000000000005 - type: nauc_precision_at_5_max value: 10.3074 - type: nauc_precision_at_5_std value: 31.929000000000002 - type: nauc_precision_at_5_diff1 value: -11.0482 - type: nauc_precision_at_10_max value: 12.5554 - type: nauc_precision_at_10_std value: 35.6601 - type: nauc_precision_at_10_diff1 value: -9.7417 - type: nauc_precision_at_20_max value: 18.0836 - type: nauc_precision_at_20_std value: 35.891 - type: nauc_precision_at_20_diff1 value: -11.1218 - type: nauc_precision_at_100_max value: 26.936799999999998 - type: nauc_precision_at_100_std value: 36.4249 - type: nauc_precision_at_100_diff1 value: -0.5038 - type: nauc_precision_at_1000_max value: 16.145899999999997 - type: nauc_precision_at_1000_std value: 8.6919 - type: nauc_precision_at_1000_diff1 value: 1.8063 - type: nauc_mrr_at_1_max value: 15.3147 - type: nauc_mrr_at_1_std value: 38.4547 - type: nauc_mrr_at_1_diff1 value: 5.2894 - type: nauc_mrr_at_3_max value: 14.596899999999998 - type: nauc_mrr_at_3_std value: 38.254599999999996 - type: nauc_mrr_at_3_diff1 value: 3.8059000000000003 - type: nauc_mrr_at_5_max value: 13.7185 - type: nauc_mrr_at_5_std value: 38.0272 - type: nauc_mrr_at_5_diff1 value: 2.6057 - type: nauc_mrr_at_10_max value: 15.759300000000001 - type: nauc_mrr_at_10_std value: 38.3291 - type: nauc_mrr_at_10_diff1 value: 4.0205 - type: nauc_mrr_at_20_max value: 16.0776 - type: nauc_mrr_at_20_std value: 38.6794 - type: nauc_mrr_at_20_diff1 value: 3.9222 - type: nauc_mrr_at_100_max value: 16.1438 - type: nauc_mrr_at_100_std value: 39.0121 - type: nauc_mrr_at_100_diff1 value: 4.0474000000000006 - type: nauc_mrr_at_1000_max value: 16.1074 - type: nauc_mrr_at_1000_std value: 38.9799 - type: nauc_mrr_at_1000_diff1 value: 4.018800000000001 - type: main_score value: 18.285 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: psychiatry_and_psychology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 34.5 - type: ndcg_at_3 value: 31.302999999999997 - type: ndcg_at_5 value: 30.329 - type: ndcg_at_10 value: 30.092000000000002 - type: ndcg_at_20 value: 31.141000000000002 - type: ndcg_at_100 value: 38.452 - type: ndcg_at_1000 value: 49.441 - type: map_at_1 value: 3.762 - type: map_at_3 value: 6.194 - type: map_at_5 value: 7.827000000000001 - type: map_at_10 value: 10.530000000000001 - type: map_at_20 value: 13.624 - type: map_at_100 value: 19.580000000000002 - type: map_at_1000 value: 23.517 - type: recall_at_1 value: 3.762 - type: recall_at_3 value: 6.935 - type: recall_at_5 value: 9.469 - type: recall_at_10 value: 14.774999999999999 - type: recall_at_20 value: 23.272000000000002 - type: recall_at_100 value: 47.899 - type: recall_at_1000 value: 79.336 - type: precision_at_1 value: 45.0 - type: precision_at_3 value: 37.5 - type: precision_at_5 value: 34.5 - type: precision_at_10 value: 30.65 - type: precision_at_20 value: 26.125 - type: precision_at_100 value: 14.75 - type: precision_at_1000 value: 4.055000000000001 - type: mrr_at_1 value: 45.0 - type: mrr_at_3 value: 53.41669999999999 - type: mrr_at_5 value: 54.8167 - type: mrr_at_10 value: 56.003400000000006 - type: mrr_at_20 value: 56.599900000000005 - type: mrr_at_100 value: 56.8053 - type: mrr_at_1000 value: 56.8178 - type: nauc_ndcg_at_1_max value: 21.7329 - type: nauc_ndcg_at_1_std value: 28.6663 - type: nauc_ndcg_at_1_diff1 value: 21.7183 - type: nauc_ndcg_at_3_max value: 33.7072 - type: nauc_ndcg_at_3_std value: 35.4064 - type: nauc_ndcg_at_3_diff1 value: 20.9404 - type: nauc_ndcg_at_5_max value: 36.1588 - type: nauc_ndcg_at_5_std value: 41.077799999999996 - type: nauc_ndcg_at_5_diff1 value: 18.8722 - type: nauc_ndcg_at_10_max value: 36.8808 - type: nauc_ndcg_at_10_std value: 42.6325 - type: nauc_ndcg_at_10_diff1 value: 17.203 - type: nauc_ndcg_at_20_max value: 40.5287 - type: nauc_ndcg_at_20_std value: 48.8626 - type: nauc_ndcg_at_20_diff1 value: 12.2369 - type: nauc_ndcg_at_100_max value: 52.285000000000004 - type: nauc_ndcg_at_100_std value: 62.0211 - type: nauc_ndcg_at_100_diff1 value: 7.3617 - type: nauc_ndcg_at_1000_max value: 52.5857 - type: nauc_ndcg_at_1000_std value: 64.48270000000001 - type: nauc_ndcg_at_1000_diff1 value: 11.462 - type: nauc_map_at_1_max value: -4.9144 - type: nauc_map_at_1_std value: 9.7903 - type: nauc_map_at_1_diff1 value: 31.9263 - type: nauc_map_at_3_max value: 11.4121 - type: nauc_map_at_3_std value: 20.4973 - type: nauc_map_at_3_diff1 value: 22.2687 - type: nauc_map_at_5_max value: 18.5748 - type: nauc_map_at_5_std value: 27.561799999999998 - type: nauc_map_at_5_diff1 value: 17.8388 - type: nauc_map_at_10_max value: 27.125799999999998 - type: nauc_map_at_10_std value: 33.840399999999995 - type: nauc_map_at_10_diff1 value: 12.963099999999999 - type: nauc_map_at_20_max value: 33.469500000000004 - type: nauc_map_at_20_std value: 40.9153 - type: nauc_map_at_20_diff1 value: 10.1626 - type: nauc_map_at_100_max value: 47.2695 - type: nauc_map_at_100_std value: 56.0531 - type: nauc_map_at_100_diff1 value: 8.020199999999999 - type: nauc_map_at_1000_max value: 47.847899999999996 - type: nauc_map_at_1000_std value: 58.7227 - type: nauc_map_at_1000_diff1 value: 8.697199999999999 - type: nauc_recall_at_1_max value: -4.9144 - type: nauc_recall_at_1_std value: 9.7903 - type: nauc_recall_at_1_diff1 value: 31.9263 - type: nauc_recall_at_3_max value: 11.7904 - type: nauc_recall_at_3_std value: 21.4997 - type: nauc_recall_at_3_diff1 value: 19.6193 - type: nauc_recall_at_5_max value: 19.039900000000003 - type: nauc_recall_at_5_std value: 29.3609 - type: nauc_recall_at_5_diff1 value: 13.7371 - type: nauc_recall_at_10_max value: 25.318800000000003 - type: nauc_recall_at_10_std value: 30.9394 - type: nauc_recall_at_10_diff1 value: 7.7126 - type: nauc_recall_at_20_max value: 31.7363 - type: nauc_recall_at_20_std value: 41.167300000000004 - type: nauc_recall_at_20_diff1 value: 1.2625000000000002 - type: nauc_recall_at_100_max value: 49.5017 - type: nauc_recall_at_100_std value: 62.5455 - type: nauc_recall_at_100_diff1 value: -6.0973 - type: nauc_recall_at_1000_max value: 62.021800000000006 - type: nauc_recall_at_1000_std value: 82.6167 - type: nauc_recall_at_1000_diff1 value: -7.6475 - type: nauc_precision_at_1_max value: 24.5636 - type: nauc_precision_at_1_std value: 30.032700000000002 - type: nauc_precision_at_1_diff1 value: 20.9605 - type: nauc_precision_at_3_max value: 38.6768 - type: nauc_precision_at_3_std value: 36.7568 - type: nauc_precision_at_3_diff1 value: 13.77 - type: nauc_precision_at_5_max value: 40.002900000000004 - type: nauc_precision_at_5_std value: 43.7761 - type: nauc_precision_at_5_diff1 value: 9.8675 - type: nauc_precision_at_10_max value: 39.5 - type: nauc_precision_at_10_std value: 43.282900000000005 - type: nauc_precision_at_10_diff1 value: 8.8047 - type: nauc_precision_at_20_max value: 38.8534 - type: nauc_precision_at_20_std value: 43.6591 - type: nauc_precision_at_20_diff1 value: 3.0647 - type: nauc_precision_at_100_max value: 28.808600000000002 - type: nauc_precision_at_100_std value: 34.5524 - type: nauc_precision_at_100_diff1 value: 2.9145 - type: nauc_precision_at_1000_max value: 0.8090999999999999 - type: nauc_precision_at_1000_std value: 3.0013 - type: nauc_precision_at_1000_diff1 value: 12.712499999999999 - type: nauc_mrr_at_1_max value: 24.5636 - type: nauc_mrr_at_1_std value: 30.032700000000002 - type: nauc_mrr_at_1_diff1 value: 20.9605 - type: nauc_mrr_at_3_max value: 29.911199999999997 - type: nauc_mrr_at_3_std value: 36.0742 - type: nauc_mrr_at_3_diff1 value: 22.6104 - type: nauc_mrr_at_5_max value: 29.8299 - type: nauc_mrr_at_5_std value: 36.584 - type: nauc_mrr_at_5_diff1 value: 22.5318 - type: nauc_mrr_at_10_max value: 29.4982 - type: nauc_mrr_at_10_std value: 35.9069 - type: nauc_mrr_at_10_diff1 value: 22.4779 - type: nauc_mrr_at_20_max value: 29.834600000000002 - type: nauc_mrr_at_20_std value: 36.0663 - type: nauc_mrr_at_20_diff1 value: 22.4151 - type: nauc_mrr_at_100_max value: 29.674699999999998 - type: nauc_mrr_at_100_std value: 35.988 - type: nauc_mrr_at_100_diff1 value: 22.2975 - type: nauc_mrr_at_1000_max value: 29.6539 - type: nauc_mrr_at_1000_std value: 35.9683 - type: nauc_mrr_at_1000_diff1 value: 22.3039 - type: main_score value: 30.092000000000002 task: type: Retrieval - dataset: config: en name: MTEB CUREv1 (en) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: pulmonology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 64.0 - type: ndcg_at_3 value: 57.730000000000004 - type: ndcg_at_5 value: 56.537000000000006 - type: ndcg_at_10 value: 55.355 - type: ndcg_at_20 value: 56.119 - type: ndcg_at_100 value: 62.271 - type: ndcg_at_1000 value: 70.458 - type: map_at_1 value: 12.120000000000001 - type: map_at_3 value: 19.903000000000002 - type: map_at_5 value: 23.996000000000002 - type: map_at_10 value: 28.767 - type: map_at_20 value: 32.574999999999996 - type: map_at_100 value: 39.367999999999995 - type: map_at_1000 value: 43.169999999999995 - type: recall_at_1 value: 12.120000000000001 - type: recall_at_3 value: 21.934 - type: recall_at_5 value: 28.09 - type: recall_at_10 value: 36.939 - type: recall_at_20 value: 46.831 - type: recall_at_100 value: 71.05900000000001 - type: recall_at_1000 value: 95.543 - type: precision_at_1 value: 73.5 - type: precision_at_3 value: 57.667 - type: precision_at_5 value: 51.1 - type: precision_at_10 value: 40.400000000000006 - type: precision_at_20 value: 30.925000000000004 - type: precision_at_100 value: 15.310000000000002 - type: precision_at_1000 value: 3.4299999999999997 - type: mrr_at_1 value: 73.5 - type: mrr_at_3 value: 80.91669999999999 - type: mrr_at_5 value: 82.36670000000001 - type: mrr_at_10 value: 82.6792 - type: mrr_at_20 value: 82.8638 - type: mrr_at_100 value: 82.8638 - type: mrr_at_1000 value: 82.8638 - type: nauc_ndcg_at_1_max value: 52.9944 - type: nauc_ndcg_at_1_std value: 10.6488 - type: nauc_ndcg_at_1_diff1 value: 56.655199999999994 - type: nauc_ndcg_at_3_max value: 51.953 - type: nauc_ndcg_at_3_std value: 13.8674 - type: nauc_ndcg_at_3_diff1 value: 40.9794 - type: nauc_ndcg_at_5_max value: 50.5956 - type: nauc_ndcg_at_5_std value: 14.1906 - type: nauc_ndcg_at_5_diff1 value: 39.0879 - type: nauc_ndcg_at_10_max value: 46.3615 - type: nauc_ndcg_at_10_std value: 17.5279 - type: nauc_ndcg_at_10_diff1 value: 35.3915 - type: nauc_ndcg_at_20_max value: 45.8408 - type: nauc_ndcg_at_20_std value: 14.768500000000001 - type: nauc_ndcg_at_20_diff1 value: 37.9504 - type: nauc_ndcg_at_100_max value: 51.45269999999999 - type: nauc_ndcg_at_100_std value: 25.4131 - type: nauc_ndcg_at_100_diff1 value: 41.646699999999996 - type: nauc_ndcg_at_1000_max value: 53.3425 - type: nauc_ndcg_at_1000_std value: 26.118599999999997 - type: nauc_ndcg_at_1000_diff1 value: 39.6503 - type: nauc_map_at_1_max value: 25.144899999999996 - type: nauc_map_at_1_std value: 6.5377 - type: nauc_map_at_1_diff1 value: 55.247 - type: nauc_map_at_3_max value: 29.1222 - type: nauc_map_at_3_std value: 12.2911 - type: nauc_map_at_3_diff1 value: 44.0726 - type: nauc_map_at_5_max value: 31.3535 - type: nauc_map_at_5_std value: 14.696200000000001 - type: nauc_map_at_5_diff1 value: 38.48 - type: nauc_map_at_10_max value: 33.285 - type: nauc_map_at_10_std value: 16.5916 - type: nauc_map_at_10_diff1 value: 33.197700000000005 - type: nauc_map_at_20_max value: 35.9366 - type: nauc_map_at_20_std value: 16.9539 - type: nauc_map_at_20_diff1 value: 33.4713 - type: nauc_map_at_100_max value: 39.4216 - type: nauc_map_at_100_std value: 23.5005 - type: nauc_map_at_100_diff1 value: 32.3674 - type: nauc_map_at_1000_max value: 38.9144 - type: nauc_map_at_1000_std value: 22.9379 - type: nauc_map_at_1000_diff1 value: 29.9875 - type: nauc_recall_at_1_max value: 25.144899999999996 - type: nauc_recall_at_1_std value: 6.5377 - type: nauc_recall_at_1_diff1 value: 55.247 - type: nauc_recall_at_3_max value: 24.9541 - type: nauc_recall_at_3_std value: 12.2005 - type: nauc_recall_at_3_diff1 value: 39.8432 - type: nauc_recall_at_5_max value: 26.035000000000004 - type: nauc_recall_at_5_std value: 15.7753 - type: nauc_recall_at_5_diff1 value: 33.602900000000005 - type: nauc_recall_at_10_max value: 29.224099999999996 - type: nauc_recall_at_10_std value: 16.9009 - type: nauc_recall_at_10_diff1 value: 27.7753 - type: nauc_recall_at_20_max value: 30.2261 - type: nauc_recall_at_20_std value: 14.480599999999999 - type: nauc_recall_at_20_diff1 value: 29.133599999999998 - type: nauc_recall_at_100_max value: 38.1305 - type: nauc_recall_at_100_std value: 31.1575 - type: nauc_recall_at_100_diff1 value: 35.969 - type: nauc_recall_at_1000_max value: 58.8372 - type: nauc_recall_at_1000_std value: 76.1479 - type: nauc_recall_at_1000_diff1 value: 36.6072 - type: nauc_precision_at_1_max value: 57.537099999999995 - type: nauc_precision_at_1_std value: 15.8427 - type: nauc_precision_at_1_diff1 value: 50.3828 - type: nauc_precision_at_3_max value: 33.5024 - type: nauc_precision_at_3_std value: 11.182699999999999 - type: nauc_precision_at_3_diff1 value: -0.21960000000000002 - type: nauc_precision_at_5_max value: 24.208199999999998 - type: nauc_precision_at_5_std value: 7.292 - type: nauc_precision_at_5_diff1 value: -9.8187 - type: nauc_precision_at_10_max value: 11.2527 - type: nauc_precision_at_10_std value: 5.273 - type: nauc_precision_at_10_diff1 value: -20.255300000000002 - type: nauc_precision_at_20_max value: 4.5758 - type: nauc_precision_at_20_std value: 1.9272999999999998 - type: nauc_precision_at_20_diff1 value: -18.9272 - type: nauc_precision_at_100_max value: -6.4853 - type: nauc_precision_at_100_std value: -0.35729999999999995 - type: nauc_precision_at_100_diff1 value: -19.8495 - type: nauc_precision_at_1000_max value: -15.5076 - type: nauc_precision_at_1000_std value: -19.7998 - type: nauc_precision_at_1000_diff1 value: -23.3793 - type: nauc_mrr_at_1_max value: 57.537099999999995 - type: nauc_mrr_at_1_std value: 15.8427 - type: nauc_mrr_at_1_diff1 value: 50.3828 - type: nauc_mrr_at_3_max value: 60.519400000000005 - type: nauc_mrr_at_3_std value: 25.4011 - type: nauc_mrr_at_3_diff1 value: 47.106700000000004 - type: nauc_mrr_at_5_max value: 59.9727 - type: nauc_mrr_at_5_std value: 23.0994 - type: nauc_mrr_at_5_diff1 value: 46.9457 - type: nauc_mrr_at_10_max value: 59.78869999999999 - type: nauc_mrr_at_10_std value: 22.3365 - type: nauc_mrr_at_10_diff1 value: 46.8909 - type: nauc_mrr_at_20_max value: 59.5865 - type: nauc_mrr_at_20_std value: 22.070600000000002 - type: nauc_mrr_at_20_diff1 value: 47.1936 - type: nauc_mrr_at_100_max value: 59.5865 - type: nauc_mrr_at_100_std value: 22.070600000000002 - type: nauc_mrr_at_100_diff1 value: 47.1936 - type: nauc_mrr_at_1000_max value: 59.5865 - type: nauc_mrr_at_1000_std value: 22.070600000000002 - type: nauc_mrr_at_1000_diff1 value: 47.1936 - type: main_score value: 55.355 task: type: Retrieval - dataset: config: es name: MTEB CUREv1 (es) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: pulmonology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 18.0 - type: ndcg_at_3 value: 17.581 - type: ndcg_at_5 value: 17.521 - type: ndcg_at_10 value: 17.929000000000002 - type: ndcg_at_20 value: 19.194 - type: ndcg_at_100 value: 24.951 - type: ndcg_at_1000 value: 33.986 - type: map_at_1 value: 3.5130000000000003 - type: map_at_3 value: 5.822 - type: map_at_5 value: 6.927 - type: map_at_10 value: 8.567 - type: map_at_20 value: 9.804 - type: map_at_100 value: 12.208 - type: map_at_1000 value: 13.876 - type: recall_at_1 value: 3.5130000000000003 - type: recall_at_3 value: 7.031 - type: recall_at_5 value: 9.747 - type: recall_at_10 value: 14.171 - type: recall_at_20 value: 19.304 - type: recall_at_100 value: 37.382 - type: recall_at_1000 value: 67.855 - type: precision_at_1 value: 22.5 - type: precision_at_3 value: 19.0 - type: precision_at_5 value: 16.7 - type: precision_at_10 value: 13.700000000000001 - type: precision_at_20 value: 11.125 - type: precision_at_100 value: 6.61 - type: precision_at_1000 value: 2.091 - type: mrr_at_1 value: 22.5 - type: mrr_at_3 value: 28.8333 - type: mrr_at_5 value: 30.8333 - type: mrr_at_10 value: 32.0377 - type: mrr_at_20 value: 32.6114 - type: mrr_at_100 value: 32.983000000000004 - type: mrr_at_1000 value: 33.0393 - type: nauc_ndcg_at_1_max value: 43.932500000000005 - type: nauc_ndcg_at_1_std value: 30.9222 - type: nauc_ndcg_at_1_diff1 value: 19.9076 - type: nauc_ndcg_at_3_max value: 52.4309 - type: nauc_ndcg_at_3_std value: 33.2502 - type: nauc_ndcg_at_3_diff1 value: 21.6392 - type: nauc_ndcg_at_5_max value: 53.7562 - type: nauc_ndcg_at_5_std value: 38.7857 - type: nauc_ndcg_at_5_diff1 value: 20.2335 - type: nauc_ndcg_at_10_max value: 56.3609 - type: nauc_ndcg_at_10_std value: 41.5947 - type: nauc_ndcg_at_10_diff1 value: 22.6265 - type: nauc_ndcg_at_20_max value: 59.8171 - type: nauc_ndcg_at_20_std value: 44.1169 - type: nauc_ndcg_at_20_diff1 value: 22.7763 - type: nauc_ndcg_at_100_max value: 63.490100000000005 - type: nauc_ndcg_at_100_std value: 50.1014 - type: nauc_ndcg_at_100_diff1 value: 20.4293 - type: nauc_ndcg_at_1000_max value: 63.620200000000004 - type: nauc_ndcg_at_1000_std value: 51.35809999999999 - type: nauc_ndcg_at_1000_diff1 value: 17.1076 - type: nauc_map_at_1_max value: 42.8805 - type: nauc_map_at_1_std value: 34.9412 - type: nauc_map_at_1_diff1 value: 51.257299999999994 - type: nauc_map_at_3_max value: 49.3844 - type: nauc_map_at_3_std value: 31.5361 - type: nauc_map_at_3_diff1 value: 42.1771 - type: nauc_map_at_5_max value: 52.0142 - type: nauc_map_at_5_std value: 38.156099999999995 - type: nauc_map_at_5_diff1 value: 34.7349 - type: nauc_map_at_10_max value: 55.47410000000001 - type: nauc_map_at_10_std value: 41.1528 - type: nauc_map_at_10_diff1 value: 30.3408 - type: nauc_map_at_20_max value: 57.971399999999996 - type: nauc_map_at_20_std value: 43.3687 - type: nauc_map_at_20_diff1 value: 28.723300000000002 - type: nauc_map_at_100_max value: 61.5196 - type: nauc_map_at_100_std value: 47.762100000000004 - type: nauc_map_at_100_diff1 value: 26.5933 - type: nauc_map_at_1000_max value: 62.327299999999994 - type: nauc_map_at_1000_std value: 48.6106 - type: nauc_map_at_1000_diff1 value: 24.8347 - type: nauc_recall_at_1_max value: 42.8805 - type: nauc_recall_at_1_std value: 34.9412 - type: nauc_recall_at_1_diff1 value: 51.257299999999994 - type: nauc_recall_at_3_max value: 46.008500000000005 - type: nauc_recall_at_3_std value: 25.9357 - type: nauc_recall_at_3_diff1 value: 39.5208 - type: nauc_recall_at_5_max value: 45.2357 - type: nauc_recall_at_5_std value: 41.3928 - type: nauc_recall_at_5_diff1 value: 24.9619 - type: nauc_recall_at_10_max value: 46.6621 - type: nauc_recall_at_10_std value: 38.1389 - type: nauc_recall_at_10_diff1 value: 22.9227 - type: nauc_recall_at_20_max value: 51.809400000000004 - type: nauc_recall_at_20_std value: 44.9357 - type: nauc_recall_at_20_diff1 value: 19.8542 - type: nauc_recall_at_100_max value: 56.9719 - type: nauc_recall_at_100_std value: 55.10810000000001 - type: nauc_recall_at_100_diff1 value: 15.0191 - type: nauc_recall_at_1000_max value: 61.3058 - type: nauc_recall_at_1000_std value: 74.4173 - type: nauc_recall_at_1000_diff1 value: 11.5847 - type: nauc_precision_at_1_max value: 44.6099 - type: nauc_precision_at_1_std value: 34.2709 - type: nauc_precision_at_1_diff1 value: 12.620600000000001 - type: nauc_precision_at_3_max value: 52.7269 - type: nauc_precision_at_3_std value: 35.4988 - type: nauc_precision_at_3_diff1 value: 5.5428 - type: nauc_precision_at_5_max value: 52.2888 - type: nauc_precision_at_5_std value: 40.1329 - type: nauc_precision_at_5_diff1 value: 0.7597999999999999 - type: nauc_precision_at_10_max value: 53.4776 - type: nauc_precision_at_10_std value: 41.308099999999996 - type: nauc_precision_at_10_diff1 value: 1.2727 - type: nauc_precision_at_20_max value: 51.581900000000005 - type: nauc_precision_at_20_std value: 41.4412 - type: nauc_precision_at_20_diff1 value: 1.8686999999999998 - type: nauc_precision_at_100_max value: 37.2452 - type: nauc_precision_at_100_std value: 29.8866 - type: nauc_precision_at_100_diff1 value: -2.653 - type: nauc_precision_at_1000_max value: 10.8915 - type: nauc_precision_at_1000_std value: -1.9983 - type: nauc_precision_at_1000_diff1 value: -7.8149 - type: nauc_mrr_at_1_max value: 44.6099 - type: nauc_mrr_at_1_std value: 34.2709 - type: nauc_mrr_at_1_diff1 value: 12.620600000000001 - type: nauc_mrr_at_3_max value: 47.1393 - type: nauc_mrr_at_3_std value: 33.9716 - type: nauc_mrr_at_3_diff1 value: 9.6941 - type: nauc_mrr_at_5_max value: 46.381099999999996 - type: nauc_mrr_at_5_std value: 36.4238 - type: nauc_mrr_at_5_diff1 value: 8.3056 - type: nauc_mrr_at_10_max value: 47.5779 - type: nauc_mrr_at_10_std value: 36.6239 - type: nauc_mrr_at_10_diff1 value: 8.732099999999999 - type: nauc_mrr_at_20_max value: 48.2236 - type: nauc_mrr_at_20_std value: 37.2659 - type: nauc_mrr_at_20_diff1 value: 9.1473 - type: nauc_mrr_at_100_max value: 48.231 - type: nauc_mrr_at_100_std value: 37.335499999999996 - type: nauc_mrr_at_100_diff1 value: 8.94 - type: nauc_mrr_at_1000_max value: 48.1705 - type: nauc_mrr_at_1000_std value: 37.278299999999994 - type: nauc_mrr_at_1000_diff1 value: 8.9861 - type: main_score value: 17.929000000000002 task: type: Retrieval - dataset: config: fr name: MTEB CUREv1 (fr) revision: 3bcf51c91e04d04a8a3329dfbe988b964c5cbe83 split: pulmonology type: clinia/CUREv1 metrics: - type: ndcg_at_1 value: 22.0 - type: ndcg_at_3 value: 19.67 - type: ndcg_at_5 value: 19.637999999999998 - type: ndcg_at_10 value: 20.093 - type: ndcg_at_20 value: 22.037000000000003 - type: ndcg_at_100 value: 27.633999999999997 - type: ndcg_at_1000 value: 36.504999999999995 - type: map_at_1 value: 4.806 - type: map_at_3 value: 6.3839999999999995 - type: map_at_5 value: 7.617 - type: map_at_10 value: 9.371 - type: map_at_20 value: 10.822 - type: map_at_100 value: 13.616 - type: map_at_1000 value: 15.429 - type: recall_at_1 value: 4.806 - type: recall_at_3 value: 7.5009999999999994 - type: recall_at_5 value: 10.041 - type: recall_at_10 value: 14.588999999999999 - type: recall_at_20 value: 22.256 - type: recall_at_100 value: 40.071 - type: recall_at_1000 value: 69.741 - type: precision_at_1 value: 27.500000000000004 - type: precision_at_3 value: 19.833000000000002 - type: precision_at_5 value: 18.3 - type: precision_at_10 value: 15.5 - type: precision_at_20 value: 12.875 - type: precision_at_100 value: 7.380000000000001 - type: precision_at_1000 value: 2.186 - type: mrr_at_1 value: 27.500000000000004 - type: mrr_at_3 value: 33.6667 - type: mrr_at_5 value: 35.341699999999996 - type: mrr_at_10 value: 36.432700000000004 - type: mrr_at_20 value: 37.297200000000004 - type: mrr_at_100 value: 37.675599999999996 - type: mrr_at_1000 value: 37.722899999999996 - type: nauc_ndcg_at_1_max value: 42.6021 - type: nauc_ndcg_at_1_std value: 35.9813 - type: nauc_ndcg_at_1_diff1 value: 22.290399999999998 - type: nauc_ndcg_at_3_max value: 49.1559 - type: nauc_ndcg_at_3_std value: 40.0896 - type: nauc_ndcg_at_3_diff1 value: 22.279799999999998 - type: nauc_ndcg_at_5_max value: 51.4668 - type: nauc_ndcg_at_5_std value: 39.624900000000004 - type: nauc_ndcg_at_5_diff1 value: 22.3741 - type: nauc_ndcg_at_10_max value: 54.519 - type: nauc_ndcg_at_10_std value: 42.104200000000006 - type: nauc_ndcg_at_10_diff1 value: 22.4078 - type: nauc_ndcg_at_20_max value: 56.2174 - type: nauc_ndcg_at_20_std value: 46.512100000000004 - type: nauc_ndcg_at_20_diff1 value: 23.6843 - type: nauc_ndcg_at_100_max value: 61.04619999999999 - type: nauc_ndcg_at_100_std value: 57.2706 - type: nauc_ndcg_at_100_diff1 value: 23.2432 - type: nauc_ndcg_at_1000_max value: 61.364200000000004 - type: nauc_ndcg_at_1000_std value: 56.0934 - type: nauc_ndcg_at_1000_diff1 value: 17.2488 - type: nauc_map_at_1_max value: 44.9878 - type: nauc_map_at_1_std value: 34.9461 - type: nauc_map_at_1_diff1 value: 53.675399999999996 - type: nauc_map_at_3_max value: 44.4329 - type: nauc_map_at_3_std value: 36.0323 - type: nauc_map_at_3_diff1 value: 46.4853 - type: nauc_map_at_5_max value: 46.2582 - type: nauc_map_at_5_std value: 36.9879 - type: nauc_map_at_5_diff1 value: 41.4037 - type: nauc_map_at_10_max value: 50.330200000000005 - type: nauc_map_at_10_std value: 40.0156 - type: nauc_map_at_10_diff1 value: 38.1695 - type: nauc_map_at_20_max value: 51.7941 - type: nauc_map_at_20_std value: 44.193599999999996 - type: nauc_map_at_20_diff1 value: 35.8223 - type: nauc_map_at_100_max value: 54.651 - type: nauc_map_at_100_std value: 50.6749 - type: nauc_map_at_100_diff1 value: 31.0403 - type: nauc_map_at_1000_max value: 54.2956 - type: nauc_map_at_1000_std value: 50.2575 - type: nauc_map_at_1000_diff1 value: 26.8654 - type: nauc_recall_at_1_max value: 44.9878 - type: nauc_recall_at_1_std value: 34.9461 - type: nauc_recall_at_1_diff1 value: 53.675399999999996 - type: nauc_recall_at_3_max value: 40.9893 - type: nauc_recall_at_3_std value: 34.5723 - type: nauc_recall_at_3_diff1 value: 41.6989 - type: nauc_recall_at_5_max value: 43.0016 - type: nauc_recall_at_5_std value: 32.7617 - type: nauc_recall_at_5_diff1 value: 34.6036 - type: nauc_recall_at_10_max value: 51.5082 - type: nauc_recall_at_10_std value: 39.9773 - type: nauc_recall_at_10_diff1 value: 30.900899999999996 - type: nauc_recall_at_20_max value: 48.149300000000004 - type: nauc_recall_at_20_std value: 45.0434 - type: nauc_recall_at_20_diff1 value: 28.865800000000004 - type: nauc_recall_at_100_max value: 56.163799999999995 - type: nauc_recall_at_100_std value: 65.4097 - type: nauc_recall_at_100_diff1 value: 24.0152 - type: nauc_recall_at_1000_max value: 65.8655 - type: nauc_recall_at_1000_std value: 74.5213 - type: nauc_recall_at_1000_diff1 value: 15.1259 - type: nauc_precision_at_1_max value: 39.8972 - type: nauc_precision_at_1_std value: 33.6627 - type: nauc_precision_at_1_diff1 value: 17.1124 - type: nauc_precision_at_3_max value: 44.7058 - type: nauc_precision_at_3_std value: 37.6182 - type: nauc_precision_at_3_diff1 value: 8.8976 - type: nauc_precision_at_5_max value: 43.9751 - type: nauc_precision_at_5_std value: 34.65 - type: nauc_precision_at_5_diff1 value: 2.9363 - type: nauc_precision_at_10_max value: 41.3611 - type: nauc_precision_at_10_std value: 35.741299999999995 - type: nauc_precision_at_10_diff1 value: -0.1869 - type: nauc_precision_at_20_max value: 37.6713 - type: nauc_precision_at_20_std value: 37.3803 - type: nauc_precision_at_20_diff1 value: -6.543699999999999 - type: nauc_precision_at_100_max value: 21.4265 - type: nauc_precision_at_100_std value: 24.3006 - type: nauc_precision_at_100_diff1 value: -18.1773 - type: nauc_precision_at_1000_max value: 0.7231 - type: nauc_precision_at_1000_std value: -7.500500000000001 - type: nauc_precision_at_1000_diff1 value: -27.672400000000003 - type: nauc_mrr_at_1_max value: 39.8972 - type: nauc_mrr_at_1_std value: 33.6627 - type: nauc_mrr_at_1_diff1 value: 17.1124 - type: nauc_mrr_at_3_max value: 44.0551 - type: nauc_mrr_at_3_std value: 35.9342 - type: nauc_mrr_at_3_diff1 value: 16.797 - type: nauc_mrr_at_5_max value: 44.575700000000005 - type: nauc_mrr_at_5_std value: 35.5613 - type: nauc_mrr_at_5_diff1 value: 15.8189 - type: nauc_mrr_at_10_max value: 45.3991 - type: nauc_mrr_at_10_std value: 37.3274 - type: nauc_mrr_at_10_diff1 value: 15.509899999999998 - type: nauc_mrr_at_20_max value: 45.6017 - type: nauc_mrr_at_20_std value: 37.7156 - type: nauc_mrr_at_20_diff1 value: 15.357499999999998 - type: nauc_mrr_at_100_max value: 45.5056 - type: nauc_mrr_at_100_std value: 37.734899999999996 - type: nauc_mrr_at_100_diff1 value: 15.382599999999998 - type: nauc_mrr_at_1000_max value: 45.4736 - type: nauc_mrr_at_1000_std value: 37.6894 - type: nauc_mrr_at_1000_diff1 value: 15.3948 - type: main_score value: 20.093 task: type: Retrieval - dataset: config: default name: MTEB CmedqaRetrieval (default) revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301 split: dev type: C-MTEB/CmedqaRetrieval metrics: - type: ndcg_at_1 value: 1.975 - type: ndcg_at_3 value: 1.8800000000000001 - type: ndcg_at_5 value: 2.064 - type: ndcg_at_10 value: 2.244 - type: ndcg_at_20 value: 2.444 - type: ndcg_at_100 value: 3.1419999999999995 - type: ndcg_at_1000 value: 4.436 - type: map_at_1 value: 1.0959999999999999 - type: map_at_3 value: 1.459 - type: map_at_5 value: 1.5970000000000002 - type: map_at_10 value: 1.678 - type: map_at_20 value: 1.73 - type: map_at_100 value: 1.817 - type: map_at_1000 value: 1.854 - type: recall_at_1 value: 1.0959999999999999 - type: recall_at_3 value: 1.8399999999999999 - type: recall_at_5 value: 2.411 - type: recall_at_10 value: 2.9690000000000003 - type: recall_at_20 value: 3.65 - type: recall_at_100 value: 6.882000000000001 - type: recall_at_1000 value: 16.221 - type: precision_at_1 value: 1.975 - type: precision_at_3 value: 1.134 - type: precision_at_5 value: 0.855 - type: precision_at_10 value: 0.545 - type: precision_at_20 value: 0.34299999999999997 - type: precision_at_100 value: 0.136 - type: precision_at_1000 value: 0.031 - type: mrr_at_1 value: 1.9755000000000003 - type: mrr_at_3 value: 2.4548 - type: mrr_at_5 value: 2.6336 - type: mrr_at_10 value: 2.7566 - type: mrr_at_20 value: 2.834 - type: mrr_at_100 value: 2.948 - type: mrr_at_1000 value: 2.9941 - type: nauc_ndcg_at_1_max value: 44.5578 - type: nauc_ndcg_at_1_std value: -7.0793 - type: nauc_ndcg_at_1_diff1 value: 41.342800000000004 - type: nauc_ndcg_at_3_max value: 38.824799999999996 - type: nauc_ndcg_at_3_std value: -7.721400000000001 - type: nauc_ndcg_at_3_diff1 value: 29.5746 - type: nauc_ndcg_at_5_max value: 35.3778 - type: nauc_ndcg_at_5_std value: -8.0892 - type: nauc_ndcg_at_5_diff1 value: 28.1742 - type: nauc_ndcg_at_10_max value: 33.9489 - type: nauc_ndcg_at_10_std value: -6.5869 - type: nauc_ndcg_at_10_diff1 value: 25.1357 - type: nauc_ndcg_at_20_max value: 31.6749 - type: nauc_ndcg_at_20_std value: -6.1487 - type: nauc_ndcg_at_20_diff1 value: 22.4549 - type: nauc_ndcg_at_100_max value: 25.5529 - type: nauc_ndcg_at_100_std value: -3.3800999999999997 - type: nauc_ndcg_at_100_diff1 value: 18.2302 - type: nauc_ndcg_at_1000_max value: 21.4586 - type: nauc_ndcg_at_1000_std value: 0.3195 - type: nauc_ndcg_at_1000_diff1 value: 15.187800000000001 - type: nauc_map_at_1_max value: 48.9561 - type: nauc_map_at_1_std value: -8.7168 - type: nauc_map_at_1_diff1 value: 41.5248 - type: nauc_map_at_3_max value: 41.7416 - type: nauc_map_at_3_std value: -8.9731 - type: nauc_map_at_3_diff1 value: 30.8288 - type: nauc_map_at_5_max value: 39.2026 - type: nauc_map_at_5_std value: -9.1096 - type: nauc_map_at_5_diff1 value: 29.8599 - type: nauc_map_at_10_max value: 38.3244 - type: nauc_map_at_10_std value: -8.3488 - type: nauc_map_at_10_diff1 value: 28.4995 - type: nauc_map_at_20_max value: 37.4247 - type: nauc_map_at_20_std value: -8.132200000000001 - type: nauc_map_at_20_diff1 value: 27.494000000000003 - type: nauc_map_at_100_max value: 35.9232 - type: nauc_map_at_100_std value: -7.3904 - type: nauc_map_at_100_diff1 value: 26.3365 - type: nauc_map_at_1000_max value: 35.5077 - type: nauc_map_at_1000_std value: -7.093000000000001 - type: nauc_map_at_1000_diff1 value: 25.9599 - type: nauc_recall_at_1_max value: 48.9561 - type: nauc_recall_at_1_std value: -8.7168 - type: nauc_recall_at_1_diff1 value: 41.5248 - type: nauc_recall_at_3_max value: 35.0824 - type: nauc_recall_at_3_std value: -6.9858 - type: nauc_recall_at_3_diff1 value: 24.1649 - type: nauc_recall_at_5_max value: 29.3684 - type: nauc_recall_at_5_std value: -8.2378 - type: nauc_recall_at_5_diff1 value: 23.042299999999997 - type: nauc_recall_at_10_max value: 27.4364 - type: nauc_recall_at_10_std value: -5.7612 - type: nauc_recall_at_10_diff1 value: 18.4245 - type: nauc_recall_at_20_max value: 23.4658 - type: nauc_recall_at_20_std value: -5.0295 - type: nauc_recall_at_20_diff1 value: 13.6712 - type: nauc_recall_at_100_max value: 14.056199999999999 - type: nauc_recall_at_100_std value: -0.0629 - type: nauc_recall_at_100_diff1 value: 9.0138 - type: nauc_recall_at_1000_max value: 10.1503 - type: nauc_recall_at_1000_std value: 6.651 - type: nauc_recall_at_1000_diff1 value: 6.7867999999999995 - type: nauc_precision_at_1_max value: 44.5578 - type: nauc_precision_at_1_std value: -7.0793 - type: nauc_precision_at_1_diff1 value: 41.342800000000004 - type: nauc_precision_at_3_max value: 35.142 - type: nauc_precision_at_3_std value: -6.3729 - type: nauc_precision_at_3_diff1 value: 23.5105 - type: nauc_precision_at_5_max value: 31.5182 - type: nauc_precision_at_5_std value: -5.9487 - type: nauc_precision_at_5_diff1 value: 23.244899999999998 - type: nauc_precision_at_10_max value: 28.1848 - type: nauc_precision_at_10_std value: -2.1538000000000004 - type: nauc_precision_at_10_diff1 value: 18.1237 - type: nauc_precision_at_20_max value: 23.6964 - type: nauc_precision_at_20_std value: -1.5044 - type: nauc_precision_at_20_diff1 value: 13.669300000000002 - type: nauc_precision_at_100_max value: 14.132900000000001 - type: nauc_precision_at_100_std value: 3.6346000000000003 - type: nauc_precision_at_100_diff1 value: 9.3907 - type: nauc_precision_at_1000_max value: 9.5808 - type: nauc_precision_at_1000_std value: 7.452400000000001 - type: nauc_precision_at_1000_diff1 value: 7.3365 - type: nauc_mrr_at_1_max value: 44.5578 - type: nauc_mrr_at_1_std value: -7.0793 - type: nauc_mrr_at_1_diff1 value: 41.342800000000004 - type: nauc_mrr_at_3_max value: 38.055299999999995 - type: nauc_mrr_at_3_std value: -6.456199999999999 - type: nauc_mrr_at_3_diff1 value: 32.067099999999996 - type: nauc_mrr_at_5_max value: 36.5336 - type: nauc_mrr_at_5_std value: -6.546399999999999 - type: nauc_mrr_at_5_diff1 value: 31.506899999999998 - type: nauc_mrr_at_10_max value: 35.57 - type: nauc_mrr_at_10_std value: -5.7427 - type: nauc_mrr_at_10_diff1 value: 30.0035 - type: nauc_mrr_at_20_max value: 34.6904 - type: nauc_mrr_at_20_std value: -5.6437 - type: nauc_mrr_at_20_diff1 value: 29.002200000000002 - type: nauc_mrr_at_100_max value: 33.377 - type: nauc_mrr_at_100_std value: -5.2307 - type: nauc_mrr_at_100_diff1 value: 27.822999999999997 - type: nauc_mrr_at_1000_max value: 33.1079 - type: nauc_mrr_at_1000_std value: -5.066 - type: nauc_mrr_at_1000_diff1 value: 27.535700000000002 - type: main_score value: 2.244 task: type: Retrieval - dataset: config: default name: MTEB MedicalQARetrieval (default) revision: ae763399273d8b20506b80cf6f6f9a31a6a2b238 split: test type: mteb/medical_qa metrics: - type: ndcg_at_1 value: 61.816 - type: ndcg_at_3 value: 70.759 - type: ndcg_at_5 value: 72.968 - type: ndcg_at_10 value: 74.824 - type: ndcg_at_20 value: 75.82799999999999 - type: ndcg_at_100 value: 76.846 - type: ndcg_at_1000 value: 77.164 - type: map_at_1 value: 61.816 - type: map_at_3 value: 68.612 - type: map_at_5 value: 69.83500000000001 - type: map_at_10 value: 70.616 - type: map_at_20 value: 70.899 - type: map_at_100 value: 71.043 - type: map_at_1000 value: 71.055 - type: recall_at_1 value: 61.816 - type: recall_at_3 value: 76.953 - type: recall_at_5 value: 82.324 - type: recall_at_10 value: 87.988 - type: recall_at_20 value: 91.89500000000001 - type: recall_at_100 value: 97.363 - type: recall_at_1000 value: 99.854 - type: precision_at_1 value: 61.816 - type: precision_at_3 value: 25.651000000000003 - type: precision_at_5 value: 16.465 - type: precision_at_10 value: 8.799 - type: precision_at_20 value: 4.595 - type: precision_at_100 value: 0.9740000000000001 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 61.914100000000005 - type: mrr_at_3 value: 68.6686 - type: mrr_at_5 value: 69.8942 - type: mrr_at_10 value: 70.6751 - type: mrr_at_20 value: 70.9589 - type: mrr_at_100 value: 71.102 - type: mrr_at_1000 value: 71.11420000000001 - type: nauc_ndcg_at_1_max value: 43.0416 - type: nauc_ndcg_at_1_std value: 1.4047 - type: nauc_ndcg_at_1_diff1 value: 68.2899 - type: nauc_ndcg_at_3_max value: 44.2637 - type: nauc_ndcg_at_3_std value: -2.6632 - type: nauc_ndcg_at_3_diff1 value: 62.948499999999996 - type: nauc_ndcg_at_5_max value: 44.3739 - type: nauc_ndcg_at_5_std value: -3.1071 - type: nauc_ndcg_at_5_diff1 value: 62.4586 - type: nauc_ndcg_at_10_max value: 45.7976 - type: nauc_ndcg_at_10_std value: -4.037 - type: nauc_ndcg_at_10_diff1 value: 62.5016 - type: nauc_ndcg_at_20_max value: 45.4416 - type: nauc_ndcg_at_20_std value: -3.3076000000000003 - type: nauc_ndcg_at_20_diff1 value: 62.64619999999999 - type: nauc_ndcg_at_100_max value: 45.216 - type: nauc_ndcg_at_100_std value: -2.6107 - type: nauc_ndcg_at_100_diff1 value: 63.1536 - type: nauc_ndcg_at_1000_max value: 44.8511 - type: nauc_ndcg_at_1000_std value: -2.2796 - type: nauc_ndcg_at_1000_diff1 value: 63.5203 - type: nauc_map_at_1_max value: 43.0416 - type: nauc_map_at_1_std value: 1.4047 - type: nauc_map_at_1_diff1 value: 68.2899 - type: nauc_map_at_3_max value: 43.8357 - type: nauc_map_at_3_std value: -1.4727999999999999 - type: nauc_map_at_3_diff1 value: 64.3811 - type: nauc_map_at_5_max value: 43.865300000000005 - type: nauc_map_at_5_std value: -1.6657000000000002 - type: nauc_map_at_5_diff1 value: 64.1906 - type: nauc_map_at_10_max value: 44.376599999999996 - type: nauc_map_at_10_std value: -2.004 - type: nauc_map_at_10_diff1 value: 64.2587 - type: nauc_map_at_20_max value: 44.289 - type: nauc_map_at_20_std value: -1.8488999999999998 - type: nauc_map_at_20_diff1 value: 64.3021 - type: nauc_map_at_100_max value: 44.2802 - type: nauc_map_at_100_std value: -1.7392999999999998 - type: nauc_map_at_100_diff1 value: 64.3597 - type: nauc_map_at_1000_max value: 44.2691 - type: nauc_map_at_1000_std value: -1.72 - type: nauc_map_at_1000_diff1 value: 64.37349999999999 - type: nauc_recall_at_1_max value: 43.0416 - type: nauc_recall_at_1_std value: 1.4047 - type: nauc_recall_at_1_diff1 value: 68.2899 - type: nauc_recall_at_3_max value: 45.8586 - type: nauc_recall_at_3_std value: -7.0022 - type: nauc_recall_at_3_diff1 value: 57.791599999999995 - type: nauc_recall_at_5_max value: 46.698299999999996 - type: nauc_recall_at_5_std value: -9.518699999999999 - type: nauc_recall_at_5_diff1 value: 54.8081 - type: nauc_recall_at_10_max value: 55.617700000000006 - type: nauc_recall_at_10_std value: -17.0066 - type: nauc_recall_at_10_diff1 value: 51.7949 - type: nauc_recall_at_20_max value: 56.2234 - type: nauc_recall_at_20_std value: -14.6945 - type: nauc_recall_at_20_diff1 value: 48.6265 - type: nauc_recall_at_100_max value: 66.3957 - type: nauc_recall_at_100_std value: -13.4665 - type: nauc_recall_at_100_diff1 value: 41.1961 - type: nauc_recall_at_1000_max value: 47.873599999999996 - type: nauc_recall_at_1000_std value: 61.4994 - type: nauc_recall_at_1000_diff1 value: 33.012 - type: nauc_precision_at_1_max value: 43.0416 - type: nauc_precision_at_1_std value: 1.4047 - type: nauc_precision_at_1_diff1 value: 68.2899 - type: nauc_precision_at_3_max value: 45.8586 - type: nauc_precision_at_3_std value: -7.0022 - type: nauc_precision_at_3_diff1 value: 57.791599999999995 - type: nauc_precision_at_5_max value: 46.698299999999996 - type: nauc_precision_at_5_std value: -9.518699999999999 - type: nauc_precision_at_5_diff1 value: 54.8081 - type: nauc_precision_at_10_max value: 55.617700000000006 - type: nauc_precision_at_10_std value: -17.0066 - type: nauc_precision_at_10_diff1 value: 51.7949 - type: nauc_precision_at_20_max value: 56.2234 - type: nauc_precision_at_20_std value: -14.6945 - type: nauc_precision_at_20_diff1 value: 48.6265 - type: nauc_precision_at_100_max value: 66.3957 - type: nauc_precision_at_100_std value: -13.4665 - type: nauc_precision_at_100_diff1 value: 41.1961 - type: nauc_precision_at_1000_max value: 47.873599999999996 - type: nauc_precision_at_1000_std value: 61.4994 - type: nauc_precision_at_1000_diff1 value: 33.012 - type: nauc_mrr_at_1_max value: 43.147000000000006 - type: nauc_mrr_at_1_std value: 1.6458 - type: nauc_mrr_at_1_diff1 value: 68.0902 - type: nauc_mrr_at_3_max value: 43.904199999999996 - type: nauc_mrr_at_3_std value: -1.3296999999999999 - type: nauc_mrr_at_3_diff1 value: 64.24459999999999 - type: nauc_mrr_at_5_max value: 43.9347 - type: nauc_mrr_at_5_std value: -1.5104 - type: nauc_mrr_at_5_diff1 value: 64.0435 - type: nauc_mrr_at_10_max value: 44.4408 - type: nauc_mrr_at_10_std value: -1.8647 - type: nauc_mrr_at_10_diff1 value: 64.1077 - type: nauc_mrr_at_20_max value: 44.3536 - type: nauc_mrr_at_20_std value: -1.7078 - type: nauc_mrr_at_20_diff1 value: 64.1495 - type: nauc_mrr_at_100_max value: 44.3452 - type: nauc_mrr_at_100_std value: -1.5973000000000002 - type: nauc_mrr_at_100_diff1 value: 64.2063 - type: nauc_mrr_at_1000_max value: 44.3341 - type: nauc_mrr_at_1000_std value: -1.5779 - type: nauc_mrr_at_1000_diff1 value: 64.22 - type: main_score value: 74.824 task: type: Retrieval - dataset: config: default name: MTEB MedrxivClusteringP2P.v2 (default) revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 split: test type: mteb/medrxiv-clustering-p2p metrics: - type: v_measure value: 36.208200000000005 - type: v_measure_std value: 1.3513000000000002 - type: main_score value: 36.208200000000005 task: type: Clustering - dataset: config: default name: MTEB MedrxivClusteringS2S.v2 (default) revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 split: test type: mteb/medrxiv-clustering-s2s metrics: - type: v_measure value: 34.1653 - type: v_measure_std value: 0.5923 - type: main_score value: 34.1653 task: type: Clustering - dataset: config: default name: MTEB NFCorpus (default) revision: ec0fa4fe99da2ff19ca1214b7966684033a58814 split: test type: mteb/nfcorpus metrics: - type: ndcg_at_1 value: 47.368 - type: ndcg_at_3 value: 43.149 - type: ndcg_at_5 value: 40.262 - type: ndcg_at_10 value: 37.07 - type: ndcg_at_20 value: 35.258 - type: ndcg_at_100 value: 34.742 - type: ndcg_at_1000 value: 43.864999999999995 - type: map_at_1 value: 7.298 - type: map_at_3 value: 11.259 - type: map_at_5 value: 12.756 - type: map_at_10 value: 14.829 - type: map_at_20 value: 16.543 - type: map_at_100 value: 18.755 - type: map_at_1000 value: 20.302999999999997 - type: recall_at_1 value: 7.298 - type: recall_at_3 value: 12.056000000000001 - type: recall_at_5 value: 14.184 - type: recall_at_10 value: 18.224999999999998 - type: recall_at_20 value: 23.009 - type: recall_at_100 value: 35.398 - type: recall_at_1000 value: 67.74900000000001 - type: precision_at_1 value: 48.916 - type: precision_at_3 value: 39.938 - type: precision_at_5 value: 33.994 - type: precision_at_10 value: 26.779999999999998 - type: precision_at_20 value: 20.355999999999998 - type: precision_at_100 value: 8.458 - type: precision_at_1000 value: 2.176 - type: mrr_at_1 value: 48.916399999999996 - type: mrr_at_3 value: 55.985600000000005 - type: mrr_at_5 value: 57.0537 - type: mrr_at_10 value: 57.7149 - type: mrr_at_20 value: 58.1229 - type: mrr_at_100 value: 58.397299999999994 - type: mrr_at_1000 value: 58.4248 - type: nauc_ndcg_at_1_max value: 41.3116 - type: nauc_ndcg_at_1_std value: 22.451 - type: nauc_ndcg_at_1_diff1 value: 37.2109 - type: nauc_ndcg_at_3_max value: 40.1035 - type: nauc_ndcg_at_3_std value: 24.5928 - type: nauc_ndcg_at_3_diff1 value: 24.430699999999998 - type: nauc_ndcg_at_5_max value: 39.734 - type: nauc_ndcg_at_5_std value: 25.881500000000003 - type: nauc_ndcg_at_5_diff1 value: 22.0016 - type: nauc_ndcg_at_10_max value: 38.4436 - type: nauc_ndcg_at_10_std value: 27.4864 - type: nauc_ndcg_at_10_diff1 value: 21.456400000000002 - type: nauc_ndcg_at_20_max value: 37.5296 - type: nauc_ndcg_at_20_std value: 26.714100000000002 - type: nauc_ndcg_at_20_diff1 value: 22.9013 - type: nauc_ndcg_at_100_max value: 40.7159 - type: nauc_ndcg_at_100_std value: 27.7585 - type: nauc_ndcg_at_100_diff1 value: 25.681700000000003 - type: nauc_ndcg_at_1000_max value: 45.828799999999994 - type: nauc_ndcg_at_1000_std value: 35.1004 - type: nauc_ndcg_at_1000_diff1 value: 25.220799999999997 - type: nauc_map_at_1_max value: 15.6695 - type: nauc_map_at_1_std value: -10.7993 - type: nauc_map_at_1_diff1 value: 42.3173 - type: nauc_map_at_3_max value: 20.509900000000002 - type: nauc_map_at_3_std value: -5.0278 - type: nauc_map_at_3_diff1 value: 39.326299999999996 - type: nauc_map_at_5_max value: 23.020699999999998 - type: nauc_map_at_5_std value: -2.0001 - type: nauc_map_at_5_diff1 value: 36.3273 - type: nauc_map_at_10_max value: 25.6403 - type: nauc_map_at_10_std value: 3.4666 - type: nauc_map_at_10_diff1 value: 32.1434 - type: nauc_map_at_20_max value: 27.7762 - type: nauc_map_at_20_std value: 8.83 - type: nauc_map_at_20_diff1 value: 29.428700000000003 - type: nauc_map_at_100_max value: 30.4644 - type: nauc_map_at_100_std value: 14.0983 - type: nauc_map_at_100_diff1 value: 27.7847 - type: nauc_map_at_1000_max value: 31.1749 - type: nauc_map_at_1000_std value: 16.8873 - type: nauc_map_at_1000_diff1 value: 26.736500000000003 - type: nauc_recall_at_1_max value: 15.6695 - type: nauc_recall_at_1_std value: -10.7993 - type: nauc_recall_at_1_diff1 value: 42.3173 - type: nauc_recall_at_3_max value: 19.5384 - type: nauc_recall_at_3_std value: -4.2829 - type: nauc_recall_at_3_diff1 value: 37.924 - type: nauc_recall_at_5_max value: 21.6262 - type: nauc_recall_at_5_std value: -0.6713 - type: nauc_recall_at_5_diff1 value: 34.423500000000004 - type: nauc_recall_at_10_max value: 21.7819 - type: nauc_recall_at_10_std value: 3.4214 - type: nauc_recall_at_10_diff1 value: 28.447499999999998 - type: nauc_recall_at_20_max value: 23.880499999999998 - type: nauc_recall_at_20_std value: 9.232899999999999 - type: nauc_recall_at_20_diff1 value: 26.002 - type: nauc_recall_at_100_max value: 25.6814 - type: nauc_recall_at_100_std value: 15.804599999999999 - type: nauc_recall_at_100_diff1 value: 16.955000000000002 - type: nauc_recall_at_1000_max value: 22.5706 - type: nauc_recall_at_1000_std value: 25.684600000000003 - type: nauc_recall_at_1000_diff1 value: 7.257099999999999 - type: nauc_precision_at_1_max value: 43.9778 - type: nauc_precision_at_1_std value: 23.9525 - type: nauc_precision_at_1_diff1 value: 34.5267 - type: nauc_precision_at_3_max value: 38.661899999999996 - type: nauc_precision_at_3_std value: 29.4867 - type: nauc_precision_at_3_diff1 value: 12.2659 - type: nauc_precision_at_5_max value: 37.0542 - type: nauc_precision_at_5_std value: 33.0248 - type: nauc_precision_at_5_diff1 value: 3.8455999999999997 - type: nauc_precision_at_10_max value: 34.932900000000004 - type: nauc_precision_at_10_std value: 38.5304 - type: nauc_precision_at_10_diff1 value: -2.7742 - type: nauc_precision_at_20_max value: 29.228199999999998 - type: nauc_precision_at_20_std value: 39.0561 - type: nauc_precision_at_20_diff1 value: -6.049 - type: nauc_precision_at_100_max value: 18.4449 - type: nauc_precision_at_100_std value: 35.3197 - type: nauc_precision_at_100_diff1 value: -8.9064 - type: nauc_precision_at_1000_max value: 3.8112 - type: nauc_precision_at_1000_std value: 23.8397 - type: nauc_precision_at_1000_diff1 value: -13.475200000000001 - type: nauc_mrr_at_1_max value: 43.9778 - type: nauc_mrr_at_1_std value: 23.9525 - type: nauc_mrr_at_1_diff1 value: 34.5267 - type: nauc_mrr_at_3_max value: 46.333 - type: nauc_mrr_at_3_std value: 27.969 - type: nauc_mrr_at_3_diff1 value: 34.537600000000005 - type: nauc_mrr_at_5_max value: 47.0493 - type: nauc_mrr_at_5_std value: 29.6659 - type: nauc_mrr_at_5_diff1 value: 33.9539 - type: nauc_mrr_at_10_max value: 47.1933 - type: nauc_mrr_at_10_std value: 30.4836 - type: nauc_mrr_at_10_diff1 value: 33.794200000000004 - type: nauc_mrr_at_20_max value: 47.5342 - type: nauc_mrr_at_20_std value: 30.714999999999996 - type: nauc_mrr_at_20_diff1 value: 34.042899999999996 - type: nauc_mrr_at_100_max value: 47.5202 - type: nauc_mrr_at_100_std value: 30.534299999999998 - type: nauc_mrr_at_100_diff1 value: 33.8418 - type: nauc_mrr_at_1000_max value: 47.4983 - type: nauc_mrr_at_1000_std value: 30.5077 - type: nauc_mrr_at_1000_diff1 value: 33.856700000000004 - type: main_score value: 37.07 task: type: Retrieval - dataset: config: arabic name: MTEB PublicHealthQA (arabic) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 3.488 - type: ndcg_at_3 value: 7.738 - type: ndcg_at_5 value: 9.639000000000001 - type: ndcg_at_10 value: 12.396 - type: ndcg_at_20 value: 17.115 - type: ndcg_at_100 value: 27.512999999999998 - type: ndcg_at_1000 value: 27.512999999999998 - type: map_at_1 value: 3.488 - type: map_at_3 value: 6.783 - type: map_at_5 value: 7.829 - type: map_at_10 value: 9.044 - type: map_at_20 value: 10.347000000000001 - type: map_at_100 value: 11.632000000000001 - type: map_at_1000 value: 11.632000000000001 - type: recall_at_1 value: 3.488 - type: recall_at_3 value: 10.465 - type: recall_at_5 value: 15.116 - type: recall_at_10 value: 23.256 - type: recall_at_20 value: 41.86 - type: recall_at_100 value: 100.0 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 3.488 - type: precision_at_3 value: 3.488 - type: precision_at_5 value: 3.023 - type: precision_at_10 value: 2.326 - type: precision_at_20 value: 2.093 - type: precision_at_100 value: 1.0 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 3.4884 - type: mrr_at_3 value: 6.7829 - type: mrr_at_5 value: 7.8295 - type: mrr_at_10 value: 9.0439 - type: mrr_at_20 value: 10.3465 - type: mrr_at_100 value: 11.6317 - type: mrr_at_1000 value: 11.6317 - type: nauc_ndcg_at_1_max value: 59.715700000000005 - type: nauc_ndcg_at_1_std value: 37.2948 - type: nauc_ndcg_at_1_diff1 value: 77.5791 - type: nauc_ndcg_at_3_max value: 35.650999999999996 - type: nauc_ndcg_at_3_std value: 4.0767 - type: nauc_ndcg_at_3_diff1 value: 48.6041 - type: nauc_ndcg_at_5_max value: 26.639400000000002 - type: nauc_ndcg_at_5_std value: -2.3021 - type: nauc_ndcg_at_5_diff1 value: 53.8186 - type: nauc_ndcg_at_10_max value: 20.5654 - type: nauc_ndcg_at_10_std value: -0.2084 - type: nauc_ndcg_at_10_diff1 value: 46.2555 - type: nauc_ndcg_at_20_max value: 18.3581 - type: nauc_ndcg_at_20_std value: 7.5809 - type: nauc_ndcg_at_20_diff1 value: 39.1175 - type: nauc_ndcg_at_100_max value: 23.2254 - type: nauc_ndcg_at_100_std value: 4.8341 - type: nauc_ndcg_at_100_diff1 value: 45.2908 - type: nauc_ndcg_at_1000_max value: 23.2254 - type: nauc_ndcg_at_1000_std value: 4.8341 - type: nauc_ndcg_at_1000_diff1 value: 45.2908 - type: nauc_map_at_1_max value: 59.715700000000005 - type: nauc_map_at_1_std value: 37.2948 - type: nauc_map_at_1_diff1 value: 77.5791 - type: nauc_map_at_3_max value: 37.5253 - type: nauc_map_at_3_std value: 7.3128 - type: nauc_map_at_3_diff1 value: 52.1115 - type: nauc_map_at_5_max value: 31.4148 - type: nauc_map_at_5_std value: 2.8417000000000003 - type: nauc_map_at_5_diff1 value: 55.038399999999996 - type: nauc_map_at_10_max value: 27.1445 - type: nauc_map_at_10_std value: 3.4858000000000002 - type: nauc_map_at_10_diff1 value: 50.8304 - type: nauc_map_at_20_max value: 25.901000000000003 - type: nauc_map_at_20_std value: 6.0125 - type: nauc_map_at_20_diff1 value: 48.234500000000004 - type: nauc_map_at_100_max value: 26.7912 - type: nauc_map_at_100_std value: 5.5458 - type: nauc_map_at_100_diff1 value: 49.1744 - type: nauc_map_at_1000_max value: 26.7912 - type: nauc_map_at_1000_std value: 5.5458 - type: nauc_map_at_1000_diff1 value: 49.1744 - type: nauc_recall_at_1_max value: 59.715700000000005 - type: nauc_recall_at_1_std value: 37.2948 - type: nauc_recall_at_1_diff1 value: 77.5791 - type: nauc_recall_at_3_max value: 32.4424 - type: nauc_recall_at_3_std value: -1.7395999999999998 - type: nauc_recall_at_3_diff1 value: 42.067 - type: nauc_recall_at_5_max value: 18.4679 - type: nauc_recall_at_5_std value: -10.892 - type: nauc_recall_at_5_diff1 value: 52.4896 - type: nauc_recall_at_10_max value: 10.9455 - type: nauc_recall_at_10_std value: -4.9153 - type: nauc_recall_at_10_diff1 value: 39.267 - type: nauc_recall_at_20_max value: 8.365599999999999 - type: nauc_recall_at_20_std value: 13.2074 - type: nauc_recall_at_20_diff1 value: 24.5364 - type: nauc_recall_at_100_max value: .nan - type: nauc_recall_at_100_std value: .nan - type: nauc_recall_at_100_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 59.715700000000005 - type: nauc_precision_at_1_std value: 37.2948 - type: nauc_precision_at_1_diff1 value: 77.5791 - type: nauc_precision_at_3_max value: 32.4424 - type: nauc_precision_at_3_std value: -1.7395999999999998 - type: nauc_precision_at_3_diff1 value: 42.067 - type: nauc_precision_at_5_max value: 18.4679 - type: nauc_precision_at_5_std value: -10.892 - type: nauc_precision_at_5_diff1 value: 52.4896 - type: nauc_precision_at_10_max value: 10.9455 - type: nauc_precision_at_10_std value: -4.9153 - type: nauc_precision_at_10_diff1 value: 39.267 - type: nauc_precision_at_20_max value: 8.365599999999999 - type: nauc_precision_at_20_std value: 13.2074 - type: nauc_precision_at_20_diff1 value: 24.5364 - type: nauc_precision_at_100_max value: 100.0 - type: nauc_precision_at_100_std value: 100.0 - type: nauc_precision_at_100_diff1 value: 100.0 - type: nauc_precision_at_1000_max value: 100.0 - type: nauc_precision_at_1000_std value: 100.0 - type: nauc_precision_at_1000_diff1 value: 100.0 - type: nauc_mrr_at_1_max value: 59.715700000000005 - type: nauc_mrr_at_1_std value: 37.2948 - type: nauc_mrr_at_1_diff1 value: 77.5791 - type: nauc_mrr_at_3_max value: 37.5253 - type: nauc_mrr_at_3_std value: 7.3128 - type: nauc_mrr_at_3_diff1 value: 52.1115 - type: nauc_mrr_at_5_max value: 31.4148 - type: nauc_mrr_at_5_std value: 2.8417000000000003 - type: nauc_mrr_at_5_diff1 value: 55.038399999999996 - type: nauc_mrr_at_10_max value: 27.1445 - type: nauc_mrr_at_10_std value: 3.4858000000000002 - type: nauc_mrr_at_10_diff1 value: 50.8304 - type: nauc_mrr_at_20_max value: 25.901000000000003 - type: nauc_mrr_at_20_std value: 6.0125 - type: nauc_mrr_at_20_diff1 value: 48.234500000000004 - type: nauc_mrr_at_100_max value: 26.7912 - type: nauc_mrr_at_100_std value: 5.5458 - type: nauc_mrr_at_100_diff1 value: 49.1744 - type: nauc_mrr_at_1000_max value: 26.7912 - type: nauc_mrr_at_1000_std value: 5.5458 - type: nauc_mrr_at_1000_diff1 value: 49.1744 - type: main_score value: 12.396 task: type: Retrieval - dataset: config: chinese name: MTEB PublicHealthQA (chinese) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 17.791 - type: ndcg_at_3 value: 23.809 - type: ndcg_at_5 value: 26.344 - type: ndcg_at_10 value: 29.694 - type: ndcg_at_20 value: 32.551 - type: ndcg_at_100 value: 38.822 - type: ndcg_at_1000 value: 40.119 - type: map_at_1 value: 17.791 - type: map_at_3 value: 22.29 - type: map_at_5 value: 23.701 - type: map_at_10 value: 25.070999999999998 - type: map_at_20 value: 25.889 - type: map_at_100 value: 26.680999999999997 - type: map_at_1000 value: 26.749000000000002 - type: recall_at_1 value: 17.791 - type: recall_at_3 value: 28.221 - type: recall_at_5 value: 34.355999999999995 - type: recall_at_10 value: 44.785000000000004 - type: recall_at_20 value: 55.828 - type: recall_at_100 value: 90.798 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 17.791 - type: precision_at_3 value: 9.407 - type: precision_at_5 value: 6.8709999999999996 - type: precision_at_10 value: 4.479 - type: precision_at_20 value: 2.791 - type: precision_at_100 value: 0.9079999999999999 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 17.7914 - type: mrr_at_3 value: 22.290399999999998 - type: mrr_at_5 value: 23.7014 - type: mrr_at_10 value: 25.0711 - type: mrr_at_20 value: 25.8893 - type: mrr_at_100 value: 26.6812 - type: mrr_at_1000 value: 26.7494 - type: nauc_ndcg_at_1_max value: 29.8804 - type: nauc_ndcg_at_1_std value: 6.9849 - type: nauc_ndcg_at_1_diff1 value: 42.0644 - type: nauc_ndcg_at_3_max value: 26.8668 - type: nauc_ndcg_at_3_std value: 5.0143 - type: nauc_ndcg_at_3_diff1 value: 36.0569 - type: nauc_ndcg_at_5_max value: 27.4273 - type: nauc_ndcg_at_5_std value: 8.097999999999999 - type: nauc_ndcg_at_5_diff1 value: 31.8002 - type: nauc_ndcg_at_10_max value: 26.9388 - type: nauc_ndcg_at_10_std value: 8.001999999999999 - type: nauc_ndcg_at_10_diff1 value: 28.4373 - type: nauc_ndcg_at_20_max value: 27.6432 - type: nauc_ndcg_at_20_std value: 10.1775 - type: nauc_ndcg_at_20_diff1 value: 28.0765 - type: nauc_ndcg_at_100_max value: 27.2202 - type: nauc_ndcg_at_100_std value: 8.144400000000001 - type: nauc_ndcg_at_100_diff1 value: 31.2131 - type: nauc_ndcg_at_1000_max value: 27.6147 - type: nauc_ndcg_at_1000_std value: 8.203100000000001 - type: nauc_ndcg_at_1000_diff1 value: 31.8949 - type: nauc_map_at_1_max value: 29.8804 - type: nauc_map_at_1_std value: 6.9849 - type: nauc_map_at_1_diff1 value: 42.0644 - type: nauc_map_at_3_max value: 27.971 - type: nauc_map_at_3_std value: 5.8834 - type: nauc_map_at_3_diff1 value: 37.4543 - type: nauc_map_at_5_max value: 28.3068 - type: nauc_map_at_5_std value: 7.7293 - type: nauc_map_at_5_diff1 value: 34.9238 - type: nauc_map_at_10_max value: 28.1389 - type: nauc_map_at_10_std value: 7.6501 - type: nauc_map_at_10_diff1 value: 33.5384 - type: nauc_map_at_20_max value: 28.363 - type: nauc_map_at_20_std value: 8.2814 - type: nauc_map_at_20_diff1 value: 33.5464 - type: nauc_map_at_100_max value: 28.2036 - type: nauc_map_at_100_std value: 7.8896 - type: nauc_map_at_100_diff1 value: 34.0184 - type: nauc_map_at_1000_max value: 28.2323 - type: nauc_map_at_1000_std value: 7.8995 - type: nauc_map_at_1000_diff1 value: 34.050599999999996 - type: nauc_recall_at_1_max value: 29.8804 - type: nauc_recall_at_1_std value: 6.9849 - type: nauc_recall_at_1_diff1 value: 42.0644 - type: nauc_recall_at_3_max value: 23.8995 - type: nauc_recall_at_3_std value: 2.6474 - type: nauc_recall_at_3_diff1 value: 32.4726 - type: nauc_recall_at_5_max value: 25.1623 - type: nauc_recall_at_5_std value: 9.1546 - type: nauc_recall_at_5_diff1 value: 23.6453 - type: nauc_recall_at_10_max value: 23.647199999999998 - type: nauc_recall_at_10_std value: 9.0445 - type: nauc_recall_at_10_diff1 value: 14.225299999999999 - type: nauc_recall_at_20_max value: 25.882699999999996 - type: nauc_recall_at_20_std value: 17.0687 - type: nauc_recall_at_20_diff1 value: 11.1392 - type: nauc_recall_at_100_max value: 20.4424 - type: nauc_recall_at_100_std value: 7.8594 - type: nauc_recall_at_100_diff1 value: 17.3056 - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 29.8804 - type: nauc_precision_at_1_std value: 6.9849 - type: nauc_precision_at_1_diff1 value: 42.0644 - type: nauc_precision_at_3_max value: 23.8995 - type: nauc_precision_at_3_std value: 2.6474 - type: nauc_precision_at_3_diff1 value: 32.4726 - type: nauc_precision_at_5_max value: 25.1623 - type: nauc_precision_at_5_std value: 9.1546 - type: nauc_precision_at_5_diff1 value: 23.6453 - type: nauc_precision_at_10_max value: 23.647199999999998 - type: nauc_precision_at_10_std value: 9.0445 - type: nauc_precision_at_10_diff1 value: 14.225299999999999 - type: nauc_precision_at_20_max value: 25.882699999999996 - type: nauc_precision_at_20_std value: 17.0687 - type: nauc_precision_at_20_diff1 value: 11.1392 - type: nauc_precision_at_100_max value: 20.4424 - type: nauc_precision_at_100_std value: 7.8594 - type: nauc_precision_at_100_diff1 value: 17.3056 - type: nauc_precision_at_1000_max value: 100.0 - type: nauc_precision_at_1000_std value: 100.0 - type: nauc_precision_at_1000_diff1 value: 100.0 - type: nauc_mrr_at_1_max value: 29.8804 - type: nauc_mrr_at_1_std value: 6.9849 - type: nauc_mrr_at_1_diff1 value: 42.0644 - type: nauc_mrr_at_3_max value: 27.971 - type: nauc_mrr_at_3_std value: 5.8834 - type: nauc_mrr_at_3_diff1 value: 37.4543 - type: nauc_mrr_at_5_max value: 28.3068 - type: nauc_mrr_at_5_std value: 7.7293 - type: nauc_mrr_at_5_diff1 value: 34.9238 - type: nauc_mrr_at_10_max value: 28.1389 - type: nauc_mrr_at_10_std value: 7.6501 - type: nauc_mrr_at_10_diff1 value: 33.5384 - type: nauc_mrr_at_20_max value: 28.363 - type: nauc_mrr_at_20_std value: 8.2814 - type: nauc_mrr_at_20_diff1 value: 33.5464 - type: nauc_mrr_at_100_max value: 28.2036 - type: nauc_mrr_at_100_std value: 7.8896 - type: nauc_mrr_at_100_diff1 value: 34.0184 - type: nauc_mrr_at_1000_max value: 28.2323 - type: nauc_mrr_at_1000_std value: 7.8995 - type: nauc_mrr_at_1000_diff1 value: 34.050599999999996 - type: main_score value: 29.694 task: type: Retrieval - dataset: config: english name: MTEB PublicHealthQA (english) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 68.605 - type: ndcg_at_3 value: 79.063 - type: ndcg_at_5 value: 80.989 - type: ndcg_at_10 value: 82.368 - type: ndcg_at_20 value: 82.816 - type: ndcg_at_100 value: 83.246 - type: ndcg_at_1000 value: 83.327 - type: map_at_1 value: 68.605 - type: map_at_3 value: 76.453 - type: map_at_5 value: 77.529 - type: map_at_10 value: 78.13600000000001 - type: map_at_20 value: 78.263 - type: map_at_100 value: 78.322 - type: map_at_1000 value: 78.326 - type: recall_at_1 value: 68.605 - type: recall_at_3 value: 86.628 - type: recall_at_5 value: 91.279 - type: recall_at_10 value: 95.34899999999999 - type: recall_at_20 value: 97.09299999999999 - type: recall_at_100 value: 99.419 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 68.605 - type: precision_at_3 value: 28.876 - type: precision_at_5 value: 18.256 - type: precision_at_10 value: 9.535 - type: precision_at_20 value: 4.855 - type: precision_at_100 value: 0.9939999999999999 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 68.6047 - type: mrr_at_3 value: 76.45349999999999 - type: mrr_at_5 value: 77.5291 - type: mrr_at_10 value: 78.1363 - type: mrr_at_20 value: 78.2626 - type: mrr_at_100 value: 78.322 - type: mrr_at_1000 value: 78.3261 - type: nauc_ndcg_at_1_max value: 39.230399999999996 - type: nauc_ndcg_at_1_std value: -14.3013 - type: nauc_ndcg_at_1_diff1 value: 63.6784 - type: nauc_ndcg_at_3_max value: 42.6008 - type: nauc_ndcg_at_3_std value: -14.300699999999999 - type: nauc_ndcg_at_3_diff1 value: 59.948 - type: nauc_ndcg_at_5_max value: 44.91 - type: nauc_ndcg_at_5_std value: -8.611099999999999 - type: nauc_ndcg_at_5_diff1 value: 60.5873 - type: nauc_ndcg_at_10_max value: 43.5913 - type: nauc_ndcg_at_10_std value: -11.4792 - type: nauc_ndcg_at_10_diff1 value: 60.8896 - type: nauc_ndcg_at_20_max value: 44.405699999999996 - type: nauc_ndcg_at_20_std value: -9.0507 - type: nauc_ndcg_at_20_diff1 value: 60.295100000000005 - type: nauc_ndcg_at_100_max value: 43.6858 - type: nauc_ndcg_at_100_std value: -10.0097 - type: nauc_ndcg_at_100_diff1 value: 61.163900000000005 - type: nauc_ndcg_at_1000_max value: 43.375 - type: nauc_ndcg_at_1000_std value: -10.617 - type: nauc_ndcg_at_1000_diff1 value: 61.021499999999996 - type: nauc_map_at_1_max value: 39.230399999999996 - type: nauc_map_at_1_std value: -14.3013 - type: nauc_map_at_1_diff1 value: 63.6784 - type: nauc_map_at_3_max value: 41.875 - type: nauc_map_at_3_std value: -13.911299999999999 - type: nauc_map_at_3_diff1 value: 61.1028 - type: nauc_map_at_5_max value: 43.0246 - type: nauc_map_at_5_std value: -10.9924 - type: nauc_map_at_5_diff1 value: 61.381600000000006 - type: nauc_map_at_10_max value: 42.5303 - type: nauc_map_at_10_std value: -12.0992 - type: nauc_map_at_10_diff1 value: 61.5417 - type: nauc_map_at_20_max value: 42.7267 - type: nauc_map_at_20_std value: -11.5421 - type: nauc_map_at_20_diff1 value: 61.4179 - type: nauc_map_at_100_max value: 42.653200000000005 - type: nauc_map_at_100_std value: -11.5826 - type: nauc_map_at_100_diff1 value: 61.47709999999999 - type: nauc_map_at_1000_max value: 42.640899999999995 - type: nauc_map_at_1000_std value: -11.6066 - type: nauc_map_at_1000_diff1 value: 61.4717 - type: nauc_recall_at_1_max value: 39.230399999999996 - type: nauc_recall_at_1_std value: -14.3013 - type: nauc_recall_at_1_diff1 value: 63.6784 - type: nauc_recall_at_3_max value: 45.7626 - type: nauc_recall_at_3_std value: -16.3689 - type: nauc_recall_at_3_diff1 value: 54.581599999999995 - type: nauc_recall_at_5_max value: 57.456700000000005 - type: nauc_recall_at_5_std value: 7.8857 - type: nauc_recall_at_5_diff1 value: 55.8095 - type: nauc_recall_at_10_max value: 54.461999999999996 - type: nauc_recall_at_10_std value: -6.1557 - type: nauc_recall_at_10_diff1 value: 54.03980000000001 - type: nauc_recall_at_20_max value: 76.5834 - type: nauc_recall_at_20_std value: 45.8102 - type: nauc_recall_at_20_diff1 value: 37.551 - type: nauc_recall_at_100_max value: 100.0 - type: nauc_recall_at_100_std value: 100.0 - type: nauc_recall_at_100_diff1 value: 86.945 - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 39.230399999999996 - type: nauc_precision_at_1_std value: -14.3013 - type: nauc_precision_at_1_diff1 value: 63.6784 - type: nauc_precision_at_3_max value: 45.7626 - type: nauc_precision_at_3_std value: -16.3689 - type: nauc_precision_at_3_diff1 value: 54.581599999999995 - type: nauc_precision_at_5_max value: 57.456700000000005 - type: nauc_precision_at_5_std value: 7.8857 - type: nauc_precision_at_5_diff1 value: 55.8095 - type: nauc_precision_at_10_max value: 54.461999999999996 - type: nauc_precision_at_10_std value: -6.1557 - type: nauc_precision_at_10_diff1 value: 54.03980000000001 - type: nauc_precision_at_20_max value: 76.5834 - type: nauc_precision_at_20_std value: 45.8102 - type: nauc_precision_at_20_diff1 value: 37.551 - type: nauc_precision_at_100_max value: 100.0 - type: nauc_precision_at_100_std value: 100.0 - type: nauc_precision_at_100_diff1 value: 86.945 - type: nauc_precision_at_1000_max value: 100.0 - type: nauc_precision_at_1000_std value: 100.0 - type: nauc_precision_at_1000_diff1 value: 100.0 - type: nauc_mrr_at_1_max value: 39.230399999999996 - type: nauc_mrr_at_1_std value: -14.3013 - type: nauc_mrr_at_1_diff1 value: 63.6784 - type: nauc_mrr_at_3_max value: 41.875 - type: nauc_mrr_at_3_std value: -13.911299999999999 - type: nauc_mrr_at_3_diff1 value: 61.1028 - type: nauc_mrr_at_5_max value: 43.0246 - type: nauc_mrr_at_5_std value: -10.9924 - type: nauc_mrr_at_5_diff1 value: 61.381600000000006 - type: nauc_mrr_at_10_max value: 42.5303 - type: nauc_mrr_at_10_std value: -12.0992 - type: nauc_mrr_at_10_diff1 value: 61.5417 - type: nauc_mrr_at_20_max value: 42.7267 - type: nauc_mrr_at_20_std value: -11.5421 - type: nauc_mrr_at_20_diff1 value: 61.4179 - type: nauc_mrr_at_100_max value: 42.653200000000005 - type: nauc_mrr_at_100_std value: -11.5826 - type: nauc_mrr_at_100_diff1 value: 61.47709999999999 - type: nauc_mrr_at_1000_max value: 42.640899999999995 - type: nauc_mrr_at_1000_std value: -11.6064 - type: nauc_mrr_at_1000_diff1 value: 61.4717 - type: main_score value: 82.368 task: type: Retrieval - dataset: config: french name: MTEB PublicHealthQA (french) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 57.647000000000006 - type: ndcg_at_3 value: 68.16499999999999 - type: ndcg_at_5 value: 70.14 - type: ndcg_at_10 value: 74.08500000000001 - type: ndcg_at_20 value: 75.297 - type: ndcg_at_100 value: 75.97699999999999 - type: ndcg_at_1000 value: 75.97699999999999 - type: map_at_1 value: 57.647000000000006 - type: map_at_3 value: 65.686 - type: map_at_5 value: 66.804 - type: map_at_10 value: 68.516 - type: map_at_20 value: 68.86 - type: map_at_100 value: 68.959 - type: map_at_1000 value: 68.959 - type: recall_at_1 value: 57.647000000000006 - type: recall_at_3 value: 75.29400000000001 - type: recall_at_5 value: 80.0 - type: recall_at_10 value: 91.765 - type: recall_at_20 value: 96.47099999999999 - type: recall_at_100 value: 100.0 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 57.647000000000006 - type: precision_at_3 value: 25.098 - type: precision_at_5 value: 16.0 - type: precision_at_10 value: 9.176 - type: precision_at_20 value: 4.824 - type: precision_at_100 value: 1.0 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 57.647099999999995 - type: mrr_at_3 value: 65.6863 - type: mrr_at_5 value: 66.8039 - type: mrr_at_10 value: 68.5159 - type: mrr_at_20 value: 68.8596 - type: mrr_at_100 value: 68.9587 - type: mrr_at_1000 value: 68.9587 - type: nauc_ndcg_at_1_max value: 36.299 - type: nauc_ndcg_at_1_std value: -0.31129999999999997 - type: nauc_ndcg_at_1_diff1 value: 52.98349999999999 - type: nauc_ndcg_at_3_max value: 30.7772 - type: nauc_ndcg_at_3_std value: 15.1285 - type: nauc_ndcg_at_3_diff1 value: 42.4886 - type: nauc_ndcg_at_5_max value: 34.1327 - type: nauc_ndcg_at_5_std value: 17.4173 - type: nauc_ndcg_at_5_diff1 value: 42.7984 - type: nauc_ndcg_at_10_max value: 38.8615 - type: nauc_ndcg_at_10_std value: 17.1227 - type: nauc_ndcg_at_10_diff1 value: 50.4977 - type: nauc_ndcg_at_20_max value: 37.0022 - type: nauc_ndcg_at_20_std value: 13.7289 - type: nauc_ndcg_at_20_diff1 value: 48.4148 - type: nauc_ndcg_at_100_max value: 35.965 - type: nauc_ndcg_at_100_std value: 12.9999 - type: nauc_ndcg_at_100_diff1 value: 47.7781 - type: nauc_ndcg_at_1000_max value: 35.965 - type: nauc_ndcg_at_1000_std value: 12.9999 - type: nauc_ndcg_at_1000_diff1 value: 47.7781 - type: nauc_map_at_1_max value: 36.299 - type: nauc_map_at_1_std value: -0.31129999999999997 - type: nauc_map_at_1_diff1 value: 52.98349999999999 - type: nauc_map_at_3_max value: 32.4585 - type: nauc_map_at_3_std value: 11.4142 - type: nauc_map_at_3_diff1 value: 44.913 - type: nauc_map_at_5_max value: 34.2207 - type: nauc_map_at_5_std value: 12.4555 - type: nauc_map_at_5_diff1 value: 45.1962 - type: nauc_map_at_10_max value: 36.0741 - type: nauc_map_at_10_std value: 12.3794 - type: nauc_map_at_10_diff1 value: 48.3191 - type: nauc_map_at_20_max value: 35.6485 - type: nauc_map_at_20_std value: 11.5573 - type: nauc_map_at_20_diff1 value: 47.8261 - type: nauc_map_at_100_max value: 35.5227 - type: nauc_map_at_100_std value: 11.4605 - type: nauc_map_at_100_diff1 value: 47.7512 - type: nauc_map_at_1000_max value: 35.5227 - type: nauc_map_at_1000_std value: 11.4605 - type: nauc_map_at_1000_diff1 value: 47.7512 - type: nauc_recall_at_1_max value: 36.299 - type: nauc_recall_at_1_std value: -0.31129999999999997 - type: nauc_recall_at_1_diff1 value: 52.98349999999999 - type: nauc_recall_at_3_max value: 24.5979 - type: nauc_recall_at_3_std value: 28.3054 - type: nauc_recall_at_3_diff1 value: 33.9486 - type: nauc_recall_at_5_max value: 34.3725 - type: nauc_recall_at_5_std value: 38.2391 - type: nauc_recall_at_5_diff1 value: 33.080999999999996 - type: nauc_recall_at_10_max value: 69.187 - type: nauc_recall_at_10_std value: 57.02739999999999 - type: nauc_recall_at_10_diff1 value: 77.4776 - type: nauc_recall_at_20_max value: 66.5024 - type: nauc_recall_at_20_std value: 34.1342 - type: nauc_recall_at_20_diff1 value: 66.6574 - type: nauc_recall_at_100_max value: .nan - type: nauc_recall_at_100_std value: .nan - type: nauc_recall_at_100_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 36.299 - type: nauc_precision_at_1_std value: -0.31129999999999997 - type: nauc_precision_at_1_diff1 value: 52.98349999999999 - type: nauc_precision_at_3_max value: 24.5979 - type: nauc_precision_at_3_std value: 28.3054 - type: nauc_precision_at_3_diff1 value: 33.9486 - type: nauc_precision_at_5_max value: 34.3725 - type: nauc_precision_at_5_std value: 38.2391 - type: nauc_precision_at_5_diff1 value: 33.080999999999996 - type: nauc_precision_at_10_max value: 69.187 - type: nauc_precision_at_10_std value: 57.02739999999999 - type: nauc_precision_at_10_diff1 value: 77.4776 - type: nauc_precision_at_20_max value: 66.5024 - type: nauc_precision_at_20_std value: 34.1342 - type: nauc_precision_at_20_diff1 value: 66.6574 - type: nauc_precision_at_100_max value: 100.0 - type: nauc_precision_at_100_std value: 100.0 - type: nauc_precision_at_100_diff1 value: 100.0 - type: nauc_precision_at_1000_max value: 100.0 - type: nauc_precision_at_1000_std value: 100.0 - type: nauc_precision_at_1000_diff1 value: 100.0 - type: nauc_mrr_at_1_max value: 36.299 - type: nauc_mrr_at_1_std value: -0.31129999999999997 - type: nauc_mrr_at_1_diff1 value: 52.98349999999999 - type: nauc_mrr_at_3_max value: 32.4585 - type: nauc_mrr_at_3_std value: 11.4142 - type: nauc_mrr_at_3_diff1 value: 44.913 - type: nauc_mrr_at_5_max value: 34.2207 - type: nauc_mrr_at_5_std value: 12.4555 - type: nauc_mrr_at_5_diff1 value: 45.1962 - type: nauc_mrr_at_10_max value: 36.0741 - type: nauc_mrr_at_10_std value: 12.3794 - type: nauc_mrr_at_10_diff1 value: 48.3191 - type: nauc_mrr_at_20_max value: 35.6485 - type: nauc_mrr_at_20_std value: 11.5573 - type: nauc_mrr_at_20_diff1 value: 47.8261 - type: nauc_mrr_at_100_max value: 35.5227 - type: nauc_mrr_at_100_std value: 11.4605 - type: nauc_mrr_at_100_diff1 value: 47.7512 - type: nauc_mrr_at_1000_max value: 35.5227 - type: nauc_mrr_at_1000_std value: 11.4605 - type: nauc_mrr_at_1000_diff1 value: 47.7512 - type: main_score value: 74.08500000000001 task: type: Retrieval - dataset: config: korean name: MTEB PublicHealthQA (korean) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 15.584000000000001 - type: ndcg_at_3 value: 25.046000000000003 - type: ndcg_at_5 value: 30.857 - type: ndcg_at_10 value: 34.729 - type: ndcg_at_20 value: 38.313 - type: ndcg_at_100 value: 43.254999999999995 - type: ndcg_at_1000 value: 43.254999999999995 - type: map_at_1 value: 15.584000000000001 - type: map_at_3 value: 22.511 - type: map_at_5 value: 25.692999999999998 - type: map_at_10 value: 27.35 - type: map_at_20 value: 28.317999999999998 - type: map_at_100 value: 28.947 - type: map_at_1000 value: 28.947 - type: recall_at_1 value: 15.584000000000001 - type: recall_at_3 value: 32.468 - type: recall_at_5 value: 46.753 - type: recall_at_10 value: 58.44200000000001 - type: recall_at_20 value: 72.727 - type: recall_at_100 value: 100.0 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 15.584000000000001 - type: precision_at_3 value: 10.823 - type: precision_at_5 value: 9.350999999999999 - type: precision_at_10 value: 5.844 - type: precision_at_20 value: 3.636 - type: precision_at_100 value: 1.0 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 15.5844 - type: mrr_at_3 value: 22.5108 - type: mrr_at_5 value: 25.6926 - type: mrr_at_10 value: 27.35 - type: mrr_at_20 value: 28.3185 - type: mrr_at_100 value: 28.9469 - type: mrr_at_1000 value: 28.9469 - type: nauc_ndcg_at_1_max value: 22.6619 - type: nauc_ndcg_at_1_std value: 26.3584 - type: nauc_ndcg_at_1_diff1 value: 30.0107 - type: nauc_ndcg_at_3_max value: 26.0589 - type: nauc_ndcg_at_3_std value: 27.4435 - type: nauc_ndcg_at_3_diff1 value: 20.6848 - type: nauc_ndcg_at_5_max value: 29.0957 - type: nauc_ndcg_at_5_std value: 28.6226 - type: nauc_ndcg_at_5_diff1 value: 12.873999999999999 - type: nauc_ndcg_at_10_max value: 22.5377 - type: nauc_ndcg_at_10_std value: 22.9366 - type: nauc_ndcg_at_10_diff1 value: 6.500100000000001 - type: nauc_ndcg_at_20_max value: 23.9022 - type: nauc_ndcg_at_20_std value: 25.572499999999998 - type: nauc_ndcg_at_20_diff1 value: 10.0398 - type: nauc_ndcg_at_100_max value: 24.9972 - type: nauc_ndcg_at_100_std value: 26.282299999999996 - type: nauc_ndcg_at_100_diff1 value: 14.2371 - type: nauc_ndcg_at_1000_max value: 24.9972 - type: nauc_ndcg_at_1000_std value: 26.282299999999996 - type: nauc_ndcg_at_1000_diff1 value: 14.2371 - type: nauc_map_at_1_max value: 22.6619 - type: nauc_map_at_1_std value: 26.3584 - type: nauc_map_at_1_diff1 value: 30.0107 - type: nauc_map_at_3_max value: 25.584600000000002 - type: nauc_map_at_3_std value: 27.6868 - type: nauc_map_at_3_diff1 value: 22.831599999999998 - type: nauc_map_at_5_max value: 27.417399999999997 - type: nauc_map_at_5_std value: 28.5693 - type: nauc_map_at_5_diff1 value: 18.212999999999997 - type: nauc_map_at_10_max value: 24.4972 - type: nauc_map_at_10_std value: 25.991500000000002 - type: nauc_map_at_10_diff1 value: 15.3749 - type: nauc_map_at_20_max value: 24.8498 - type: nauc_map_at_20_std value: 26.7085 - type: nauc_map_at_20_diff1 value: 16.4134 - type: nauc_map_at_100_max value: 25.1019 - type: nauc_map_at_100_std value: 26.8269 - type: nauc_map_at_100_diff1 value: 17.0376 - type: nauc_map_at_1000_max value: 25.1019 - type: nauc_map_at_1000_std value: 26.8269 - type: nauc_map_at_1000_diff1 value: 17.0376 - type: nauc_recall_at_1_max value: 22.6619 - type: nauc_recall_at_1_std value: 26.3584 - type: nauc_recall_at_1_diff1 value: 30.0107 - type: nauc_recall_at_3_max value: 27.1171 - type: nauc_recall_at_3_std value: 26.706400000000002 - type: nauc_recall_at_3_diff1 value: 15.479499999999998 - type: nauc_recall_at_5_max value: 33.3129 - type: nauc_recall_at_5_std value: 28.5407 - type: nauc_recall_at_5_diff1 value: -0.7628 - type: nauc_recall_at_10_max value: 15.459100000000001 - type: nauc_recall_at_10_std value: 13.001199999999999 - type: nauc_recall_at_10_diff1 value: -19.4151 - type: nauc_recall_at_20_max value: 19.689400000000003 - type: nauc_recall_at_20_std value: 22.014300000000002 - type: nauc_recall_at_20_diff1 value: -11.3263 - type: nauc_recall_at_100_max value: .nan - type: nauc_recall_at_100_std value: .nan - type: nauc_recall_at_100_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 22.6619 - type: nauc_precision_at_1_std value: 26.3584 - type: nauc_precision_at_1_diff1 value: 30.0107 - type: nauc_precision_at_3_max value: 27.1171 - type: nauc_precision_at_3_std value: 26.706400000000002 - type: nauc_precision_at_3_diff1 value: 15.479499999999998 - type: nauc_precision_at_5_max value: 33.3129 - type: nauc_precision_at_5_std value: 28.5407 - type: nauc_precision_at_5_diff1 value: -0.7628 - type: nauc_precision_at_10_max value: 15.459100000000001 - type: nauc_precision_at_10_std value: 13.001199999999999 - type: nauc_precision_at_10_diff1 value: -19.4151 - type: nauc_precision_at_20_max value: 19.689400000000003 - type: nauc_precision_at_20_std value: 22.014300000000002 - type: nauc_precision_at_20_diff1 value: -11.3263 - type: nauc_precision_at_100_max value: 100.0 - type: nauc_precision_at_100_std value: 100.0 - type: nauc_precision_at_100_diff1 value: 100.0 - type: nauc_precision_at_1000_max value: .nan - type: nauc_precision_at_1000_std value: .nan - type: nauc_precision_at_1000_diff1 value: .nan - type: nauc_mrr_at_1_max value: 22.6619 - type: nauc_mrr_at_1_std value: 26.3584 - type: nauc_mrr_at_1_diff1 value: 30.0107 - type: nauc_mrr_at_3_max value: 25.584600000000002 - type: nauc_mrr_at_3_std value: 27.6868 - type: nauc_mrr_at_3_diff1 value: 22.831599999999998 - type: nauc_mrr_at_5_max value: 27.417399999999997 - type: nauc_mrr_at_5_std value: 28.5693 - type: nauc_mrr_at_5_diff1 value: 18.212999999999997 - type: nauc_mrr_at_10_max value: 24.4972 - type: nauc_mrr_at_10_std value: 25.991500000000002 - type: nauc_mrr_at_10_diff1 value: 15.3749 - type: nauc_mrr_at_20_max value: 24.8498 - type: nauc_mrr_at_20_std value: 26.7085 - type: nauc_mrr_at_20_diff1 value: 16.4134 - type: nauc_mrr_at_100_max value: 25.1019 - type: nauc_mrr_at_100_std value: 26.8269 - type: nauc_mrr_at_100_diff1 value: 17.0376 - type: nauc_mrr_at_1000_max value: 25.1019 - type: nauc_mrr_at_1000_std value: 26.8269 - type: nauc_mrr_at_1000_diff1 value: 17.0376 - type: main_score value: 34.729 task: type: Retrieval - dataset: config: russian name: MTEB PublicHealthQA (russian) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 38.462 - type: ndcg_at_3 value: 39.432 - type: ndcg_at_5 value: 43.801 - type: ndcg_at_10 value: 47.365 - type: ndcg_at_20 value: 50.529 - type: ndcg_at_100 value: 55.525999999999996 - type: ndcg_at_1000 value: 55.525999999999996 - type: map_at_1 value: 38.462 - type: map_at_3 value: 39.231 - type: map_at_5 value: 41.615 - type: map_at_10 value: 43.134 - type: map_at_20 value: 44.027 - type: map_at_100 value: 44.762 - type: map_at_1000 value: 44.762 - type: recall_at_1 value: 38.462 - type: recall_at_3 value: 40.0 - type: recall_at_5 value: 50.769 - type: recall_at_10 value: 61.538000000000004 - type: recall_at_20 value: 73.846 - type: recall_at_100 value: 100.0 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 38.462 - type: precision_at_3 value: 13.333 - type: precision_at_5 value: 10.154 - type: precision_at_10 value: 6.154 - type: precision_at_20 value: 3.692 - type: precision_at_100 value: 1.0 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 38.4615 - type: mrr_at_3 value: 39.2308 - type: mrr_at_5 value: 41.6154 - type: mrr_at_10 value: 43.133700000000005 - type: mrr_at_20 value: 44.0268 - type: mrr_at_100 value: 44.7615 - type: mrr_at_1000 value: 44.7615 - type: nauc_ndcg_at_1_max value: 35.7196 - type: nauc_ndcg_at_1_std value: 31.128899999999998 - type: nauc_ndcg_at_1_diff1 value: 48.9534 - type: nauc_ndcg_at_3_max value: 33.2144 - type: nauc_ndcg_at_3_std value: 28.644599999999997 - type: nauc_ndcg_at_3_diff1 value: 53.3224 - type: nauc_ndcg_at_5_max value: 35.9717 - type: nauc_ndcg_at_5_std value: 34.1499 - type: nauc_ndcg_at_5_diff1 value: 53.568000000000005 - type: nauc_ndcg_at_10_max value: 35.6701 - type: nauc_ndcg_at_10_std value: 30.9807 - type: nauc_ndcg_at_10_diff1 value: 50.645799999999994 - type: nauc_ndcg_at_20_max value: 37.1344 - type: nauc_ndcg_at_20_std value: 31.297399999999996 - type: nauc_ndcg_at_20_diff1 value: 47.5693 - type: nauc_ndcg_at_100_max value: 35.5966 - type: nauc_ndcg_at_100_std value: 31.0317 - type: nauc_ndcg_at_100_diff1 value: 50.1695 - type: nauc_ndcg_at_1000_max value: 35.5966 - type: nauc_ndcg_at_1000_std value: 31.0317 - type: nauc_ndcg_at_1000_diff1 value: 50.1695 - type: nauc_map_at_1_max value: 35.7196 - type: nauc_map_at_1_std value: 31.128899999999998 - type: nauc_map_at_1_diff1 value: 48.9534 - type: nauc_map_at_3_max value: 33.7324 - type: nauc_map_at_3_std value: 29.158299999999997 - type: nauc_map_at_3_diff1 value: 52.419000000000004 - type: nauc_map_at_5_max value: 35.1746 - type: nauc_map_at_5_std value: 31.917 - type: nauc_map_at_5_diff1 value: 52.615100000000005 - type: nauc_map_at_10_max value: 35.009299999999996 - type: nauc_map_at_10_std value: 30.4855 - type: nauc_map_at_10_diff1 value: 51.4068 - type: nauc_map_at_20_max value: 35.3619 - type: nauc_map_at_20_std value: 30.5305 - type: nauc_map_at_20_diff1 value: 50.6054 - type: nauc_map_at_100_max value: 35.1648 - type: nauc_map_at_100_std value: 30.605300000000003 - type: nauc_map_at_100_diff1 value: 50.85809999999999 - type: nauc_map_at_1000_max value: 35.1648 - type: nauc_map_at_1000_std value: 30.605300000000003 - type: nauc_map_at_1000_diff1 value: 50.85809999999999 - type: nauc_recall_at_1_max value: 35.7196 - type: nauc_recall_at_1_std value: 31.128899999999998 - type: nauc_recall_at_1_diff1 value: 48.9534 - type: nauc_recall_at_3_max value: 31.759500000000003 - type: nauc_recall_at_3_std value: 27.2018 - type: nauc_recall_at_3_diff1 value: 55.859700000000004 - type: nauc_recall_at_5_max value: 38.8865 - type: nauc_recall_at_5_std value: 41.984 - type: nauc_recall_at_5_diff1 value: 56.2765 - type: nauc_recall_at_10_max value: 38.400600000000004 - type: nauc_recall_at_10_std value: 32.869 - type: nauc_recall_at_10_diff1 value: 46.7236 - type: nauc_recall_at_20_max value: 47.502 - type: nauc_recall_at_20_std value: 35.5301 - type: nauc_recall_at_20_diff1 value: 28.7689 - type: nauc_recall_at_100_max value: .nan - type: nauc_recall_at_100_std value: .nan - type: nauc_recall_at_100_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 35.7196 - type: nauc_precision_at_1_std value: 31.128899999999998 - type: nauc_precision_at_1_diff1 value: 48.9534 - type: nauc_precision_at_3_max value: 31.759500000000003 - type: nauc_precision_at_3_std value: 27.2018 - type: nauc_precision_at_3_diff1 value: 55.859700000000004 - type: nauc_precision_at_5_max value: 38.8865 - type: nauc_precision_at_5_std value: 41.984 - type: nauc_precision_at_5_diff1 value: 56.2765 - type: nauc_precision_at_10_max value: 38.400600000000004 - type: nauc_precision_at_10_std value: 32.869 - type: nauc_precision_at_10_diff1 value: 46.7236 - type: nauc_precision_at_20_max value: 47.502 - type: nauc_precision_at_20_std value: 35.5301 - type: nauc_precision_at_20_diff1 value: 28.7689 - type: nauc_precision_at_100_max value: .nan - type: nauc_precision_at_100_std value: .nan - type: nauc_precision_at_100_diff1 value: .nan - type: nauc_precision_at_1000_max value: 100.0 - type: nauc_precision_at_1000_std value: 100.0 - type: nauc_precision_at_1000_diff1 value: 100.0 - type: nauc_mrr_at_1_max value: 35.7196 - type: nauc_mrr_at_1_std value: 31.128899999999998 - type: nauc_mrr_at_1_diff1 value: 48.9534 - type: nauc_mrr_at_3_max value: 33.7324 - type: nauc_mrr_at_3_std value: 29.158299999999997 - type: nauc_mrr_at_3_diff1 value: 52.419000000000004 - type: nauc_mrr_at_5_max value: 35.1746 - type: nauc_mrr_at_5_std value: 31.917 - type: nauc_mrr_at_5_diff1 value: 52.615100000000005 - type: nauc_mrr_at_10_max value: 35.009299999999996 - type: nauc_mrr_at_10_std value: 30.4855 - type: nauc_mrr_at_10_diff1 value: 51.4068 - type: nauc_mrr_at_20_max value: 35.3619 - type: nauc_mrr_at_20_std value: 30.5305 - type: nauc_mrr_at_20_diff1 value: 50.6054 - type: nauc_mrr_at_100_max value: 35.1648 - type: nauc_mrr_at_100_std value: 30.605300000000003 - type: nauc_mrr_at_100_diff1 value: 50.85809999999999 - type: nauc_mrr_at_1000_max value: 35.1648 - type: nauc_mrr_at_1000_std value: 30.605300000000003 - type: nauc_mrr_at_1000_diff1 value: 50.85809999999999 - type: main_score value: 47.365 task: type: Retrieval - dataset: config: spanish name: MTEB PublicHealthQA (spanish) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 39.506 - type: ndcg_at_3 value: 54.247 - type: ndcg_at_5 value: 57.541 - type: ndcg_at_10 value: 60.328 - type: ndcg_at_20 value: 62.331 - type: ndcg_at_100 value: 63.794 - type: ndcg_at_1000 value: 64.05999999999999 - type: map_at_1 value: 39.506 - type: map_at_3 value: 50.412 - type: map_at_5 value: 52.233 - type: map_at_10 value: 53.38 - type: map_at_20 value: 53.917 - type: map_at_100 value: 54.107000000000006 - type: map_at_1000 value: 54.122 - type: recall_at_1 value: 39.506 - type: recall_at_3 value: 65.432 - type: recall_at_5 value: 73.457 - type: recall_at_10 value: 82.099 - type: recall_at_20 value: 90.12299999999999 - type: recall_at_100 value: 98.148 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 39.506 - type: precision_at_3 value: 21.811 - type: precision_at_5 value: 14.691 - type: precision_at_10 value: 8.21 - type: precision_at_20 value: 4.506 - type: precision_at_100 value: 0.9809999999999999 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 39.5062 - type: mrr_at_3 value: 50.4115 - type: mrr_at_5 value: 52.2325 - type: mrr_at_10 value: 53.380399999999995 - type: mrr_at_20 value: 53.9174 - type: mrr_at_100 value: 54.1072 - type: mrr_at_1000 value: 54.122400000000006 - type: nauc_ndcg_at_1_max value: 33.465 - type: nauc_ndcg_at_1_std value: -4.077 - type: nauc_ndcg_at_1_diff1 value: 63.7579 - type: nauc_ndcg_at_3_max value: 34.5067 - type: nauc_ndcg_at_3_std value: -7.5327 - type: nauc_ndcg_at_3_diff1 value: 54.14020000000001 - type: nauc_ndcg_at_5_max value: 35.6912 - type: nauc_ndcg_at_5_std value: -0.911 - type: nauc_ndcg_at_5_diff1 value: 53.2654 - type: nauc_ndcg_at_10_max value: 35.2651 - type: nauc_ndcg_at_10_std value: -0.0039 - type: nauc_ndcg_at_10_diff1 value: 55.3022 - type: nauc_ndcg_at_20_max value: 36.071799999999996 - type: nauc_ndcg_at_20_std value: 1.6840000000000002 - type: nauc_ndcg_at_20_diff1 value: 55.6957 - type: nauc_ndcg_at_100_max value: 35.964400000000005 - type: nauc_ndcg_at_100_std value: -1.6336 - type: nauc_ndcg_at_100_diff1 value: 56.626 - type: nauc_ndcg_at_1000_max value: 35.551899999999996 - type: nauc_ndcg_at_1000_std value: -1.4092 - type: nauc_ndcg_at_1000_diff1 value: 56.64979999999999 - type: nauc_map_at_1_max value: 33.465 - type: nauc_map_at_1_std value: -4.077 - type: nauc_map_at_1_diff1 value: 63.7579 - type: nauc_map_at_3_max value: 34.3502 - type: nauc_map_at_3_std value: -5.9047 - type: nauc_map_at_3_diff1 value: 56.5011 - type: nauc_map_at_5_max value: 35.0286 - type: nauc_map_at_5_std value: -2.464 - type: nauc_map_at_5_diff1 value: 56.152 - type: nauc_map_at_10_max value: 34.8474 - type: nauc_map_at_10_std value: -2.2748999999999997 - type: nauc_map_at_10_diff1 value: 57.052499999999995 - type: nauc_map_at_20_max value: 34.9946 - type: nauc_map_at_20_std value: -1.9411999999999998 - type: nauc_map_at_20_diff1 value: 57.17 - type: nauc_map_at_100_max value: 35.051500000000004 - type: nauc_map_at_100_std value: -2.2917 - type: nauc_map_at_100_diff1 value: 57.3365 - type: nauc_map_at_1000_max value: 35.0333 - type: nauc_map_at_1000_std value: -2.2848 - type: nauc_map_at_1000_diff1 value: 57.338 - type: nauc_recall_at_1_max value: 33.465 - type: nauc_recall_at_1_std value: -4.077 - type: nauc_recall_at_1_diff1 value: 63.7579 - type: nauc_recall_at_3_max value: 34.977799999999995 - type: nauc_recall_at_3_std value: -13.3675 - type: nauc_recall_at_3_diff1 value: 46.262100000000004 - type: nauc_recall_at_5_max value: 38.3639 - type: nauc_recall_at_5_std value: 6.2278 - type: nauc_recall_at_5_diff1 value: 41.7151 - type: nauc_recall_at_10_max value: 37.2498 - type: nauc_recall_at_10_std value: 14.6038 - type: nauc_recall_at_10_diff1 value: 46.9591 - type: nauc_recall_at_20_max value: 47.9805 - type: nauc_recall_at_20_std value: 43.1323 - type: nauc_recall_at_20_diff1 value: 45.1942 - type: nauc_recall_at_100_max value: 74.239 - type: nauc_recall_at_100_std value: -23.8365 - type: nauc_recall_at_100_diff1 value: 54.4918 - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 33.465 - type: nauc_precision_at_1_std value: -4.077 - type: nauc_precision_at_1_diff1 value: 63.7579 - type: nauc_precision_at_3_max value: 34.977799999999995 - type: nauc_precision_at_3_std value: -13.3675 - type: nauc_precision_at_3_diff1 value: 46.262100000000004 - type: nauc_precision_at_5_max value: 38.3639 - type: nauc_precision_at_5_std value: 6.2278 - type: nauc_precision_at_5_diff1 value: 41.7151 - type: nauc_precision_at_10_max value: 37.2498 - type: nauc_precision_at_10_std value: 14.6038 - type: nauc_precision_at_10_diff1 value: 46.9591 - type: nauc_precision_at_20_max value: 47.9805 - type: nauc_precision_at_20_std value: 43.1323 - type: nauc_precision_at_20_diff1 value: 45.1942 - type: nauc_precision_at_100_max value: 74.239 - type: nauc_precision_at_100_std value: -23.8365 - type: nauc_precision_at_100_diff1 value: 54.4918 - type: nauc_precision_at_1000_max value: 100.0 - type: nauc_precision_at_1000_std value: 100.0 - type: nauc_precision_at_1000_diff1 value: 100.0 - type: nauc_mrr_at_1_max value: 33.465 - type: nauc_mrr_at_1_std value: -4.077 - type: nauc_mrr_at_1_diff1 value: 63.7579 - type: nauc_mrr_at_3_max value: 34.3502 - type: nauc_mrr_at_3_std value: -5.9047 - type: nauc_mrr_at_3_diff1 value: 56.5011 - type: nauc_mrr_at_5_max value: 35.0286 - type: nauc_mrr_at_5_std value: -2.464 - type: nauc_mrr_at_5_diff1 value: 56.152 - type: nauc_mrr_at_10_max value: 34.8474 - type: nauc_mrr_at_10_std value: -2.2748999999999997 - type: nauc_mrr_at_10_diff1 value: 57.052499999999995 - type: nauc_mrr_at_20_max value: 34.9946 - type: nauc_mrr_at_20_std value: -1.9411999999999998 - type: nauc_mrr_at_20_diff1 value: 57.17 - type: nauc_mrr_at_100_max value: 35.051500000000004 - type: nauc_mrr_at_100_std value: -2.2917 - type: nauc_mrr_at_100_diff1 value: 57.3365 - type: nauc_mrr_at_1000_max value: 35.0333 - type: nauc_mrr_at_1000_std value: -2.2848 - type: nauc_mrr_at_1000_diff1 value: 57.338 - type: main_score value: 60.328 task: type: Retrieval - dataset: config: vietnamese name: MTEB PublicHealthQA (vietnamese) revision: main split: test type: xhluca/publichealth-qa metrics: - type: ndcg_at_1 value: 31.169000000000004 - type: ndcg_at_3 value: 41.79 - type: ndcg_at_5 value: 43.354 - type: ndcg_at_10 value: 45.957 - type: ndcg_at_20 value: 48.631 - type: ndcg_at_100 value: 53.940999999999995 - type: ndcg_at_1000 value: 53.940999999999995 - type: map_at_1 value: 31.169000000000004 - type: map_at_3 value: 39.177 - type: map_at_5 value: 40.022000000000006 - type: map_at_10 value: 41.144999999999996 - type: map_at_20 value: 41.905 - type: map_at_100 value: 42.641 - type: map_at_1000 value: 42.641 - type: recall_at_1 value: 31.169000000000004 - type: recall_at_3 value: 49.351 - type: recall_at_5 value: 53.247 - type: recall_at_10 value: 61.039 - type: recall_at_20 value: 71.429 - type: recall_at_100 value: 100.0 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 31.169000000000004 - type: precision_at_3 value: 16.45 - type: precision_at_5 value: 10.649000000000001 - type: precision_at_10 value: 6.104 - type: precision_at_20 value: 3.5709999999999997 - type: precision_at_100 value: 1.0 - type: precision_at_1000 value: 0.1 - type: mrr_at_1 value: 31.1688 - type: mrr_at_3 value: 39.177499999999995 - type: mrr_at_5 value: 40.0216 - type: mrr_at_10 value: 41.1451 - type: mrr_at_20 value: 41.9054 - type: mrr_at_100 value: 42.6408 - type: mrr_at_1000 value: 42.6408 - type: nauc_ndcg_at_1_max value: 24.196 - type: nauc_ndcg_at_1_std value: -4.1928 - type: nauc_ndcg_at_1_diff1 value: 52.1914 - type: nauc_ndcg_at_3_max value: 20.099800000000002 - type: nauc_ndcg_at_3_std value: -0.4444 - type: nauc_ndcg_at_3_diff1 value: 46.0998 - type: nauc_ndcg_at_5_max value: 23.3937 - type: nauc_ndcg_at_5_std value: -1.4628999999999999 - type: nauc_ndcg_at_5_diff1 value: 48.0126 - type: nauc_ndcg_at_10_max value: 20.7063 - type: nauc_ndcg_at_10_std value: -4.7612000000000005 - type: nauc_ndcg_at_10_diff1 value: 46.1062 - type: nauc_ndcg_at_20_max value: 21.5561 - type: nauc_ndcg_at_20_std value: -0.5067 - type: nauc_ndcg_at_20_diff1 value: 47.6175 - type: nauc_ndcg_at_100_max value: 21.5381 - type: nauc_ndcg_at_100_std value: -0.7255 - type: nauc_ndcg_at_100_diff1 value: 46.822900000000004 - type: nauc_ndcg_at_1000_max value: 21.5381 - type: nauc_ndcg_at_1000_std value: -0.7255 - type: nauc_ndcg_at_1000_diff1 value: 46.822900000000004 - type: nauc_map_at_1_max value: 24.196 - type: nauc_map_at_1_std value: -4.1928 - type: nauc_map_at_1_diff1 value: 52.1914 - type: nauc_map_at_3_max value: 21.205199999999998 - type: nauc_map_at_3_std value: -1.0412 - type: nauc_map_at_3_diff1 value: 46.8352 - type: nauc_map_at_5_max value: 23.086000000000002 - type: nauc_map_at_5_std value: -1.5817999999999999 - type: nauc_map_at_5_diff1 value: 47.8869 - type: nauc_map_at_10_max value: 21.9837 - type: nauc_map_at_10_std value: -2.8929 - type: nauc_map_at_10_diff1 value: 47.045500000000004 - type: nauc_map_at_20_max value: 22.1671 - type: nauc_map_at_20_std value: -1.7763999999999998 - type: nauc_map_at_20_diff1 value: 47.388999999999996 - type: nauc_map_at_100_max value: 22.0523 - type: nauc_map_at_100_std value: -1.4708 - type: nauc_map_at_100_diff1 value: 47.2193 - type: nauc_map_at_1000_max value: 22.0523 - type: nauc_map_at_1000_std value: -1.4708 - type: nauc_map_at_1000_diff1 value: 47.2193 - type: nauc_recall_at_1_max value: 24.196 - type: nauc_recall_at_1_std value: -4.1928 - type: nauc_recall_at_1_diff1 value: 52.1914 - type: nauc_recall_at_3_max value: 16.862199999999998 - type: nauc_recall_at_3_std value: 1.1521 - type: nauc_recall_at_3_diff1 value: 44.2778 - type: nauc_recall_at_5_max value: 24.669 - type: nauc_recall_at_5_std value: -1.4235 - type: nauc_recall_at_5_diff1 value: 48.987199999999994 - type: nauc_recall_at_10_max value: 15.8398 - type: nauc_recall_at_10_std value: -12.6435 - type: nauc_recall_at_10_diff1 value: 43.087900000000005 - type: nauc_recall_at_20_max value: 19.5526 - type: nauc_recall_at_20_std value: 7.2501 - type: nauc_recall_at_20_diff1 value: 50.6734 - type: nauc_recall_at_100_max value: .nan - type: nauc_recall_at_100_std value: .nan - type: nauc_recall_at_100_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 24.196 - type: nauc_precision_at_1_std value: -4.1928 - type: nauc_precision_at_1_diff1 value: 52.1914 - type: nauc_precision_at_3_max value: 16.862199999999998 - type: nauc_precision_at_3_std value: 1.1521 - type: nauc_precision_at_3_diff1 value: 44.2778 - type: nauc_precision_at_5_max value: 24.669 - type: nauc_precision_at_5_std value: -1.4235 - type: nauc_precision_at_5_diff1 value: 48.987199999999994 - type: nauc_precision_at_10_max value: 15.8398 - type: nauc_precision_at_10_std value: -12.6435 - type: nauc_precision_at_10_diff1 value: 43.087900000000005 - type: nauc_precision_at_20_max value: 19.5526 - type: nauc_precision_at_20_std value: 7.2501 - type: nauc_precision_at_20_diff1 value: 50.6734 - type: nauc_precision_at_100_max value: 100.0 - type: nauc_precision_at_100_std value: 100.0 - type: nauc_precision_at_100_diff1 value: 100.0 - type: nauc_precision_at_1000_max value: .nan - type: nauc_precision_at_1000_std value: .nan - type: nauc_precision_at_1000_diff1 value: .nan - type: nauc_mrr_at_1_max value: 24.196 - type: nauc_mrr_at_1_std value: -4.1928 - type: nauc_mrr_at_1_diff1 value: 52.1914 - type: nauc_mrr_at_3_max value: 21.205199999999998 - type: nauc_mrr_at_3_std value: -1.0412 - type: nauc_mrr_at_3_diff1 value: 46.8352 - type: nauc_mrr_at_5_max value: 23.086000000000002 - type: nauc_mrr_at_5_std value: -1.5817999999999999 - type: nauc_mrr_at_5_diff1 value: 47.8869 - type: nauc_mrr_at_10_max value: 21.9837 - type: nauc_mrr_at_10_std value: -2.8929 - type: nauc_mrr_at_10_diff1 value: 47.045500000000004 - type: nauc_mrr_at_20_max value: 22.1671 - type: nauc_mrr_at_20_std value: -1.7763999999999998 - type: nauc_mrr_at_20_diff1 value: 47.388999999999996 - type: nauc_mrr_at_100_max value: 22.0523 - type: nauc_mrr_at_100_std value: -1.4708 - type: nauc_mrr_at_100_diff1 value: 47.2193 - type: nauc_mrr_at_1000_max value: 22.0523 - type: nauc_mrr_at_1000_std value: -1.4708 - type: nauc_mrr_at_1000_diff1 value: 47.2193 - type: main_score value: 45.957 task: type: Retrieval - dataset: config: default name: MTEB SciFact (default) revision: 0228b52cf27578f30900b9e5271d331663a030d7 split: test type: mteb/scifact metrics: - type: ndcg_at_1 value: 61.333000000000006 - type: ndcg_at_3 value: 68.60199999999999 - type: ndcg_at_5 value: 70.411 - type: ndcg_at_10 value: 73.413 - type: ndcg_at_20 value: 74.58200000000001 - type: ndcg_at_100 value: 75.592 - type: ndcg_at_1000 value: 76.062 - type: map_at_1 value: 58.428000000000004 - type: map_at_3 value: 65.924 - type: map_at_5 value: 67.187 - type: map_at_10 value: 68.755 - type: map_at_20 value: 69.122 - type: map_at_100 value: 69.26299999999999 - type: map_at_1000 value: 69.284 - type: recall_at_1 value: 58.428000000000004 - type: recall_at_3 value: 73.489 - type: recall_at_5 value: 78.328 - type: recall_at_10 value: 86.722 - type: recall_at_20 value: 91.056 - type: recall_at_100 value: 96.5 - type: recall_at_1000 value: 100.0 - type: precision_at_1 value: 61.333000000000006 - type: precision_at_3 value: 26.889000000000003 - type: precision_at_5 value: 17.467 - type: precision_at_10 value: 9.866999999999999 - type: precision_at_20 value: 5.183 - type: precision_at_100 value: 1.093 - type: precision_at_1000 value: 0.11299999999999999 - type: mrr_at_1 value: 61.3333 - type: mrr_at_3 value: 67.61110000000001 - type: mrr_at_5 value: 68.61110000000001 - type: mrr_at_10 value: 69.6848 - type: mrr_at_20 value: 69.9864 - type: mrr_at_100 value: 70.1174 - type: mrr_at_1000 value: 70.13719999999999 - type: nauc_ndcg_at_1_max value: 64.11149999999999 - type: nauc_ndcg_at_1_std value: 14.688 - type: nauc_ndcg_at_1_diff1 value: 79.292 - type: nauc_ndcg_at_3_max value: 62.202400000000004 - type: nauc_ndcg_at_3_std value: 12.7864 - type: nauc_ndcg_at_3_diff1 value: 74.5286 - type: nauc_ndcg_at_5_max value: 61.394099999999995 - type: nauc_ndcg_at_5_std value: 14.099800000000002 - type: nauc_ndcg_at_5_diff1 value: 73.0227 - type: nauc_ndcg_at_10_max value: 66.13560000000001 - type: nauc_ndcg_at_10_std value: 15.5687 - type: nauc_ndcg_at_10_diff1 value: 74.4028 - type: nauc_ndcg_at_20_max value: 66.3651 - type: nauc_ndcg_at_20_std value: 16.8451 - type: nauc_ndcg_at_20_diff1 value: 75.0316 - type: nauc_ndcg_at_100_max value: 65.5397 - type: nauc_ndcg_at_100_std value: 17.0616 - type: nauc_ndcg_at_100_diff1 value: 74.8458 - type: nauc_ndcg_at_1000_max value: 65.1443 - type: nauc_ndcg_at_1000_std value: 16.2639 - type: nauc_ndcg_at_1000_diff1 value: 75.0683 - type: nauc_map_at_1_max value: 58.674499999999995 - type: nauc_map_at_1_std value: 4.7646 - type: nauc_map_at_1_diff1 value: 80.0349 - type: nauc_map_at_3_max value: 60.1997 - type: nauc_map_at_3_std value: 9.1883 - type: nauc_map_at_3_diff1 value: 75.9478 - type: nauc_map_at_5_max value: 60.7738 - type: nauc_map_at_5_std value: 10.9619 - type: nauc_map_at_5_diff1 value: 74.8759 - type: nauc_map_at_10_max value: 63.547399999999996 - type: nauc_map_at_10_std value: 12.7719 - type: nauc_map_at_10_diff1 value: 75.208 - type: nauc_map_at_20_max value: 63.5955 - type: nauc_map_at_20_std value: 13.279 - type: nauc_map_at_20_diff1 value: 75.3911 - type: nauc_map_at_100_max value: 63.5034 - type: nauc_map_at_100_std value: 13.3433 - type: nauc_map_at_100_diff1 value: 75.3801 - type: nauc_map_at_1000_max value: 63.4857 - type: nauc_map_at_1000_std value: 13.3133 - type: nauc_map_at_1000_diff1 value: 75.3873 - type: nauc_recall_at_1_max value: 58.674499999999995 - type: nauc_recall_at_1_std value: 4.7646 - type: nauc_recall_at_1_diff1 value: 80.0349 - type: nauc_recall_at_3_max value: 59.0557 - type: nauc_recall_at_3_std value: 9.031799999999999 - type: nauc_recall_at_3_diff1 value: 70.52929999999999 - type: nauc_recall_at_5_max value: 57.2901 - type: nauc_recall_at_5_std value: 15.6434 - type: nauc_recall_at_5_diff1 value: 64.9777 - type: nauc_recall_at_10_max value: 75.7042 - type: nauc_recall_at_10_std value: 18.401600000000002 - type: nauc_recall_at_10_diff1 value: 69.11580000000001 - type: nauc_recall_at_20_max value: 82.5429 - type: nauc_recall_at_20_std value: 29.412899999999997 - type: nauc_recall_at_20_diff1 value: 73.2713 - type: nauc_recall_at_100_max value: 81.7594 - type: nauc_recall_at_100_std value: 51.816300000000005 - type: nauc_recall_at_100_diff1 value: 65.2683 - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_precision_at_1_max value: 64.11149999999999 - type: nauc_precision_at_1_std value: 14.688 - type: nauc_precision_at_1_diff1 value: 79.292 - type: nauc_precision_at_3_max value: 49.7731 - type: nauc_precision_at_3_std value: 29.6594 - type: nauc_precision_at_3_diff1 value: 39.8605 - type: nauc_precision_at_5_max value: 45.1136 - type: nauc_precision_at_5_std value: 38.2477 - type: nauc_precision_at_5_diff1 value: 23.9286 - type: nauc_precision_at_10_max value: 43.4848 - type: nauc_precision_at_10_std value: 45.5735 - type: nauc_precision_at_10_diff1 value: 5.6292 - type: nauc_precision_at_20_max value: 37.7011 - type: nauc_precision_at_20_std value: 51.105199999999996 - type: nauc_precision_at_20_diff1 value: -2.2522 - type: nauc_precision_at_100_max value: 27.0942 - type: nauc_precision_at_100_std value: 58.4669 - type: nauc_precision_at_100_diff1 value: -19.608 - type: nauc_precision_at_1000_max value: 17.333000000000002 - type: nauc_precision_at_1000_std value: 56.925599999999996 - type: nauc_precision_at_1000_diff1 value: -32.3578 - type: nauc_mrr_at_1_max value: 64.11149999999999 - type: nauc_mrr_at_1_std value: 14.688 - type: nauc_mrr_at_1_diff1 value: 79.292 - type: nauc_mrr_at_3_max value: 64.43100000000001 - type: nauc_mrr_at_3_std value: 16.4446 - type: nauc_mrr_at_3_diff1 value: 75.9528 - type: nauc_mrr_at_5_max value: 64.1166 - type: nauc_mrr_at_5_std value: 17.9375 - type: nauc_mrr_at_5_diff1 value: 75.14150000000001 - type: nauc_mrr_at_10_max value: 65.4043 - type: nauc_mrr_at_10_std value: 17.5929 - type: nauc_mrr_at_10_diff1 value: 75.9545 - type: nauc_mrr_at_20_max value: 65.4177 - type: nauc_mrr_at_20_std value: 17.6251 - type: nauc_mrr_at_20_diff1 value: 76.06540000000001 - type: nauc_mrr_at_100_max value: 65.3124 - type: nauc_mrr_at_100_std value: 17.6635 - type: nauc_mrr_at_100_diff1 value: 76.04570000000001 - type: nauc_mrr_at_1000_max value: 65.2954 - type: nauc_mrr_at_1000_std value: 17.6416 - type: nauc_mrr_at_1000_diff1 value: 76.0508 - type: main_score value: 73.413 task: type: Retrieval - dataset: config: default name: MTEB SciFact-PL (default) revision: 47932a35f045ef8ed01ba82bf9ff67f6e109207e split: test type: clarin-knext/scifact-pl metrics: - type: ndcg_at_1 value: 28.666999999999998 - type: ndcg_at_3 value: 34.56 - type: ndcg_at_5 value: 36.183 - type: ndcg_at_10 value: 38.862 - type: ndcg_at_20 value: 39.871 - type: ndcg_at_100 value: 42.934 - type: ndcg_at_1000 value: 45.439 - type: map_at_1 value: 26.789 - type: map_at_3 value: 32.231 - type: map_at_5 value: 33.306999999999995 - type: map_at_10 value: 34.538999999999994 - type: map_at_20 value: 34.838 - type: map_at_100 value: 35.22 - type: map_at_1000 value: 35.312 - type: recall_at_1 value: 26.789 - type: recall_at_3 value: 39.078 - type: recall_at_5 value: 43.217 - type: recall_at_10 value: 51.00599999999999 - type: recall_at_20 value: 54.922000000000004 - type: recall_at_100 value: 72.044 - type: recall_at_1000 value: 91.822 - type: precision_at_1 value: 28.666999999999998 - type: precision_at_3 value: 14.333000000000002 - type: precision_at_5 value: 9.667 - type: precision_at_10 value: 5.867 - type: precision_at_20 value: 3.167 - type: precision_at_100 value: 0.8130000000000001 - type: precision_at_1000 value: 0.104 - type: mrr_at_1 value: 28.6667 - type: mrr_at_3 value: 33.8333 - type: mrr_at_5 value: 34.75 - type: mrr_at_10 value: 35.753800000000005 - type: mrr_at_20 value: 35.983599999999996 - type: mrr_at_100 value: 36.344300000000004 - type: mrr_at_1000 value: 36.4276 - type: nauc_ndcg_at_1_max value: 47.3964 - type: nauc_ndcg_at_1_std value: 23.1255 - type: nauc_ndcg_at_1_diff1 value: 61.3553 - type: nauc_ndcg_at_3_max value: 46.8095 - type: nauc_ndcg_at_3_std value: 20.8113 - type: nauc_ndcg_at_3_diff1 value: 50.5186 - type: nauc_ndcg_at_5_max value: 48.563 - type: nauc_ndcg_at_5_std value: 22.9279 - type: nauc_ndcg_at_5_diff1 value: 52.3505 - type: nauc_ndcg_at_10_max value: 51.0144 - type: nauc_ndcg_at_10_std value: 25.165399999999998 - type: nauc_ndcg_at_10_diff1 value: 51.928399999999996 - type: nauc_ndcg_at_20_max value: 51.2402 - type: nauc_ndcg_at_20_std value: 26.264599999999998 - type: nauc_ndcg_at_20_diff1 value: 52.085800000000006 - type: nauc_ndcg_at_100_max value: 51.486900000000006 - type: nauc_ndcg_at_100_std value: 27.043499999999998 - type: nauc_ndcg_at_100_diff1 value: 51.4942 - type: nauc_ndcg_at_1000_max value: 50.8764 - type: nauc_ndcg_at_1000_std value: 26.6247 - type: nauc_ndcg_at_1000_diff1 value: 52.4891 - type: nauc_map_at_1_max value: 45.3571 - type: nauc_map_at_1_std value: 22.224 - type: nauc_map_at_1_diff1 value: 60.9856 - type: nauc_map_at_3_max value: 46.641 - type: nauc_map_at_3_std value: 21.3558 - type: nauc_map_at_3_diff1 value: 53.9335 - type: nauc_map_at_5_max value: 47.604600000000005 - type: nauc_map_at_5_std value: 22.3532 - type: nauc_map_at_5_diff1 value: 54.727599999999995 - type: nauc_map_at_10_max value: 48.8623 - type: nauc_map_at_10_std value: 23.3967 - type: nauc_map_at_10_diff1 value: 54.4613 - type: nauc_map_at_20_max value: 48.9766 - type: nauc_map_at_20_std value: 23.848 - type: nauc_map_at_20_diff1 value: 54.476 - type: nauc_map_at_100_max value: 48.9984 - type: nauc_map_at_100_std value: 23.9562 - type: nauc_map_at_100_diff1 value: 54.4079 - type: nauc_map_at_1000_max value: 48.9822 - type: nauc_map_at_1000_std value: 23.9555 - type: nauc_map_at_1000_diff1 value: 54.4478 - type: nauc_recall_at_1_max value: 45.3571 - type: nauc_recall_at_1_std value: 22.224 - type: nauc_recall_at_1_diff1 value: 60.9856 - type: nauc_recall_at_3_max value: 44.850699999999996 - type: nauc_recall_at_3_std value: 18.1467 - type: nauc_recall_at_3_diff1 value: 41.9454 - type: nauc_recall_at_5_max value: 48.7443 - type: nauc_recall_at_5_std value: 22.654 - type: nauc_recall_at_5_diff1 value: 45.2269 - type: nauc_recall_at_10_max value: 55.4795 - type: nauc_recall_at_10_std value: 29.117700000000003 - type: nauc_recall_at_10_diff1 value: 44.5212 - type: nauc_recall_at_20_max value: 56.4723 - type: nauc_recall_at_20_std value: 33.0929 - type: nauc_recall_at_20_diff1 value: 45.1359 - type: nauc_recall_at_100_max value: 60.0848 - type: nauc_recall_at_100_std value: 40.9103 - type: nauc_recall_at_100_diff1 value: 38.1492 - type: nauc_recall_at_1000_max value: 59.6927 - type: nauc_recall_at_1000_std value: 53.2595 - type: nauc_recall_at_1000_diff1 value: 42.1092 - type: nauc_precision_at_1_max value: 47.3964 - type: nauc_precision_at_1_std value: 23.1255 - type: nauc_precision_at_1_diff1 value: 61.3553 - type: nauc_precision_at_3_max value: 47.0326 - type: nauc_precision_at_3_std value: 19.482499999999998 - type: nauc_precision_at_3_diff1 value: 40.994 - type: nauc_precision_at_5_max value: 49.263200000000005 - type: nauc_precision_at_5_std value: 23.4494 - type: nauc_precision_at_5_diff1 value: 42.436099999999996 - type: nauc_precision_at_10_max value: 52.249900000000004 - type: nauc_precision_at_10_std value: 28.228199999999998 - type: nauc_precision_at_10_diff1 value: 35.3869 - type: nauc_precision_at_20_max value: 51.8228 - type: nauc_precision_at_20_std value: 32.289699999999996 - type: nauc_precision_at_20_diff1 value: 33.976800000000004 - type: nauc_precision_at_100_max value: 51.063199999999995 - type: nauc_precision_at_100_std value: 35.375099999999996 - type: nauc_precision_at_100_diff1 value: 25.5759 - type: nauc_precision_at_1000_max value: 30.785600000000002 - type: nauc_precision_at_1000_std value: 26.445600000000002 - type: nauc_precision_at_1000_diff1 value: 11.6568 - type: nauc_mrr_at_1_max value: 47.3964 - type: nauc_mrr_at_1_std value: 23.1255 - type: nauc_mrr_at_1_diff1 value: 61.3553 - type: nauc_mrr_at_3_max value: 48.303000000000004 - type: nauc_mrr_at_3_std value: 22.5707 - type: nauc_mrr_at_3_diff1 value: 53.01519999999999 - type: nauc_mrr_at_5_max value: 49.1763 - type: nauc_mrr_at_5_std value: 23.907 - type: nauc_mrr_at_5_diff1 value: 54.146899999999995 - type: nauc_mrr_at_10_max value: 49.9118 - type: nauc_mrr_at_10_std value: 24.5548 - type: nauc_mrr_at_10_diff1 value: 53.9486 - type: nauc_mrr_at_20_max value: 49.9542 - type: nauc_mrr_at_20_std value: 24.729200000000002 - type: nauc_mrr_at_20_diff1 value: 54.01519999999999 - type: nauc_mrr_at_100_max value: 49.9469 - type: nauc_mrr_at_100_std value: 24.8281 - type: nauc_mrr_at_100_diff1 value: 53.9569 - type: nauc_mrr_at_1000_max value: 49.9349 - type: nauc_mrr_at_1000_std value: 24.8281 - type: nauc_mrr_at_1000_diff1 value: 53.9925 - type: main_score value: 38.862 task: type: Retrieval - dataset: config: default name: MTEB TRECCOVID (default) revision: bb9466bac8153a0349341eb1b22e06409e78ef4e split: test type: mteb/trec-covid metrics: - type: ndcg_at_1 value: 77.0 - type: ndcg_at_3 value: 77.95 - type: ndcg_at_5 value: 76.3 - type: ndcg_at_10 value: 75.729 - type: ndcg_at_20 value: 71.923 - type: ndcg_at_100 value: 57.583 - type: ndcg_at_1000 value: 51.809000000000005 - type: map_at_1 value: 0.22200000000000003 - type: map_at_3 value: 0.6459999999999999 - type: map_at_5 value: 1.044 - type: map_at_10 value: 1.986 - type: map_at_20 value: 3.544 - type: map_at_100 value: 11.19 - type: map_at_1000 value: 26.351000000000003 - type: recall_at_1 value: 0.22200000000000003 - type: recall_at_3 value: 0.677 - type: recall_at_5 value: 1.117 - type: recall_at_10 value: 2.189 - type: recall_at_20 value: 4.031 - type: recall_at_100 value: 14.512 - type: recall_at_1000 value: 48.644999999999996 - type: precision_at_1 value: 84.0 - type: precision_at_3 value: 84.667 - type: precision_at_5 value: 82.0 - type: precision_at_10 value: 80.80000000000001 - type: precision_at_20 value: 76.1 - type: precision_at_100 value: 59.160000000000004 - type: precision_at_1000 value: 22.692 - type: mrr_at_1 value: 84.0 - type: mrr_at_3 value: 90.6667 - type: mrr_at_5 value: 91.0667 - type: mrr_at_10 value: 91.0667 - type: mrr_at_20 value: 91.0667 - type: mrr_at_100 value: 91.0667 - type: mrr_at_1000 value: 91.0667 - type: nauc_ndcg_at_1_max value: 48.8547 - type: nauc_ndcg_at_1_std value: 49.0681 - type: nauc_ndcg_at_1_diff1 value: -30.4738 - type: nauc_ndcg_at_3_max value: 44.5189 - type: nauc_ndcg_at_3_std value: 51.435900000000004 - type: nauc_ndcg_at_3_diff1 value: 0.6276999999999999 - type: nauc_ndcg_at_5_max value: 45.063199999999995 - type: nauc_ndcg_at_5_std value: 53.6234 - type: nauc_ndcg_at_5_diff1 value: -3.7708 - type: nauc_ndcg_at_10_max value: 45.2686 - type: nauc_ndcg_at_10_std value: 55.53979999999999 - type: nauc_ndcg_at_10_diff1 value: -8.5233 - type: nauc_ndcg_at_20_max value: 54.0756 - type: nauc_ndcg_at_20_std value: 61.454 - type: nauc_ndcg_at_20_diff1 value: -14.1012 - type: nauc_ndcg_at_100_max value: 48.0204 - type: nauc_ndcg_at_100_std value: 76.6764 - type: nauc_ndcg_at_100_diff1 value: -21.3943 - type: nauc_ndcg_at_1000_max value: 42.2604 - type: nauc_ndcg_at_1000_std value: 76.43090000000001 - type: nauc_ndcg_at_1000_diff1 value: -10.339 - type: nauc_map_at_1_max value: 19.6618 - type: nauc_map_at_1_std value: 12.3368 - type: nauc_map_at_1_diff1 value: 5.8378 - type: nauc_map_at_3_max value: 19.9671 - type: nauc_map_at_3_std value: 14.360500000000002 - type: nauc_map_at_3_diff1 value: 11.2964 - type: nauc_map_at_5_max value: 21.484 - type: nauc_map_at_5_std value: 15.7254 - type: nauc_map_at_5_diff1 value: 8.277800000000001 - type: nauc_map_at_10_max value: 19.5425 - type: nauc_map_at_10_std value: 17.1697 - type: nauc_map_at_10_diff1 value: 4.6949 - type: nauc_map_at_20_max value: 30.469 - type: nauc_map_at_20_std value: 21.2894 - type: nauc_map_at_20_diff1 value: 5.5824 - type: nauc_map_at_100_max value: 47.6198 - type: nauc_map_at_100_std value: 52.692099999999996 - type: nauc_map_at_100_diff1 value: -4.3684 - type: nauc_map_at_1000_max value: 46.7244 - type: nauc_map_at_1000_std value: 79.96809999999999 - type: nauc_map_at_1000_diff1 value: -11.427800000000001 - type: nauc_recall_at_1_max value: 19.6618 - type: nauc_recall_at_1_std value: 12.3368 - type: nauc_recall_at_1_diff1 value: 5.8378 - type: nauc_recall_at_3_max value: 15.8659 - type: nauc_recall_at_3_std value: 9.417200000000001 - type: nauc_recall_at_3_diff1 value: 12.217 - type: nauc_recall_at_5_max value: 16.6584 - type: nauc_recall_at_5_std value: 9.642299999999999 - type: nauc_recall_at_5_diff1 value: 7.8316 - type: nauc_recall_at_10_max value: 13.4892 - type: nauc_recall_at_10_std value: 9.0525 - type: nauc_recall_at_10_diff1 value: 5.1017 - type: nauc_recall_at_20_max value: 23.8122 - type: nauc_recall_at_20_std value: 10.963199999999999 - type: nauc_recall_at_20_diff1 value: 5.496 - type: nauc_recall_at_100_max value: 34.0401 - type: nauc_recall_at_100_std value: 38.919399999999996 - type: nauc_recall_at_100_diff1 value: -2.4619999999999997 - type: nauc_recall_at_1000_max value: 33.3684 - type: nauc_recall_at_1000_std value: 67.9744 - type: nauc_recall_at_1000_diff1 value: -6.4093 - type: nauc_precision_at_1_max value: 58.4315 - type: nauc_precision_at_1_std value: 64.72139999999999 - type: nauc_precision_at_1_diff1 value: -28.375099999999996 - type: nauc_precision_at_3_max value: 43.6798 - type: nauc_precision_at_3_std value: 62.983599999999996 - type: nauc_precision_at_3_diff1 value: 4.495 - type: nauc_precision_at_5_max value: 45.5822 - type: nauc_precision_at_5_std value: 66.2558 - type: nauc_precision_at_5_diff1 value: -0.5026 - type: nauc_precision_at_10_max value: 46.4246 - type: nauc_precision_at_10_std value: 61.202999999999996 - type: nauc_precision_at_10_diff1 value: -7.648499999999999 - type: nauc_precision_at_20_max value: 59.4366 - type: nauc_precision_at_20_std value: 62.530699999999996 - type: nauc_precision_at_20_diff1 value: -14.7791 - type: nauc_precision_at_100_max value: 47.7262 - type: nauc_precision_at_100_std value: 75.8917 - type: nauc_precision_at_100_diff1 value: -21.1255 - type: nauc_precision_at_1000_max value: 26.461299999999998 - type: nauc_precision_at_1000_std value: 56.535599999999995 - type: nauc_precision_at_1000_diff1 value: -16.9377 - type: nauc_mrr_at_1_max value: 58.4315 - type: nauc_mrr_at_1_std value: 64.72139999999999 - type: nauc_mrr_at_1_diff1 value: -28.375099999999996 - type: nauc_mrr_at_3_max value: 59.156600000000005 - type: nauc_mrr_at_3_std value: 66.3882 - type: nauc_mrr_at_3_diff1 value: -30.3537 - type: nauc_mrr_at_5_max value: 59.3318 - type: nauc_mrr_at_5_std value: 66.115 - type: nauc_mrr_at_5_diff1 value: -30.9728 - type: nauc_mrr_at_10_max value: 59.3318 - type: nauc_mrr_at_10_std value: 66.115 - type: nauc_mrr_at_10_diff1 value: -30.9728 - type: nauc_mrr_at_20_max value: 59.3318 - type: nauc_mrr_at_20_std value: 66.115 - type: nauc_mrr_at_20_diff1 value: -30.9728 - type: nauc_mrr_at_100_max value: 59.3318 - type: nauc_mrr_at_100_std value: 66.115 - type: nauc_mrr_at_100_diff1 value: -30.9728 - type: nauc_mrr_at_1000_max value: 59.3318 - type: nauc_mrr_at_1000_std value: 66.115 - type: nauc_mrr_at_1000_diff1 value: -30.9728 - type: main_score value: 75.729 task: type: Retrieval - dataset: config: default name: MTEB TRECCOVID-PL (default) revision: 81bcb408f33366c2a20ac54adafad1ae7e877fdd split: test type: clarin-knext/trec-covid-pl metrics: - type: ndcg_at_1 value: 33.0 - type: ndcg_at_3 value: 35.022 - type: ndcg_at_5 value: 33.731 - type: ndcg_at_10 value: 31.52 - type: ndcg_at_20 value: 27.831 - type: ndcg_at_100 value: 21.394 - type: ndcg_at_1000 value: 19.528000000000002 - type: map_at_1 value: 0.089 - type: map_at_3 value: 0.27 - type: map_at_5 value: 0.4 - type: map_at_10 value: 0.622 - type: map_at_20 value: 0.91 - type: map_at_100 value: 2.373 - type: map_at_1000 value: 6.121 - type: recall_at_1 value: 0.089 - type: recall_at_3 value: 0.314 - type: recall_at_5 value: 0.492 - type: recall_at_10 value: 0.851 - type: recall_at_20 value: 1.367 - type: recall_at_100 value: 4.763 - type: recall_at_1000 value: 18.496000000000002 - type: precision_at_1 value: 38.0 - type: precision_at_3 value: 37.333 - type: precision_at_5 value: 35.6 - type: precision_at_10 value: 32.6 - type: precision_at_20 value: 27.900000000000002 - type: precision_at_100 value: 21.560000000000002 - type: precision_at_1000 value: 9.306000000000001 - type: mrr_at_1 value: 38.0 - type: mrr_at_3 value: 46.0 - type: mrr_at_5 value: 47.9 - type: mrr_at_10 value: 49.9214 - type: mrr_at_20 value: 50.1641 - type: mrr_at_100 value: 50.626000000000005 - type: mrr_at_1000 value: 50.6339 - type: nauc_ndcg_at_1_max value: 34.6986 - type: nauc_ndcg_at_1_std value: 32.1162 - type: nauc_ndcg_at_1_diff1 value: -10.2072 - type: nauc_ndcg_at_3_max value: 26.7849 - type: nauc_ndcg_at_3_std value: 11.6996 - type: nauc_ndcg_at_3_diff1 value: -12.581800000000001 - type: nauc_ndcg_at_5_max value: 24.4334 - type: nauc_ndcg_at_5_std value: 5.9356 - type: nauc_ndcg_at_5_diff1 value: -7.063899999999999 - type: nauc_ndcg_at_10_max value: 24.916900000000002 - type: nauc_ndcg_at_10_std value: 4.5668 - type: nauc_ndcg_at_10_diff1 value: -2.1188 - type: nauc_ndcg_at_20_max value: 26.273600000000002 - type: nauc_ndcg_at_20_std value: 7.7196 - type: nauc_ndcg_at_20_diff1 value: 0.8089000000000001 - type: nauc_ndcg_at_100_max value: 33.1944 - type: nauc_ndcg_at_100_std value: 16.3776 - type: nauc_ndcg_at_100_diff1 value: -2.5803 - type: nauc_ndcg_at_1000_max value: 51.3621 - type: nauc_ndcg_at_1000_std value: 29.2173 - type: nauc_ndcg_at_1000_diff1 value: -6.7301 - type: nauc_map_at_1_max value: 27.8013 - type: nauc_map_at_1_std value: 12.5435 - type: nauc_map_at_1_diff1 value: -16.522100000000002 - type: nauc_map_at_3_max value: 19.433500000000002 - type: nauc_map_at_3_std value: -5.2681 - type: nauc_map_at_3_diff1 value: -20.3585 - type: nauc_map_at_5_max value: 15.0378 - type: nauc_map_at_5_std value: -10.252500000000001 - type: nauc_map_at_5_diff1 value: -13.5849 - type: nauc_map_at_10_max value: 16.6286 - type: nauc_map_at_10_std value: -8.9473 - type: nauc_map_at_10_diff1 value: -11.0427 - type: nauc_map_at_20_max value: 16.3801 - type: nauc_map_at_20_std value: -3.9566 - type: nauc_map_at_20_diff1 value: -5.8741 - type: nauc_map_at_100_max value: 23.8517 - type: nauc_map_at_100_std value: 12.7052 - type: nauc_map_at_100_diff1 value: -4.0786999999999995 - type: nauc_map_at_1000_max value: 50.2364 - type: nauc_map_at_1000_std value: 45.1323 - type: nauc_map_at_1000_diff1 value: -9.7797 - type: nauc_recall_at_1_max value: 27.8013 - type: nauc_recall_at_1_std value: 12.5435 - type: nauc_recall_at_1_diff1 value: -16.522100000000002 - type: nauc_recall_at_3_max value: 12.021600000000001 - type: nauc_recall_at_3_std value: -12.195 - type: nauc_recall_at_3_diff1 value: -17.7181 - type: nauc_recall_at_5_max value: 8.317499999999999 - type: nauc_recall_at_5_std value: -17.3643 - type: nauc_recall_at_5_diff1 value: -11.0701 - type: nauc_recall_at_10_max value: 12.629199999999999 - type: nauc_recall_at_10_std value: -17.8 - type: nauc_recall_at_10_diff1 value: -3.1063 - type: nauc_recall_at_20_max value: 14.7957 - type: nauc_recall_at_20_std value: -12.6812 - type: nauc_recall_at_20_diff1 value: 2.5131 - type: nauc_recall_at_100_max value: 26.763900000000003 - type: nauc_recall_at_100_std value: 1.4699 - type: nauc_recall_at_100_diff1 value: 2.1349 - type: nauc_recall_at_1000_max value: 57.9288 - type: nauc_recall_at_1000_std value: 34.260600000000004 - type: nauc_recall_at_1000_diff1 value: -7.8182 - type: nauc_precision_at_1_max value: 35.3859 - type: nauc_precision_at_1_std value: 27.918300000000002 - type: nauc_precision_at_1_diff1 value: -9.804300000000001 - type: nauc_precision_at_3_max value: 23.3863 - type: nauc_precision_at_3_std value: 3.6164 - type: nauc_precision_at_3_diff1 value: -11.8673 - type: nauc_precision_at_5_max value: 19.648 - type: nauc_precision_at_5_std value: -1.4232 - type: nauc_precision_at_5_diff1 value: -4.7019 - type: nauc_precision_at_10_max value: 22.9189 - type: nauc_precision_at_10_std value: 0.6967 - type: nauc_precision_at_10_diff1 value: -0.757 - type: nauc_precision_at_20_max value: 27.294800000000002 - type: nauc_precision_at_20_std value: 8.280800000000001 - type: nauc_precision_at_20_diff1 value: 2.1543 - type: nauc_precision_at_100_max value: 36.647600000000004 - type: nauc_precision_at_100_std value: 21.0732 - type: nauc_precision_at_100_diff1 value: -4.3472 - type: nauc_precision_at_1000_max value: 56.1096 - type: nauc_precision_at_1000_std value: 44.7203 - type: nauc_precision_at_1000_diff1 value: -16.724800000000002 - type: nauc_mrr_at_1_max value: 35.3859 - type: nauc_mrr_at_1_std value: 27.918300000000002 - type: nauc_mrr_at_1_diff1 value: -9.804300000000001 - type: nauc_mrr_at_3_max value: 28.7924 - type: nauc_mrr_at_3_std value: 14.6514 - type: nauc_mrr_at_3_diff1 value: -5.1166 - type: nauc_mrr_at_5_max value: 27.824199999999998 - type: nauc_mrr_at_5_std value: 14.530499999999998 - type: nauc_mrr_at_5_diff1 value: -8.6874 - type: nauc_mrr_at_10_max value: 28.8111 - type: nauc_mrr_at_10_std value: 12.7967 - type: nauc_mrr_at_10_diff1 value: -2.1439 - type: nauc_mrr_at_20_max value: 28.6582 - type: nauc_mrr_at_20_std value: 12.2919 - type: nauc_mrr_at_20_diff1 value: -2.7289999999999996 - type: nauc_mrr_at_100_max value: 28.9392 - type: nauc_mrr_at_100_std value: 12.848200000000002 - type: nauc_mrr_at_100_diff1 value: -3.1357000000000004 - type: nauc_mrr_at_1000_max value: 28.9231 - type: nauc_mrr_at_1000_std value: 12.8882 - type: nauc_mrr_at_1000_diff1 value: -3.1525999999999996 - type: main_score value: 31.52 task: type: Retrieval --- # MedEmbed: Specialized Embedding Model for Medical and Clinical Information Retrieval ![benchmark-scores](https://cdn-uploads.huggingface.co/production/uploads/60c8619d95d852a24572b025/gTx5-m68LQ3eyNd6fLki2.png) ## Model Description MedEmbed is a family of embedding models fine-tuned specifically for medical and clinical data, designed to enhance performance in healthcare-related natural language processing (NLP) tasks, particularly information retrieval. **GitHub Repo:** [https://github.com/abhinand5/MedEmbed](https://github.com/abhinand5/MedEmbed) **Technical Blog Post:** [https://huggingface.co/blog/abhinand/medembed-finetuned-embedding-models-for-medical-ir](https://huggingface.co/blog/abhinand/medembed-finetuned-embedding-models-for-medical-ir) ## Intended Use This model is intended for use in medical and clinical contexts to improve information retrieval, question answering, and semantic search tasks. It can be integrated into healthcare systems, research tools, and medical literature databases to enhance search capabilities and information access. ## Training Data ![synthetic-datagen-flow](https://cdn-uploads.huggingface.co/production/uploads/60c8619d95d852a24572b025/asaA5QDO_j0PWFQV9NXCu.png) The model was trained using a simple yet effective synthetic data generation pipeline: 1. Source: Clinical notes from PubMed Central (PMC) 2. Processing: [LLaMA 3.1 70B](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) model used to generate query-response pairs 3. Augmentation: Negative sampling for challenging examples 4. Format: Triplets (query, positive response, negative response) for contrastive learning ## Performance MedEmbed consistently outperforms general-purpose embedding models across various medical NLP benchmarks: - ArguAna - MedicalQARetrieval - NFCorpus - PublicHealthQA - TRECCOVID Specific performance metrics (nDCG, MAP, Recall, Precision, MRR) are available in the full documentation. ## Limitations While highly effective for medical and clinical data, this model may not generalize well to non-medical domains. It should be used with caution in general-purpose NLP tasks. ## Ethical Considerations Users should be aware of potential biases in medical data and the ethical implications of AI in healthcare. This model should be used as a tool to assist, not replace, human expertise in medical decision-making. ## Citation If you use this model in your research, please cite: ```bibtex @software{balachandran2024medembed, author = {Balachandran, Abhinand}, title = {MedEmbed: Medical-Focused Embedding Models}, year = {2024}, url = {https://github.com/abhinand5/MedEmbed} } ``` For more detailed information, visit our GitHub repository.