
# SGPT-2.7B-weightedmean-msmarco-specb-bitfit

## Usage

For usage instructions, refer to our codebase: https://github.com/Muennighoff/sgpt 

## Evaluation Results

For eval results, refer to the eval folder or our paper: https://arxiv.org/abs/2202.08904

## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 124796 with parameters:
```
{'batch_size': 4, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss` with parameters:
  ```
  {'scale': 20.0, 'similarity_fct': 'cos_sim'}
  ```

Parameters of the fit()-Method:
```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 7.5e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 1000,
    "weight_decay": 0.01
}
```


## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 300, 'do_lower_case': False}) with Transformer model: GPTNeoModel 
  (1): Pooling({'word_embedding_dimension': 2560, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': True, 'pooling_mode_lasttoken': False})
)
```

## Citing & Authors

```bibtex
@article{muennighoff2022sgpt,
  title={SGPT: GPT Sentence Embeddings for Semantic Search},
  author={Muennighoff, Niklas},
  journal={arXiv preprint arXiv:2202.08904},
  year={2022}
}
```
{"tags": ["sentence-transformers", "feature-extraction", "sentence-similarity", "mteb"], "pipeline_tag": "sentence-similarity", "model-index": [{"name": "SGPT-2.7B-weightedmean-msmarco-specb-bitfit", "results": [{"task": {"type": "Classification"}, "dataset": {"name": "MTEB AmazonCounterfactualClassification (en)", "type": "mteb/amazon_counterfactual", "config": "en", "split": "test", "revision": "2d8a100785abf0ae21420d2a55b0c56e3e1ea996"}, "metrics": [{"type": "accuracy", "value": 67.56716417910448}, {"type": "ap", "value": 30.75574629595259}, {"type": "f1", "value": 61.805121301858655}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB AmazonPolarityClassification", "type": "mteb/amazon_polarity", "config": "default", "split": "test", "revision": "80714f8dcf8cefc218ef4f8c5a966dd83f75a0e1"}, "metrics": [{"type": "accuracy", "value": 71.439575}, {"type": "ap", "value": 65.91341330532453}, {"type": "f1", "value": 70.90561852619555}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB AmazonReviewsClassification (en)", "type": "mteb/amazon_reviews_multi", "config": "en", "split": "test", "revision": "c379a6705fec24a2493fa68e011692605f44e119"}, "metrics": [{"type": "accuracy", "value": 35.748000000000005}, {"type": "f1", "value": 35.48576287186347}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB ArguAna", "type": "arguana", "config": "default", "split": "test", "revision": "5b3e3697907184a9b77a3c99ee9ea1a9cbb1e4e3"}, "metrics": [{"type": "map_at_1", "value": 25.96}, {"type": "map_at_10", "value": 41.619}, {"type": "map_at_100", "value": 42.673}, {"type": "map_at_1000", "value": 42.684}, {"type": "map_at_3", "value": 36.569}, {"type": "map_at_5", "value": 39.397}, {"type": "mrr_at_1", "value": 26.316}, {"type": "mrr_at_10", "value": 41.772}, {"type": "mrr_at_100", "value": 42.82}, {"type": "mrr_at_1000", "value": 42.83}, {"type": "mrr_at_3", "value": 36.724000000000004}, {"type": "mrr_at_5", "value": 39.528999999999996}, {"type": "ndcg_at_1", "value": 25.96}, {"type": "ndcg_at_10", "value": 50.491}, {"type": "ndcg_at_100", "value": 54.864999999999995}, {"type": "ndcg_at_1000", "value": 55.10699999999999}, {"type": "ndcg_at_3", "value": 40.053}, {"type": "ndcg_at_5", "value": 45.134}, {"type": "precision_at_1", "value": 25.96}, {"type": "precision_at_10", "value": 7.8950000000000005}, {"type": "precision_at_100", "value": 0.9780000000000001}, {"type": "precision_at_1000", "value": 0.1}, {"type": "precision_at_3", "value": 16.714000000000002}, {"type": "precision_at_5", "value": 12.489}, {"type": "recall_at_1", "value": 25.96}, {"type": "recall_at_10", "value": 78.947}, {"type": "recall_at_100", "value": 97.795}, {"type": "recall_at_1000", "value": 99.644}, {"type": "recall_at_3", "value": 50.141999999999996}, {"type": "recall_at_5", "value": 62.446999999999996}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB ArxivClusteringP2P", "type": "mteb/arxiv-clustering-p2p", "config": "default", "split": "test", "revision": "0bbdb47bcbe3a90093699aefeed338a0f28a7ee8"}, "metrics": [{"type": "v_measure", "value": 44.72125714642202}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB ArxivClusteringS2S", "type": "mteb/arxiv-clustering-s2s", "config": "default", "split": "test", "revision": "b73bd54100e5abfa6e3a23dcafb46fe4d2438dc3"}, "metrics": [{"type": "v_measure", "value": 35.081451519142064}]}, {"task": {"type": "Reranking"}, "dataset": {"name": "MTEB AskUbuntuDupQuestions", "type": "mteb/askubuntudupquestions-reranking", "config": "default", "split": "test", "revision": "4d853f94cd57d85ec13805aeeac3ae3e5eb4c49c"}, "metrics": [{"type": "map", "value": 59.634661990392054}, {"type": "mrr", "value": 73.6813525040672}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB BIOSSES", "type": "mteb/biosses-sts", "config": "default", "split": "test", "revision": "9ee918f184421b6bd48b78f6c714d86546106103"}, "metrics": [{"type": "cos_sim_pearson", "value": 87.42754550496836}, {"type": "cos_sim_spearman", "value": 84.84289705838664}, {"type": "euclidean_pearson", "value": 85.59331970450859}, {"type": "euclidean_spearman", "value": 85.8525586184271}, {"type": "manhattan_pearson", "value": 85.41233134466698}, {"type": "manhattan_spearman", "value": 85.52303303767404}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB Banking77Classification", "type": "mteb/banking77", "config": "default", "split": "test", "revision": "44fa15921b4c889113cc5df03dd4901b49161ab7"}, "metrics": [{"type": "accuracy", "value": 83.21753246753246}, {"type": "f1", "value": 83.15394543120915}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB BiorxivClusteringP2P", "type": "mteb/biorxiv-clustering-p2p", "config": "default", "split": "test", "revision": "11d0121201d1f1f280e8cc8f3d98fb9c4d9f9c55"}, "metrics": [{"type": "v_measure", "value": 34.41414219680629}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB BiorxivClusteringS2S", "type": "mteb/biorxiv-clustering-s2s", "config": "default", "split": "test", "revision": "c0fab014e1bcb8d3a5e31b2088972a1e01547dc1"}, "metrics": [{"type": "v_measure", "value": 30.533275862270028}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB CQADupstackAndroidRetrieval", "type": "BeIR/cqadupstack", "config": "default", "split": "test", "revision": "2b9f5791698b5be7bc5e10535c8690f20043c3db"}, "metrics": [{"type": "map_at_1", "value": 30.808999999999997}, {"type": "map_at_10", "value": 40.617}, {"type": "map_at_100", "value": 41.894999999999996}, {"type": "map_at_1000", "value": 42.025}, {"type": "map_at_3", "value": 37.0}, {"type": "map_at_5", "value": 38.993}, {"type": "mrr_at_1", "value": 37.482}, {"type": "mrr_at_10", "value": 46.497}, {"type": "mrr_at_100", "value": 47.144000000000005}, {"type": "mrr_at_1000", "value": 47.189}, {"type": "mrr_at_3", "value": 43.705}, {"type": "mrr_at_5", "value": 45.193}, {"type": "ndcg_at_1", "value": 37.482}, {"type": "ndcg_at_10", "value": 46.688}, {"type": "ndcg_at_100", "value": 51.726000000000006}, {"type": "ndcg_at_1000", "value": 53.825}, {"type": "ndcg_at_3", "value": 41.242000000000004}, {"type": "ndcg_at_5", "value": 43.657000000000004}, {"type": "precision_at_1", "value": 37.482}, {"type": "precision_at_10", "value": 8.827}, {"type": "precision_at_100", "value": 1.393}, {"type": "precision_at_1000", "value": 0.186}, {"type": "precision_at_3", "value": 19.361}, {"type": "precision_at_5", "value": 14.106}, {"type": "recall_at_1", "value": 30.808999999999997}, {"type": "recall_at_10", "value": 58.47}, {"type": "recall_at_100", "value": 80.51899999999999}, {"type": "recall_at_1000", "value": 93.809}, {"type": "recall_at_3", "value": 42.462}, {"type": "recall_at_5", "value": 49.385}, {"type": "map_at_1", "value": 26.962000000000003}, {"type": "map_at_10", "value": 36.93}, {"type": "map_at_100", "value": 38.102000000000004}, {"type": "map_at_1000", "value": 38.22}, {"type": "map_at_3", "value": 34.065}, {"type": "map_at_5", "value": 35.72}, {"type": "mrr_at_1", "value": 33.567}, {"type": "mrr_at_10", "value": 42.269}, {"type": "mrr_at_100", "value": 42.99}, {"type": "mrr_at_1000", "value": 43.033}, {"type": "mrr_at_3", "value": 40.064}, {"type": "mrr_at_5", "value": 41.258}, {"type": "ndcg_at_1", "value": 33.567}, {"type": "ndcg_at_10", "value": 42.405}, {"type": "ndcg_at_100", "value": 46.847}, {"type": "ndcg_at_1000", "value": 48.951}, {"type": "ndcg_at_3", "value": 38.312000000000005}, {"type": "ndcg_at_5", "value": 40.242}, {"type": "precision_at_1", "value": 33.567}, {"type": "precision_at_10", "value": 8.032}, {"type": "precision_at_100", "value": 1.295}, {"type": "precision_at_1000", "value": 0.17600000000000002}, {"type": "precision_at_3", "value": 18.662}, {"type": "precision_at_5", "value": 13.299}, {"type": "recall_at_1", "value": 26.962000000000003}, {"type": "recall_at_10", "value": 52.489}, {"type": "recall_at_100", "value": 71.635}, {"type": "recall_at_1000", "value": 85.141}, {"type": "recall_at_3", "value": 40.28}, {"type": "recall_at_5", "value": 45.757}, {"type": "map_at_1", "value": 36.318}, {"type": "map_at_10", "value": 47.97}, {"type": "map_at_100", "value": 49.003}, {"type": "map_at_1000", "value": 49.065999999999995}, {"type": "map_at_3", "value": 45.031}, {"type": "map_at_5", "value": 46.633}, {"type": "mrr_at_1", "value": 41.504999999999995}, {"type": "mrr_at_10", "value": 51.431000000000004}, {"type": "mrr_at_100", "value": 52.129000000000005}, {"type": "mrr_at_1000", "value": 52.161}, {"type": "mrr_at_3", "value": 48.934}, {"type": "mrr_at_5", "value": 50.42}, {"type": "ndcg_at_1", "value": 41.504999999999995}, {"type": "ndcg_at_10", "value": 53.676}, {"type": "ndcg_at_100", "value": 57.867000000000004}, {"type": "ndcg_at_1000", "value": 59.166}, {"type": "ndcg_at_3", "value": 48.516}, {"type": "ndcg_at_5", "value": 50.983999999999995}, {"type": "precision_at_1", "value": 41.504999999999995}, {"type": "precision_at_10", "value": 8.608}, {"type": "precision_at_100", "value": 1.1560000000000001}, {"type": "precision_at_1000", "value": 0.133}, {"type": "precision_at_3", "value": 21.462999999999997}, {"type": "precision_at_5", "value": 14.721}, {"type": "recall_at_1", "value": 36.318}, {"type": "recall_at_10", "value": 67.066}, {"type": "recall_at_100", "value": 85.34}, {"type": "recall_at_1000", "value": 94.491}, {"type": "recall_at_3", "value": 53.215999999999994}, {"type": "recall_at_5", "value": 59.214}, {"type": "map_at_1", "value": 22.167}, {"type": "map_at_10", "value": 29.543999999999997}, {"type": "map_at_100", "value": 30.579}, {"type": "map_at_1000", "value": 30.669999999999998}, {"type": "map_at_3", "value": 26.982}, {"type": "map_at_5", "value": 28.474}, {"type": "mrr_at_1", "value": 24.068}, {"type": "mrr_at_10", "value": 31.237}, {"type": "mrr_at_100", "value": 32.222}, {"type": "mrr_at_1000", "value": 32.292}, {"type": "mrr_at_3", "value": 28.776000000000003}, {"type": "mrr_at_5", "value": 30.233999999999998}, {"type": "ndcg_at_1", "value": 24.068}, {"type": "ndcg_at_10", "value": 33.973}, {"type": "ndcg_at_100", "value": 39.135}, {"type": "ndcg_at_1000", "value": 41.443999999999996}, {"type": "ndcg_at_3", "value": 29.018}, {"type": "ndcg_at_5", "value": 31.558999999999997}, {"type": "precision_at_1", "value": 24.068}, {"type": "precision_at_10", "value": 5.299}, {"type": "precision_at_100", "value": 0.823}, {"type": "precision_at_1000", "value": 0.106}, {"type": "precision_at_3", "value": 12.166}, {"type": "precision_at_5", "value": 8.767999999999999}, {"type": "recall_at_1", "value": 22.167}, {"type": "recall_at_10", "value": 46.115}, {"type": "recall_at_100", "value": 69.867}, {"type": "recall_at_1000", "value": 87.234}, {"type": "recall_at_3", "value": 32.798}, {"type": "recall_at_5", "value": 38.951}, {"type": "map_at_1", "value": 12.033000000000001}, {"type": "map_at_10", "value": 19.314}, {"type": "map_at_100", "value": 20.562}, {"type": "map_at_1000", "value": 20.695}, {"type": "map_at_3", "value": 16.946}, {"type": "map_at_5", "value": 18.076999999999998}, {"type": "mrr_at_1", "value": 14.801}, {"type": "mrr_at_10", "value": 22.74}, {"type": "mrr_at_100", "value": 23.876}, {"type": "mrr_at_1000", "value": 23.949}, {"type": "mrr_at_3", "value": 20.211000000000002}, {"type": "mrr_at_5", "value": 21.573}, {"type": "ndcg_at_1", "value": 14.801}, {"type": "ndcg_at_10", "value": 24.038}, {"type": "ndcg_at_100", "value": 30.186}, {"type": "ndcg_at_1000", "value": 33.321}, {"type": "ndcg_at_3", "value": 19.431}, {"type": "ndcg_at_5", "value": 21.34}, {"type": "precision_at_1", "value": 14.801}, {"type": "precision_at_10", "value": 4.776}, {"type": "precision_at_100", "value": 0.897}, {"type": "precision_at_1000", "value": 0.133}, {"type": "precision_at_3", "value": 9.66}, {"type": "precision_at_5", "value": 7.239}, {"type": "recall_at_1", "value": 12.033000000000001}, {"type": "recall_at_10", "value": 35.098}, {"type": "recall_at_100", "value": 62.175000000000004}, {"type": "recall_at_1000", "value": 84.17099999999999}, {"type": "recall_at_3", "value": 22.61}, {"type": "recall_at_5", "value": 27.278999999999996}, {"type": "map_at_1", "value": 26.651000000000003}, {"type": "map_at_10", "value": 36.901}, {"type": "map_at_100", "value": 38.249}, {"type": "map_at_1000", "value": 38.361000000000004}, {"type": "map_at_3", "value": 33.891}, {"type": "map_at_5", "value": 35.439}, {"type": "mrr_at_1", "value": 32.724}, {"type": "mrr_at_10", "value": 42.504}, {"type": "mrr_at_100", "value": 43.391999999999996}, {"type": "mrr_at_1000", "value": 43.436}, {"type": "mrr_at_3", "value": 39.989999999999995}, {"type": "mrr_at_5", "value": 41.347}, {"type": "ndcg_at_1", "value": 32.724}, {"type": "ndcg_at_10", "value": 43.007}, {"type": "ndcg_at_100", "value": 48.601}, {"type": "ndcg_at_1000", "value": 50.697}, {"type": "ndcg_at_3", "value": 37.99}, {"type": "ndcg_at_5", "value": 40.083999999999996}, {"type": "precision_at_1", "value": 32.724}, {"type": "precision_at_10", "value": 7.872999999999999}, {"type": "precision_at_100", "value": 1.247}, {"type": "precision_at_1000", "value": 0.16199999999999998}, {"type": "precision_at_3", "value": 18.062}, {"type": "precision_at_5", "value": 12.666}, {"type": "recall_at_1", "value": 26.651000000000003}, {"type": "recall_at_10", "value": 55.674}, {"type": "recall_at_100", "value": 78.904}, {"type": "recall_at_1000", "value": 92.55799999999999}, {"type": "recall_at_3", "value": 41.36}, {"type": "recall_at_5", "value": 46.983999999999995}, {"type": "map_at_1", "value": 22.589000000000002}, {"type": "map_at_10", "value": 32.244}, {"type": "map_at_100", "value": 33.46}, {"type": "map_at_1000", "value": 33.593}, {"type": "map_at_3", "value": 29.21}, {"type": "map_at_5", "value": 31.019999999999996}, {"type": "mrr_at_1", "value": 28.425}, {"type": "mrr_at_10", "value": 37.282}, {"type": "mrr_at_100", "value": 38.187}, {"type": "mrr_at_1000", "value": 38.248}, {"type": "mrr_at_3", "value": 34.684}, {"type": "mrr_at_5", "value": 36.123}, {"type": "ndcg_at_1", "value": 28.425}, {"type": "ndcg_at_10", "value": 37.942}, {"type": "ndcg_at_100", "value": 43.443}, {"type": "ndcg_at_1000", "value": 45.995999999999995}, {"type": "ndcg_at_3", "value": 32.873999999999995}, {"type": "ndcg_at_5", "value": 35.325}, {"type": "precision_at_1", "value": 28.425}, {"type": "precision_at_10", "value": 7.1}, {"type": "precision_at_100", "value": 1.166}, {"type": "precision_at_1000", "value": 0.158}, {"type": "precision_at_3", "value": 16.02}, {"type": "precision_at_5", "value": 11.644}, {"type": "recall_at_1", "value": 22.589000000000002}, {"type": "recall_at_10", "value": 50.03999999999999}, {"type": "recall_at_100", "value": 73.973}, {"type": "recall_at_1000", "value": 91.128}, {"type": "recall_at_3", "value": 35.882999999999996}, {"type": "recall_at_5", "value": 42.187999999999995}, {"type": "map_at_1", "value": 23.190833333333334}, {"type": "map_at_10", "value": 31.504916666666666}, {"type": "map_at_100", "value": 32.64908333333334}, {"type": "map_at_1000", "value": 32.77075}, {"type": "map_at_3", "value": 28.82575}, {"type": "map_at_5", "value": 30.2755}, {"type": "mrr_at_1", "value": 27.427499999999995}, {"type": "mrr_at_10", "value": 35.36483333333334}, {"type": "mrr_at_100", "value": 36.23441666666666}, {"type": "mrr_at_1000", "value": 36.297583333333336}, {"type": "mrr_at_3", "value": 32.97966666666667}, {"type": "mrr_at_5", "value": 34.294583333333335}, {"type": "ndcg_at_1", "value": 27.427499999999995}, {"type": "ndcg_at_10", "value": 36.53358333333333}, {"type": "ndcg_at_100", "value": 41.64508333333333}, {"type": "ndcg_at_1000", "value": 44.14499999999999}, {"type": "ndcg_at_3", "value": 31.88908333333333}, {"type": "ndcg_at_5", "value": 33.98433333333333}, {"type": "precision_at_1", "value": 27.427499999999995}, {"type": "precision_at_10", "value": 6.481083333333333}, {"type": "precision_at_100", "value": 1.0610833333333334}, {"type": "precision_at_1000", "value": 0.14691666666666667}, {"type": "precision_at_3", "value": 14.656749999999999}, {"type": "precision_at_5", "value": 10.493583333333332}, {"type": "recall_at_1", "value": 23.190833333333334}, {"type": "recall_at_10", "value": 47.65175}, {"type": "recall_at_100", "value": 70.41016666666667}, {"type": "recall_at_1000", "value": 87.82708333333332}, {"type": "recall_at_3", "value": 34.637583333333325}, {"type": "recall_at_5", "value": 40.05008333333333}, {"type": "map_at_1", "value": 20.409}, {"type": "map_at_10", "value": 26.794}, {"type": "map_at_100", "value": 27.682000000000002}, {"type": "map_at_1000", "value": 27.783}, {"type": "map_at_3", "value": 24.461}, {"type": "map_at_5", "value": 25.668000000000003}, {"type": "mrr_at_1", "value": 22.853}, {"type": "mrr_at_10", "value": 29.296}, {"type": "mrr_at_100", "value": 30.103}, {"type": "mrr_at_1000", "value": 30.179000000000002}, {"type": "mrr_at_3", "value": 27.173000000000002}, {"type": "mrr_at_5", "value": 28.223}, {"type": "ndcg_at_1", "value": 22.853}, {"type": "ndcg_at_10", "value": 31.007}, {"type": "ndcg_at_100", "value": 35.581}, {"type": "ndcg_at_1000", "value": 38.147}, {"type": "ndcg_at_3", "value": 26.590999999999998}, {"type": "ndcg_at_5", "value": 28.43}, {"type": "precision_at_1", "value": 22.853}, {"type": "precision_at_10", "value": 5.031}, {"type": "precision_at_100", "value": 0.7939999999999999}, {"type": "precision_at_1000", "value": 0.11}, {"type": "precision_at_3", "value": 11.401}, {"type": "precision_at_5", "value": 8.16}, {"type": "recall_at_1", "value": 20.409}, {"type": "recall_at_10", "value": 41.766}, {"type": "recall_at_100", "value": 62.964}, {"type": "recall_at_1000", "value": 81.682}, {"type": "recall_at_3", "value": 29.281000000000002}, {"type": "recall_at_5", "value": 33.83}, {"type": "map_at_1", "value": 14.549000000000001}, {"type": "map_at_10", "value": 20.315}, {"type": "map_at_100", "value": 21.301000000000002}, {"type": "map_at_1000", "value": 21.425}, {"type": "map_at_3", "value": 18.132}, {"type": "map_at_5", "value": 19.429}, {"type": "mrr_at_1", "value": 17.86}, {"type": "mrr_at_10", "value": 23.860999999999997}, {"type": "mrr_at_100", "value": 24.737000000000002}, {"type": "mrr_at_1000", "value": 24.82}, {"type": "mrr_at_3", "value": 21.685}, {"type": "mrr_at_5", "value": 23.008}, {"type": "ndcg_at_1", "value": 17.86}, {"type": "ndcg_at_10", "value": 24.396}, {"type": "ndcg_at_100", "value": 29.328}, {"type": "ndcg_at_1000", "value": 32.486}, {"type": "ndcg_at_3", "value": 20.375}, {"type": "ndcg_at_5", "value": 22.411}, {"type": "precision_at_1", "value": 17.86}, {"type": "precision_at_10", "value": 4.47}, {"type": "precision_at_100", "value": 0.8099999999999999}, {"type": "precision_at_1000", "value": 0.125}, {"type": "precision_at_3", "value": 9.475}, {"type": "precision_at_5", "value": 7.170999999999999}, {"type": "recall_at_1", "value": 14.549000000000001}, {"type": "recall_at_10", "value": 33.365}, {"type": "recall_at_100", "value": 55.797}, {"type": "recall_at_1000", "value": 78.632}, {"type": "recall_at_3", "value": 22.229}, {"type": "recall_at_5", "value": 27.339000000000002}, {"type": "map_at_1", "value": 23.286}, {"type": "map_at_10", "value": 30.728}, {"type": "map_at_100", "value": 31.840000000000003}, {"type": "map_at_1000", "value": 31.953}, {"type": "map_at_3", "value": 28.302}, {"type": "map_at_5", "value": 29.615000000000002}, {"type": "mrr_at_1", "value": 27.239}, {"type": "mrr_at_10", "value": 34.408}, {"type": "mrr_at_100", "value": 35.335}, {"type": "mrr_at_1000", "value": 35.405}, {"type": "mrr_at_3", "value": 32.151999999999994}, {"type": "mrr_at_5", "value": 33.355000000000004}, {"type": "ndcg_at_1", "value": 27.239}, {"type": "ndcg_at_10", "value": 35.324}, {"type": "ndcg_at_100", "value": 40.866}, {"type": "ndcg_at_1000", "value": 43.584}, {"type": "ndcg_at_3", "value": 30.898999999999997}, {"type": "ndcg_at_5", "value": 32.812999999999995}, {"type": "precision_at_1", "value": 27.239}, {"type": "precision_at_10", "value": 5.896}, {"type": "precision_at_100", "value": 0.979}, {"type": "precision_at_1000", "value": 0.133}, {"type": "precision_at_3", "value": 13.713000000000001}, {"type": "precision_at_5", "value": 9.683}, {"type": "recall_at_1", "value": 23.286}, {"type": "recall_at_10", "value": 45.711}, {"type": "recall_at_100", "value": 70.611}, {"type": "recall_at_1000", "value": 90.029}, {"type": "recall_at_3", "value": 33.615}, {"type": "recall_at_5", "value": 38.41}, {"type": "map_at_1", "value": 23.962}, {"type": "map_at_10", "value": 31.942999999999998}, {"type": "map_at_100", "value": 33.384}, {"type": "map_at_1000", "value": 33.611000000000004}, {"type": "map_at_3", "value": 29.243000000000002}, {"type": "map_at_5", "value": 30.446}, {"type": "mrr_at_1", "value": 28.458}, {"type": "mrr_at_10", "value": 36.157000000000004}, {"type": "mrr_at_100", "value": 37.092999999999996}, {"type": "mrr_at_1000", "value": 37.163000000000004}, {"type": "mrr_at_3", "value": 33.86}, {"type": "mrr_at_5", "value": 35.086}, {"type": "ndcg_at_1", "value": 28.458}, {"type": "ndcg_at_10", "value": 37.201}, {"type": "ndcg_at_100", "value": 42.591}, {"type": "ndcg_at_1000", "value": 45.539}, {"type": "ndcg_at_3", "value": 32.889}, {"type": "ndcg_at_5", "value": 34.483000000000004}, {"type": "precision_at_1", "value": 28.458}, {"type": "precision_at_10", "value": 7.332}, {"type": "precision_at_100", "value": 1.437}, {"type": "precision_at_1000", "value": 0.233}, {"type": "precision_at_3", "value": 15.547}, {"type": "precision_at_5", "value": 11.146}, {"type": "recall_at_1", "value": 23.962}, {"type": "recall_at_10", "value": 46.751}, {"type": "recall_at_100", "value": 71.626}, {"type": "recall_at_1000", "value": 90.93900000000001}, {"type": "recall_at_3", "value": 34.138000000000005}, {"type": "recall_at_5", "value": 38.673}, {"type": "map_at_1", "value": 18.555}, {"type": "map_at_10", "value": 24.759}, {"type": "map_at_100", "value": 25.732}, {"type": "map_at_1000", "value": 25.846999999999998}, {"type": "map_at_3", "value": 22.646}, {"type": "map_at_5", "value": 23.791999999999998}, {"type": "mrr_at_1", "value": 20.148}, {"type": "mrr_at_10", "value": 26.695999999999998}, {"type": "mrr_at_100", "value": 27.605}, {"type": "mrr_at_1000", "value": 27.695999999999998}, {"type": "mrr_at_3", "value": 24.522}, {"type": "mrr_at_5", "value": 25.715}, {"type": "ndcg_at_1", "value": 20.148}, {"type": "ndcg_at_10", "value": 28.746}, {"type": "ndcg_at_100", "value": 33.57}, {"type": "ndcg_at_1000", "value": 36.584}, {"type": "ndcg_at_3", "value": 24.532}, {"type": "ndcg_at_5", "value": 26.484}, {"type": "precision_at_1", "value": 20.148}, {"type": "precision_at_10", "value": 4.529}, {"type": "precision_at_100", "value": 0.736}, {"type": "precision_at_1000", "value": 0.108}, {"type": "precision_at_3", "value": 10.351}, {"type": "precision_at_5", "value": 7.32}, {"type": "recall_at_1", "value": 18.555}, {"type": "recall_at_10", "value": 39.275999999999996}, {"type": "recall_at_100", "value": 61.511}, {"type": "recall_at_1000", "value": 84.111}, {"type": "recall_at_3", "value": 27.778999999999996}, {"type": "recall_at_5", "value": 32.591}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB ClimateFEVER", "type": "climate-fever", "config": "default", "split": "test", "revision": "392b78eb68c07badcd7c2cd8f39af108375dfcce"}, "metrics": [{"type": "map_at_1", "value": 10.366999999999999}, {"type": "map_at_10", "value": 18.953999999999997}, {"type": "map_at_100", "value": 20.674999999999997}, {"type": "map_at_1000", "value": 20.868000000000002}, {"type": "map_at_3", "value": 15.486}, {"type": "map_at_5", "value": 17.347}, {"type": "mrr_at_1", "value": 23.257}, {"type": "mrr_at_10", "value": 35.419}, {"type": "mrr_at_100", "value": 36.361}, {"type": "mrr_at_1000", "value": 36.403}, {"type": "mrr_at_3", "value": 31.747999999999998}, {"type": "mrr_at_5", "value": 34.077}, {"type": "ndcg_at_1", "value": 23.257}, {"type": "ndcg_at_10", "value": 27.11}, {"type": "ndcg_at_100", "value": 33.981}, {"type": "ndcg_at_1000", "value": 37.444}, {"type": "ndcg_at_3", "value": 21.471999999999998}, {"type": "ndcg_at_5", "value": 23.769000000000002}, {"type": "precision_at_1", "value": 23.257}, {"type": "precision_at_10", "value": 8.704}, {"type": "precision_at_100", "value": 1.606}, {"type": "precision_at_1000", "value": 0.22499999999999998}, {"type": "precision_at_3", "value": 16.287}, {"type": "precision_at_5", "value": 13.068}, {"type": "recall_at_1", "value": 10.366999999999999}, {"type": "recall_at_10", "value": 33.706}, {"type": "recall_at_100", "value": 57.375}, {"type": "recall_at_1000", "value": 76.79}, {"type": "recall_at_3", "value": 20.18}, {"type": "recall_at_5", "value": 26.215}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB DBPedia", "type": "dbpedia-entity", "config": "default", "split": "test", "revision": "f097057d03ed98220bc7309ddb10b71a54d667d6"}, "metrics": [{"type": "map_at_1", "value": 8.246}, {"type": "map_at_10", "value": 15.979}, {"type": "map_at_100", "value": 21.025}, {"type": "map_at_1000", "value": 22.189999999999998}, {"type": "map_at_3", "value": 11.997}, {"type": "map_at_5", "value": 13.697000000000001}, {"type": "mrr_at_1", "value": 60.75000000000001}, {"type": "mrr_at_10", "value": 68.70100000000001}, {"type": "mrr_at_100", "value": 69.1}, {"type": "mrr_at_1000", "value": 69.111}, {"type": "mrr_at_3", "value": 66.583}, {"type": "mrr_at_5", "value": 67.87100000000001}, {"type": "ndcg_at_1", "value": 49.75}, {"type": "ndcg_at_10", "value": 34.702}, {"type": "ndcg_at_100", "value": 37.607}, {"type": "ndcg_at_1000", "value": 44.322}, {"type": "ndcg_at_3", "value": 39.555}, {"type": "ndcg_at_5", "value": 36.684}, {"type": "precision_at_1", "value": 60.75000000000001}, {"type": "precision_at_10", "value": 26.625}, {"type": "precision_at_100", "value": 7.969999999999999}, {"type": "precision_at_1000", "value": 1.678}, {"type": "precision_at_3", "value": 41.833}, {"type": "precision_at_5", "value": 34.5}, {"type": "recall_at_1", "value": 8.246}, {"type": "recall_at_10", "value": 20.968}, {"type": "recall_at_100", "value": 42.065000000000005}, {"type": "recall_at_1000", "value": 63.671}, {"type": "recall_at_3", "value": 13.039000000000001}, {"type": "recall_at_5", "value": 16.042}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB EmotionClassification", "type": "mteb/emotion", "config": "default", "split": "test", "revision": "829147f8f75a25f005913200eb5ed41fae320aa1"}, "metrics": [{"type": "accuracy", "value": 49.214999999999996}, {"type": "f1", "value": 44.85952451163755}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB FEVER", "type": "fever", "config": "default", "split": "test", "revision": "1429cf27e393599b8b359b9b72c666f96b2525f9"}, "metrics": [{"type": "map_at_1", "value": 56.769000000000005}, {"type": "map_at_10", "value": 67.30199999999999}, {"type": "map_at_100", "value": 67.692}, {"type": "map_at_1000", "value": 67.712}, {"type": "map_at_3", "value": 65.346}, {"type": "map_at_5", "value": 66.574}, {"type": "mrr_at_1", "value": 61.370999999999995}, {"type": "mrr_at_10", "value": 71.875}, {"type": "mrr_at_100", "value": 72.195}, {"type": "mrr_at_1000", "value": 72.206}, {"type": "mrr_at_3", "value": 70.04}, {"type": "mrr_at_5", "value": 71.224}, {"type": "ndcg_at_1", "value": 61.370999999999995}, {"type": "ndcg_at_10", "value": 72.731}, {"type": "ndcg_at_100", "value": 74.468}, {"type": "ndcg_at_1000", "value": 74.91600000000001}, {"type": "ndcg_at_3", "value": 69.077}, {"type": "ndcg_at_5", "value": 71.111}, {"type": "precision_at_1", "value": 61.370999999999995}, {"type": "precision_at_10", "value": 9.325999999999999}, {"type": "precision_at_100", "value": 1.03}, {"type": "precision_at_1000", "value": 0.108}, {"type": "precision_at_3", "value": 27.303}, {"type": "precision_at_5", "value": 17.525}, {"type": "recall_at_1", "value": 56.769000000000005}, {"type": "recall_at_10", "value": 85.06}, {"type": "recall_at_100", "value": 92.767}, {"type": "recall_at_1000", "value": 95.933}, {"type": "recall_at_3", "value": 75.131}, {"type": "recall_at_5", "value": 80.17}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB FiQA2018", "type": "fiqa", "config": "default", "split": "test", "revision": "41b686a7f28c59bcaaa5791efd47c67c8ebe28be"}, "metrics": [{"type": "map_at_1", "value": 15.753}, {"type": "map_at_10", "value": 25.875999999999998}, {"type": "map_at_100", "value": 27.415}, {"type": "map_at_1000", "value": 27.590999999999998}, {"type": "map_at_3", "value": 22.17}, {"type": "map_at_5", "value": 24.236}, {"type": "mrr_at_1", "value": 31.019000000000002}, {"type": "mrr_at_10", "value": 39.977000000000004}, {"type": "mrr_at_100", "value": 40.788999999999994}, {"type": "mrr_at_1000", "value": 40.832}, {"type": "mrr_at_3", "value": 37.088}, {"type": "mrr_at_5", "value": 38.655}, {"type": "ndcg_at_1", "value": 31.019000000000002}, {"type": "ndcg_at_10", "value": 33.286}, {"type": "ndcg_at_100", "value": 39.528999999999996}, {"type": "ndcg_at_1000", "value": 42.934}, {"type": "ndcg_at_3", "value": 29.29}, {"type": "ndcg_at_5", "value": 30.615}, {"type": "precision_at_1", "value": 31.019000000000002}, {"type": "precision_at_10", "value": 9.383}, {"type": "precision_at_100", "value": 1.6019999999999999}, {"type": "precision_at_1000", "value": 0.22200000000000003}, {"type": "precision_at_3", "value": 19.753}, {"type": "precision_at_5", "value": 14.815000000000001}, {"type": "recall_at_1", "value": 15.753}, {"type": "recall_at_10", "value": 40.896}, {"type": "recall_at_100", "value": 64.443}, {"type": "recall_at_1000", "value": 85.218}, {"type": "recall_at_3", "value": 26.526}, {"type": "recall_at_5", "value": 32.452999999999996}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB HotpotQA", "type": "hotpotqa", "config": "default", "split": "test", "revision": "766870b35a1b9ca65e67a0d1913899973551fc6c"}, "metrics": [{"type": "map_at_1", "value": 32.153999999999996}, {"type": "map_at_10", "value": 43.651}, {"type": "map_at_100", "value": 44.41}, {"type": "map_at_1000", "value": 44.487}, {"type": "map_at_3", "value": 41.239}, {"type": "map_at_5", "value": 42.659000000000006}, {"type": "mrr_at_1", "value": 64.30799999999999}, {"type": "mrr_at_10", "value": 71.22500000000001}, {"type": "mrr_at_100", "value": 71.57}, {"type": "mrr_at_1000", "value": 71.59100000000001}, {"type": "mrr_at_3", "value": 69.95}, {"type": "mrr_at_5", "value": 70.738}, {"type": "ndcg_at_1", "value": 64.30799999999999}, {"type": "ndcg_at_10", "value": 52.835}, {"type": "ndcg_at_100", "value": 55.840999999999994}, {"type": "ndcg_at_1000", "value": 57.484}, {"type": "ndcg_at_3", "value": 49.014}, {"type": "ndcg_at_5", "value": 51.01599999999999}, {"type": "precision_at_1", "value": 64.30799999999999}, {"type": "precision_at_10", "value": 10.77}, {"type": "precision_at_100", "value": 1.315}, {"type": "precision_at_1000", "value": 0.153}, {"type": "precision_at_3", "value": 30.223}, {"type": "precision_at_5", "value": 19.716}, {"type": "recall_at_1", "value": 32.153999999999996}, {"type": "recall_at_10", "value": 53.849000000000004}, {"type": "recall_at_100", "value": 65.75999999999999}, {"type": "recall_at_1000", "value": 76.705}, {"type": "recall_at_3", "value": 45.334}, {"type": "recall_at_5", "value": 49.291000000000004}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB ImdbClassification", "type": "mteb/imdb", "config": "default", "split": "test", "revision": "8d743909f834c38949e8323a8a6ce8721ea6c7f4"}, "metrics": [{"type": "accuracy", "value": 63.5316}, {"type": "ap", "value": 58.90084300359825}, {"type": "f1", "value": 63.35727889030892}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB MSMARCO", "type": "msmarco", "config": "default", "split": "validation", "revision": "e6838a846e2408f22cf5cc337ebc83e0bcf77849"}, "metrics": [{"type": "map_at_1", "value": 20.566000000000003}, {"type": "map_at_10", "value": 32.229}, {"type": "map_at_100", "value": 33.445}, {"type": "map_at_1000", "value": 33.501}, {"type": "map_at_3", "value": 28.504}, {"type": "map_at_5", "value": 30.681000000000004}, {"type": "mrr_at_1", "value": 21.218}, {"type": "mrr_at_10", "value": 32.816}, {"type": "mrr_at_100", "value": 33.986}, {"type": "mrr_at_1000", "value": 34.035}, {"type": "mrr_at_3", "value": 29.15}, {"type": "mrr_at_5", "value": 31.290000000000003}, {"type": "ndcg_at_1", "value": 21.218}, {"type": "ndcg_at_10", "value": 38.832}, {"type": "ndcg_at_100", "value": 44.743}, {"type": "ndcg_at_1000", "value": 46.138}, {"type": "ndcg_at_3", "value": 31.232}, {"type": "ndcg_at_5", "value": 35.099999999999994}, {"type": "precision_at_1", "value": 21.218}, {"type": "precision_at_10", "value": 6.186}, {"type": "precision_at_100", "value": 0.914}, {"type": "precision_at_1000", "value": 0.10300000000000001}, {"type": "precision_at_3", "value": 13.314}, {"type": "precision_at_5", "value": 9.943}, {"type": "recall_at_1", "value": 20.566000000000003}, {"type": "recall_at_10", "value": 59.192}, {"type": "recall_at_100", "value": 86.626}, {"type": "recall_at_1000", "value": 97.283}, {"type": "recall_at_3", "value": 38.492}, {"type": "recall_at_5", "value": 47.760000000000005}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB MTOPDomainClassification (en)", "type": "mteb/mtop_domain", "config": "en", "split": "test", "revision": "a7e2a951126a26fc8c6a69f835f33a346ba259e3"}, "metrics": [{"type": "accuracy", "value": 92.56269949840402}, {"type": "f1", "value": 92.1020975473988}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB MTOPIntentClassification (en)", "type": "mteb/mtop_intent", "config": "en", "split": "test", "revision": "6299947a7777084cc2d4b64235bf7190381ce755"}, "metrics": [{"type": "accuracy", "value": 71.8467852257182}, {"type": "f1", "value": 53.652719348592015}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB MassiveIntentClassification (en)", "type": "mteb/amazon_massive_intent", "config": "en", "split": "test", "revision": "072a486a144adf7f4479a4a0dddb2152e161e1ea"}, "metrics": [{"type": "accuracy", "value": 69.00806993947546}, {"type": "f1", "value": 67.41429618885515}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB MassiveScenarioClassification (en)", "type": "mteb/amazon_massive_scenario", "config": "en", "split": "test", "revision": "7d571f92784cd94a019292a1f45445077d0ef634"}, "metrics": [{"type": "accuracy", "value": 75.90114324142569}, {"type": "f1", "value": 76.25183590651454}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB MedrxivClusteringP2P", "type": "mteb/medrxiv-clustering-p2p", "config": "default", "split": "test", "revision": "dcefc037ef84348e49b0d29109e891c01067226b"}, "metrics": [{"type": "v_measure", "value": 31.350109978273395}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB MedrxivClusteringS2S", "type": "mteb/medrxiv-clustering-s2s", "config": "default", "split": "test", "revision": "3cd0e71dfbe09d4de0f9e5ecba43e7ce280959dc"}, "metrics": [{"type": "v_measure", "value": 28.768923695767327}]}, {"task": {"type": "Reranking"}, "dataset": {"name": "MTEB MindSmallReranking", "type": "mteb/mind_small", "config": "default", "split": "test", "revision": "3bdac13927fdc888b903db93b2ffdbd90b295a69"}, "metrics": [{"type": "map", "value": 31.716396735210754}, {"type": "mrr", "value": 32.88970538547634}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB NFCorpus", "type": "nfcorpus", "config": "default", "split": "test", "revision": "7eb63cc0c1eb59324d709ebed25fcab851fa7610"}, "metrics": [{"type": "map_at_1", "value": 5.604}, {"type": "map_at_10", "value": 12.379999999999999}, {"type": "map_at_100", "value": 15.791}, {"type": "map_at_1000", "value": 17.327}, {"type": "map_at_3", "value": 9.15}, {"type": "map_at_5", "value": 10.599}, {"type": "mrr_at_1", "value": 45.201}, {"type": "mrr_at_10", "value": 53.374}, {"type": "mrr_at_100", "value": 54.089}, {"type": "mrr_at_1000", "value": 54.123}, {"type": "mrr_at_3", "value": 51.44499999999999}, {"type": "mrr_at_5", "value": 52.59}, {"type": "ndcg_at_1", "value": 42.879}, {"type": "ndcg_at_10", "value": 33.891}, {"type": "ndcg_at_100", "value": 31.391999999999996}, {"type": "ndcg_at_1000", "value": 40.36}, {"type": "ndcg_at_3", "value": 39.076}, {"type": "ndcg_at_5", "value": 37.047000000000004}, {"type": "precision_at_1", "value": 44.582}, {"type": "precision_at_10", "value": 25.294}, {"type": "precision_at_100", "value": 8.285}, {"type": "precision_at_1000", "value": 2.1479999999999997}, {"type": "precision_at_3", "value": 36.120000000000005}, {"type": "precision_at_5", "value": 31.95}, {"type": "recall_at_1", "value": 5.604}, {"type": "recall_at_10", "value": 16.239}, {"type": "recall_at_100", "value": 32.16}, {"type": "recall_at_1000", "value": 64.513}, {"type": "recall_at_3", "value": 10.406}, {"type": "recall_at_5", "value": 12.684999999999999}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB NQ", "type": "nq", "config": "default", "split": "test", "revision": "6062aefc120bfe8ece5897809fb2e53bfe0d128c"}, "metrics": [{"type": "map_at_1", "value": 25.881}, {"type": "map_at_10", "value": 39.501}, {"type": "map_at_100", "value": 40.615}, {"type": "map_at_1000", "value": 40.661}, {"type": "map_at_3", "value": 35.559000000000005}, {"type": "map_at_5", "value": 37.773}, {"type": "mrr_at_1", "value": 29.229}, {"type": "mrr_at_10", "value": 41.955999999999996}, {"type": "mrr_at_100", "value": 42.86}, {"type": "mrr_at_1000", "value": 42.893}, {"type": "mrr_at_3", "value": 38.562000000000005}, {"type": "mrr_at_5", "value": 40.542}, {"type": "ndcg_at_1", "value": 29.2}, {"type": "ndcg_at_10", "value": 46.703}, {"type": "ndcg_at_100", "value": 51.644}, {"type": "ndcg_at_1000", "value": 52.771}, {"type": "ndcg_at_3", "value": 39.141999999999996}, {"type": "ndcg_at_5", "value": 42.892}, {"type": "precision_at_1", "value": 29.2}, {"type": "precision_at_10", "value": 7.920000000000001}, {"type": "precision_at_100", "value": 1.0659999999999998}, {"type": "precision_at_1000", "value": 0.117}, {"type": "precision_at_3", "value": 18.105}, {"type": "precision_at_5", "value": 13.036}, {"type": "recall_at_1", "value": 25.881}, {"type": "recall_at_10", "value": 66.266}, {"type": "recall_at_100", "value": 88.116}, {"type": "recall_at_1000", "value": 96.58200000000001}, {"type": "recall_at_3", "value": 46.526}, {"type": "recall_at_5", "value": 55.154}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB QuoraRetrieval", "type": "quora", "config": "default", "split": "test", "revision": "6205996560df11e3a3da9ab4f926788fc30a7db4"}, "metrics": [{"type": "map_at_1", "value": 67.553}, {"type": "map_at_10", "value": 81.34}, {"type": "map_at_100", "value": 82.002}, {"type": "map_at_1000", "value": 82.027}, {"type": "map_at_3", "value": 78.281}, {"type": "map_at_5", "value": 80.149}, {"type": "mrr_at_1", "value": 77.72}, {"type": "mrr_at_10", "value": 84.733}, {"type": "mrr_at_100", "value": 84.878}, {"type": "mrr_at_1000", "value": 84.879}, {"type": "mrr_at_3", "value": 83.587}, {"type": "mrr_at_5", "value": 84.32600000000001}, {"type": "ndcg_at_1", "value": 77.75}, {"type": "ndcg_at_10", "value": 85.603}, {"type": "ndcg_at_100", "value": 87.069}, {"type": "ndcg_at_1000", "value": 87.25}, {"type": "ndcg_at_3", "value": 82.303}, {"type": "ndcg_at_5", "value": 84.03699999999999}, {"type": "precision_at_1", "value": 77.75}, {"type": "precision_at_10", "value": 13.04}, {"type": "precision_at_100", "value": 1.5070000000000001}, {"type": "precision_at_1000", "value": 0.156}, {"type": "precision_at_3", "value": 35.903}, {"type": "precision_at_5", "value": 23.738}, {"type": "recall_at_1", "value": 67.553}, {"type": "recall_at_10", "value": 93.903}, {"type": "recall_at_100", "value": 99.062}, {"type": "recall_at_1000", "value": 99.935}, {"type": "recall_at_3", "value": 84.58099999999999}, {"type": "recall_at_5", "value": 89.316}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB RedditClustering", "type": "mteb/reddit-clustering", "config": "default", "split": "test", "revision": "b2805658ae38990172679479369a78b86de8c390"}, "metrics": [{"type": "v_measure", "value": 46.46887711230235}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB RedditClusteringP2P", "type": "mteb/reddit-clustering-p2p", "config": "default", "split": "test", "revision": "385e3cb46b4cfa89021f56c4380204149d0efe33"}, "metrics": [{"type": "v_measure", "value": 54.166876298246926}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB SCIDOCS", "type": "scidocs", "config": "default", "split": "test", "revision": "5c59ef3e437a0a9651c8fe6fde943e7dce59fba5"}, "metrics": [{"type": "map_at_1", "value": 4.053}, {"type": "map_at_10", "value": 9.693999999999999}, {"type": "map_at_100", "value": 11.387}, {"type": "map_at_1000", "value": 11.654}, {"type": "map_at_3", "value": 7.053}, {"type": "map_at_5", "value": 8.439}, {"type": "mrr_at_1", "value": 19.900000000000002}, {"type": "mrr_at_10", "value": 29.359}, {"type": "mrr_at_100", "value": 30.484}, {"type": "mrr_at_1000", "value": 30.553}, {"type": "mrr_at_3", "value": 26.200000000000003}, {"type": "mrr_at_5", "value": 28.115000000000002}, {"type": "ndcg_at_1", "value": 19.900000000000002}, {"type": "ndcg_at_10", "value": 16.575}, {"type": "ndcg_at_100", "value": 23.655}, {"type": "ndcg_at_1000", "value": 28.853}, {"type": "ndcg_at_3", "value": 15.848}, {"type": "ndcg_at_5", "value": 14.026}, {"type": "precision_at_1", "value": 19.900000000000002}, {"type": "precision_at_10", "value": 8.450000000000001}, {"type": "precision_at_100", "value": 1.872}, {"type": "precision_at_1000", "value": 0.313}, {"type": "precision_at_3", "value": 14.667}, {"type": "precision_at_5", "value": 12.32}, {"type": "recall_at_1", "value": 4.053}, {"type": "recall_at_10", "value": 17.169999999999998}, {"type": "recall_at_100", "value": 38.025}, {"type": "recall_at_1000", "value": 63.571999999999996}, {"type": "recall_at_3", "value": 8.903}, {"type": "recall_at_5", "value": 12.477}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB SICK-R", "type": "mteb/sickr-sts", "config": "default", "split": "test", "revision": "20a6d6f312dd54037fe07a32d58e5e168867909d"}, "metrics": [{"type": "cos_sim_pearson", "value": 77.7548748519677}, {"type": "cos_sim_spearman", "value": 68.19926431966059}, {"type": "euclidean_pearson", "value": 71.69016204991725}, {"type": "euclidean_spearman", "value": 66.98099673026834}, {"type": "manhattan_pearson", "value": 71.62994072488664}, {"type": "manhattan_spearman", "value": 67.03435950744577}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS12", "type": "mteb/sts12-sts", "config": "default", "split": "test", "revision": "fdf84275bb8ce4b49c971d02e84dd1abc677a50f"}, "metrics": [{"type": "cos_sim_pearson", "value": 75.91051402657887}, {"type": "cos_sim_spearman", "value": 66.99390786191645}, {"type": "euclidean_pearson", "value": 71.54128036454578}, {"type": "euclidean_spearman", "value": 69.25605675649068}, {"type": "manhattan_pearson", "value": 71.60981030780171}, {"type": "manhattan_spearman", "value": 69.27513670128046}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS13", "type": "mteb/sts13-sts", "config": "default", "split": "test", "revision": "1591bfcbe8c69d4bf7fe2a16e2451017832cafb9"}, "metrics": [{"type": "cos_sim_pearson", "value": 77.23835466417793}, {"type": "cos_sim_spearman", "value": 77.57623085766706}, {"type": "euclidean_pearson", "value": 77.5090992200725}, {"type": "euclidean_spearman", "value": 77.88601688144924}, {"type": "manhattan_pearson", "value": 77.39045060647423}, {"type": "manhattan_spearman", "value": 77.77552718279098}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS14", "type": "mteb/sts14-sts", "config": "default", "split": "test", "revision": "e2125984e7df8b7871f6ae9949cf6b6795e7c54b"}, "metrics": [{"type": "cos_sim_pearson", "value": 77.91692485139602}, {"type": "cos_sim_spearman", "value": 72.78258293483495}, {"type": "euclidean_pearson", "value": 74.64773017077789}, {"type": "euclidean_spearman", "value": 71.81662299104619}, {"type": "manhattan_pearson", "value": 74.71043337995533}, {"type": "manhattan_spearman", "value": 71.83960860845646}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS15", "type": "mteb/sts15-sts", "config": "default", "split": "test", "revision": "1cd7298cac12a96a373b6a2f18738bb3e739a9b6"}, "metrics": [{"type": "cos_sim_pearson", "value": 82.13422113617578}, {"type": "cos_sim_spearman", "value": 82.61707296911949}, {"type": "euclidean_pearson", "value": 81.42487480400861}, {"type": "euclidean_spearman", "value": 82.17970991273835}, {"type": "manhattan_pearson", "value": 81.41985055477845}, {"type": "manhattan_spearman", "value": 82.15823204362937}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS16", "type": "mteb/sts16-sts", "config": "default", "split": "test", "revision": "360a0b2dff98700d09e634a01e1cc1624d3e42cd"}, "metrics": [{"type": "cos_sim_pearson", "value": 79.07989542843826}, {"type": "cos_sim_spearman", "value": 80.09839524406284}, {"type": "euclidean_pearson", "value": 76.43186028364195}, {"type": "euclidean_spearman", "value": 76.76720323266471}, {"type": "manhattan_pearson", "value": 76.4674747409161}, {"type": "manhattan_spearman", "value": 76.81797407068667}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS17 (en-en)", "type": "mteb/sts17-crosslingual-sts", "config": "en-en", "split": "test", "revision": "9fc37e8c632af1c87a3d23e685d49552a02582a0"}, "metrics": [{"type": "cos_sim_pearson", "value": 87.0420983224933}, {"type": "cos_sim_spearman", "value": 87.25017540413702}, {"type": "euclidean_pearson", "value": 84.56384596473421}, {"type": "euclidean_spearman", "value": 84.72557417564886}, {"type": "manhattan_pearson", "value": 84.7329954474549}, {"type": "manhattan_spearman", "value": 84.75071371008909}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STS22 (en)", "type": "mteb/sts22-crosslingual-sts", "config": "en", "split": "test", "revision": "2de6ce8c1921b71a755b262c6b57fef195dd7906"}, "metrics": [{"type": "cos_sim_pearson", "value": 68.47031320016424}, {"type": "cos_sim_spearman", "value": 68.7486910762485}, {"type": "euclidean_pearson", "value": 71.30330985913915}, {"type": "euclidean_spearman", "value": 71.59666258520735}, {"type": "manhattan_pearson", "value": 71.4423884279027}, {"type": "manhattan_spearman", "value": 71.67460706861044}]}, {"task": {"type": "STS"}, "dataset": {"name": "MTEB STSBenchmark", "type": "mteb/stsbenchmark-sts", "config": "default", "split": "test", "revision": "8913289635987208e6e7c72789e4be2fe94b6abd"}, "metrics": [{"type": "cos_sim_pearson", "value": 80.79514366062675}, {"type": "cos_sim_spearman", "value": 79.20585637461048}, {"type": "euclidean_pearson", "value": 78.6591557395699}, {"type": "euclidean_spearman", "value": 77.86455794285718}, {"type": "manhattan_pearson", "value": 78.67754806486865}, {"type": "manhattan_spearman", "value": 77.88178687200732}]}, {"task": {"type": "Reranking"}, "dataset": {"name": "MTEB SciDocsRR", "type": "mteb/scidocs-reranking", "config": "default", "split": "test", "revision": "56a6d0140cf6356659e2a7c1413286a774468d44"}, "metrics": [{"type": "map", "value": 77.71580844366375}, {"type": "mrr", "value": 93.04215845882513}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB SciFact", "type": "scifact", "config": "default", "split": "test", "revision": "a75ae049398addde9b70f6b268875f5cbce99089"}, "metrics": [{"type": "map_at_1", "value": 56.39999999999999}, {"type": "map_at_10", "value": 65.701}, {"type": "map_at_100", "value": 66.32000000000001}, {"type": "map_at_1000", "value": 66.34100000000001}, {"type": "map_at_3", "value": 62.641999999999996}, {"type": "map_at_5", "value": 64.342}, {"type": "mrr_at_1", "value": 58.667}, {"type": "mrr_at_10", "value": 66.45299999999999}, {"type": "mrr_at_100", "value": 66.967}, {"type": "mrr_at_1000", "value": 66.988}, {"type": "mrr_at_3", "value": 64.11099999999999}, {"type": "mrr_at_5", "value": 65.411}, {"type": "ndcg_at_1", "value": 58.667}, {"type": "ndcg_at_10", "value": 70.165}, {"type": "ndcg_at_100", "value": 72.938}, {"type": "ndcg_at_1000", "value": 73.456}, {"type": "ndcg_at_3", "value": 64.79}, {"type": "ndcg_at_5", "value": 67.28}, {"type": "precision_at_1", "value": 58.667}, {"type": "precision_at_10", "value": 9.4}, {"type": "precision_at_100", "value": 1.087}, {"type": "precision_at_1000", "value": 0.11299999999999999}, {"type": "precision_at_3", "value": 24.889}, {"type": "precision_at_5", "value": 16.667}, {"type": "recall_at_1", "value": 56.39999999999999}, {"type": "recall_at_10", "value": 83.122}, {"type": "recall_at_100", "value": 95.667}, {"type": "recall_at_1000", "value": 99.667}, {"type": "recall_at_3", "value": 68.378}, {"type": "recall_at_5", "value": 74.68299999999999}]}, {"task": {"type": "PairClassification"}, "dataset": {"name": "MTEB SprintDuplicateQuestions", "type": "mteb/sprintduplicatequestions-pairclassification", "config": "default", "split": "test", "revision": "5a8256d0dff9c4bd3be3ba3e67e4e70173f802ea"}, "metrics": [{"type": "cos_sim_accuracy", "value": 99.76831683168317}, {"type": "cos_sim_ap", "value": 93.47124923047998}, {"type": "cos_sim_f1", "value": 88.06122448979592}, {"type": "cos_sim_precision", "value": 89.89583333333333}, {"type": "cos_sim_recall", "value": 86.3}, {"type": "dot_accuracy", "value": 99.57326732673268}, {"type": "dot_ap", "value": 84.06577868167207}, {"type": "dot_f1", "value": 77.82629791363416}, {"type": "dot_precision", "value": 75.58906691800189}, {"type": "dot_recall", "value": 80.2}, {"type": "euclidean_accuracy", "value": 99.74257425742574}, {"type": "euclidean_ap", "value": 92.1904681653555}, {"type": "euclidean_f1", "value": 86.74821610601427}, {"type": "euclidean_precision", "value": 88.46153846153845}, {"type": "euclidean_recall", "value": 85.1}, {"type": "manhattan_accuracy", "value": 99.74554455445545}, {"type": "manhattan_ap", "value": 92.4337790809948}, {"type": "manhattan_f1", "value": 86.86765457332653}, {"type": "manhattan_precision", "value": 88.81922675026124}, {"type": "manhattan_recall", "value": 85.0}, {"type": "max_accuracy", "value": 99.76831683168317}, {"type": "max_ap", "value": 93.47124923047998}, {"type": "max_f1", "value": 88.06122448979592}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB StackExchangeClustering", "type": "mteb/stackexchange-clustering", "config": "default", "split": "test", "revision": "70a89468f6dccacc6aa2b12a6eac54e74328f235"}, "metrics": [{"type": "v_measure", "value": 59.194098673976484}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB StackExchangeClusteringP2P", "type": "mteb/stackexchange-clustering-p2p", "config": "default", "split": "test", "revision": "d88009ab563dd0b16cfaf4436abaf97fa3550cf0"}, "metrics": [{"type": "v_measure", "value": 32.5744032578115}]}, {"task": {"type": "Reranking"}, "dataset": {"name": "MTEB StackOverflowDupQuestions", "type": "mteb/stackoverflowdupquestions-reranking", "config": "default", "split": "test", "revision": "ef807ea29a75ec4f91b50fd4191cb4ee4589a9f9"}, "metrics": [{"type": "map", "value": 49.61186384154483}, {"type": "mrr", "value": 50.55424253034547}]}, {"task": {"type": "Summarization"}, "dataset": {"name": "MTEB SummEval", "type": "mteb/summeval", "config": "default", "split": "test", "revision": "8753c2788d36c01fc6f05d03fe3f7268d63f9122"}, "metrics": [{"type": "cos_sim_pearson", "value": 30.027210161713946}, {"type": "cos_sim_spearman", "value": 31.030178065751734}, {"type": "dot_pearson", "value": 30.09179785685587}, {"type": "dot_spearman", "value": 30.408303252207812}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB TRECCOVID", "type": "trec-covid", "config": "default", "split": "test", "revision": "2c8041b2c07a79b6f7ba8fe6acc72e5d9f92d217"}, "metrics": [{"type": "map_at_1", "value": 0.22300000000000003}, {"type": "map_at_10", "value": 1.762}, {"type": "map_at_100", "value": 9.984}, {"type": "map_at_1000", "value": 24.265}, {"type": "map_at_3", "value": 0.631}, {"type": "map_at_5", "value": 0.9950000000000001}, {"type": "mrr_at_1", "value": 88.0}, {"type": "mrr_at_10", "value": 92.833}, {"type": "mrr_at_100", "value": 92.833}, {"type": "mrr_at_1000", "value": 92.833}, {"type": "mrr_at_3", "value": 92.333}, {"type": "mrr_at_5", "value": 92.833}, {"type": "ndcg_at_1", "value": 83.0}, {"type": "ndcg_at_10", "value": 75.17}, {"type": "ndcg_at_100", "value": 55.432}, {"type": "ndcg_at_1000", "value": 49.482}, {"type": "ndcg_at_3", "value": 82.184}, {"type": "ndcg_at_5", "value": 79.712}, {"type": "precision_at_1", "value": 88.0}, {"type": "precision_at_10", "value": 78.60000000000001}, {"type": "precision_at_100", "value": 56.56}, {"type": "precision_at_1000", "value": 22.334}, {"type": "precision_at_3", "value": 86.667}, {"type": "precision_at_5", "value": 83.6}, {"type": "recall_at_1", "value": 0.22300000000000003}, {"type": "recall_at_10", "value": 1.9879999999999998}, {"type": "recall_at_100", "value": 13.300999999999998}, {"type": "recall_at_1000", "value": 46.587}, {"type": "recall_at_3", "value": 0.6629999999999999}, {"type": "recall_at_5", "value": 1.079}]}, {"task": {"type": "Retrieval"}, "dataset": {"name": "MTEB Touche2020", "type": "webis-touche2020", "config": "default", "split": "test", "revision": "527b7d77e16e343303e68cb6af11d6e18b9f7b3b"}, "metrics": [{"type": "map_at_1", "value": 3.047}, {"type": "map_at_10", "value": 8.792}, {"type": "map_at_100", "value": 14.631}, {"type": "map_at_1000", "value": 16.127}, {"type": "map_at_3", "value": 4.673}, {"type": "map_at_5", "value": 5.897}, {"type": "mrr_at_1", "value": 38.775999999999996}, {"type": "mrr_at_10", "value": 49.271}, {"type": "mrr_at_100", "value": 50.181}, {"type": "mrr_at_1000", "value": 50.2}, {"type": "mrr_at_3", "value": 44.558}, {"type": "mrr_at_5", "value": 47.925000000000004}, {"type": "ndcg_at_1", "value": 35.714}, {"type": "ndcg_at_10", "value": 23.44}, {"type": "ndcg_at_100", "value": 35.345}, {"type": "ndcg_at_1000", "value": 46.495}, {"type": "ndcg_at_3", "value": 26.146}, {"type": "ndcg_at_5", "value": 24.878}, {"type": "precision_at_1", "value": 38.775999999999996}, {"type": "precision_at_10", "value": 20.816000000000003}, {"type": "precision_at_100", "value": 7.428999999999999}, {"type": "precision_at_1000", "value": 1.494}, {"type": "precision_at_3", "value": 25.85}, {"type": "precision_at_5", "value": 24.082}, {"type": "recall_at_1", "value": 3.047}, {"type": "recall_at_10", "value": 14.975}, {"type": "recall_at_100", "value": 45.943}, {"type": "recall_at_1000", "value": 80.31099999999999}, {"type": "recall_at_3", "value": 5.478000000000001}, {"type": "recall_at_5", "value": 8.294}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB ToxicConversationsClassification", "type": "mteb/toxic_conversations_50k", "config": "default", "split": "test", "revision": "edfaf9da55d3dd50d43143d90c1ac476895ae6de"}, "metrics": [{"type": "accuracy", "value": 68.84080000000002}, {"type": "ap", "value": 13.135219251019848}, {"type": "f1", "value": 52.849999421995506}]}, {"task": {"type": "Classification"}, "dataset": {"name": "MTEB TweetSentimentExtractionClassification", "type": "mteb/tweet_sentiment_extraction", "config": "default", "split": "test", "revision": "62146448f05be9e52a36b8ee9936447ea787eede"}, "metrics": [{"type": "accuracy", "value": 56.68647425014149}, {"type": "f1", "value": 56.97981427365949}]}, {"task": {"type": "Clustering"}, "dataset": {"name": "MTEB TwentyNewsgroupsClustering", "type": "mteb/twentynewsgroups-clustering", "config": "default", "split": "test", "revision": "091a54f9a36281ce7d6590ec8c75dd485e7e01d4"}, "metrics": [{"type": "v_measure", "value": 40.8911707239219}]}, {"task": {"type": "PairClassification"}, "dataset": {"name": "MTEB TwitterSemEval2015", "type": "mteb/twittersemeval2015-pairclassification", "config": "default", "split": "test", "revision": "70970daeab8776df92f5ea462b6173c0b46fd2d1"}, "metrics": [{"type": "cos_sim_accuracy", "value": 83.04226023722954}, {"type": "cos_sim_ap", "value": 63.681339908301325}, {"type": "cos_sim_f1", "value": 60.349184470480125}, {"type": "cos_sim_precision", "value": 53.437754271765655}, {"type": "cos_sim_recall", "value": 69.31398416886545}, {"type": "dot_accuracy", "value": 81.46271681468677}, {"type": "dot_ap", "value": 57.78072296265885}, {"type": "dot_f1", "value": 56.28769265132901}, {"type": "dot_precision", "value": 48.7993803253292}, {"type": "dot_recall", "value": 66.49076517150397}, {"type": "euclidean_accuracy", "value": 82.16606067830959}, {"type": "euclidean_ap", "value": 59.974530371203514}, {"type": "euclidean_f1", "value": 56.856023506366306}, {"type": "euclidean_precision", "value": 53.037916857012334}, {"type": "euclidean_recall", "value": 61.2664907651715}, {"type": "manhattan_accuracy", "value": 82.16606067830959}, {"type": "manhattan_ap", "value": 59.98962379571767}, {"type": "manhattan_f1", "value": 56.98153158451947}, {"type": "manhattan_precision", "value": 51.41158989598811}, {"type": "manhattan_recall", "value": 63.90501319261214}, {"type": "max_accuracy", "value": 83.04226023722954}, {"type": "max_ap", "value": 63.681339908301325}, {"type": "max_f1", "value": 60.349184470480125}]}, {"task": {"type": "PairClassification"}, "dataset": {"name": "MTEB TwitterURLCorpus", "type": "mteb/twitterurlcorpus-pairclassification", "config": "default", "split": "test", "revision": "8b6510b0b1fa4e4c4f879467980e9be563ec1cdf"}, "metrics": [{"type": "cos_sim_accuracy", "value": 88.56871191834517}, {"type": "cos_sim_ap", "value": 84.80240716354544}, {"type": "cos_sim_f1", "value": 77.07765285922385}, {"type": "cos_sim_precision", "value": 74.84947406601378}, {"type": "cos_sim_recall", "value": 79.44256236526024}, {"type": "dot_accuracy", "value": 86.00923662048356}, {"type": "dot_ap", "value": 78.6556459012073}, {"type": "dot_f1", "value": 72.7583749109052}, {"type": "dot_precision", "value": 67.72823779193206}, {"type": "dot_recall", "value": 78.59562673236834}, {"type": "euclidean_accuracy", "value": 87.84103698529127}, {"type": "euclidean_ap", "value": 83.50424424952834}, {"type": "euclidean_f1", "value": 75.74496544549307}, {"type": "euclidean_precision", "value": 73.19402556369381}, {"type": "euclidean_recall", "value": 78.48013550970127}, {"type": "manhattan_accuracy", "value": 87.9225365777933}, {"type": "manhattan_ap", "value": 83.49479248597825}, {"type": "manhattan_f1", "value": 75.67748162447101}, {"type": "manhattan_precision", "value": 73.06810035842294}, {"type": "manhattan_recall", "value": 78.48013550970127}, {"type": "max_accuracy", "value": 88.56871191834517}, {"type": "max_ap", "value": 84.80240716354544}, {"type": "max_f1", "value": 77.07765285922385}]}]}]}
sentence-transformers pytorch gpt_neo feature-extraction sentence-similarity mteb arxiv:2202.08904 model-index endpoints_compatible region:us
TAGS
#sentence-transformers #pytorch #gpt_neo #feature-extraction #sentence-similarity #mteb #arxiv-2202.08904 #model-index #endpoints_compatible #region-us 


# SGPT-2.7B-weightedmean-msmarco-specb-bitfit

## Usage

For usage instructions, refer to our codebase: URL 

## Evaluation Results

For eval results, refer to the eval folder or our paper: URL

## Training
The model was trained with the parameters:

DataLoader:

'URL.dataloader.DataLoader' of length 124796 with parameters:


Loss:

'sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss' with parameters:
  

Parameters of the fit()-Method:



## Full Model Architecture


## Citing & Authors


# SGPT-2.7B-weightedmean-msmarco-specb-bitfit ## Usage

For usage instructions, refer to our codebase: URL ## Evaluation Results

For eval results, refer to the eval folder or our paper: URL ## Training
The model was trained with the parameters:

DataLoader:

'URL.dataloader.DataLoader' of length 124796 with parameters:


Loss:

'sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss' with parameters:
  

Parameters of the fit()-Method: ## Full Model Architecture ## Citing & Authors
TAGS
#sentence-transformers #pytorch #gpt_neo #feature-extraction #sentence-similarity #mteb #arxiv-2202.08904 #model-index #endpoints_compatible #region-us 
 # SGPT-2.7B-weightedmean-msmarco-specb-bitfit ## Usage

For usage instructions, refer to our codebase: URL ## Evaluation Results

For eval results, refer to the eval folder or our paper: URL ## Training
The model was trained with the parameters:

DataLoader:

'URL.dataloader.DataLoader' of length 124796 with parameters:


Loss:

'sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss' with parameters:
  

Parameters of the fit()-Method: ## Full Model Architecture ## Citing & Authors
passage: TAGS
#sentence-transformers #pytorch #gpt_neo #feature-extraction #sentence-similarity #mteb #arxiv-2202.08904 #model-index #endpoints_compatible #region-us 
# SGPT-2.7B-weightedmean-msmarco-specb-bitfit## Usage

For usage instructions, refer to our codebase: URL## Evaluation Results

For eval results, refer to the eval folder or our paper: URL## Training
The model was trained with the parameters:

DataLoader:

'URL.dataloader.DataLoader' of length 124796 with parameters:


Loss:

'sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss' with parameters:
  

Parameters of the fit()-Method:## Full Model Architecture## Citing & Authors