metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:35964
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: nomic-ai/nomic-embed-text-v1.5
widget:
  - source_sentence: >-
      Despite the crucial role of phosphorus in global food production, there is
      a lack of comprehensive analysis on the economic and policy aspects of
      phosphorus supply and demand, highlighting a significant knowledge gap in
      the field of natural resource economics.
    sentences:
      - >-
        The human brain is intrinsically organized into dynamic, anticorrelated
        functional networks
      - 'The story of phosphorus: Global food security and food for thought'
      - Identifying a knowledge gap in the field of study
  - source_sentence: >-
      Despite the comprehensive data sources used in this analysis, it is
      important to note that uncertainties remain in the estimation of global
      precipitation, particularly in data-sparse regions, and careful
      interpretation of the findings is advised.
    sentences:
      - >-
        The shuttle radar topography mission—a new class of digital elevation
        models acquired by spaceborne radar
      - Advising cautious interpretation of the findings
      - >-
        Global Precipitation: A 17-Year Monthly Analysis Based on Gauge
        Observations, Satellite Estimates, and Numerical Model Outputs
  - source_sentence: >-
      The study found that participants' value functions were characterized by
      loss aversion, risk aversion, and the concavity of the utility function in
      gains and the convexity in losses.
    sentences:
      - >-
        Ordered mesoporous molecular sieves synthesized by a liquid-crystal
        template mechanism
      - 'Prospect theory: An analysis of decision under risk'
      - Summarising the results section
  - source_sentence: >-
      Further research is needed to explore the potential role of individual
      amino acids in optimizing protein intake and promoting optimal health
      outcomes.
    sentences:
      - Suggestions for future work
      - Validation of a modified Early Warning Score in medical admissions
      - >-
        Dietary Reference Intakes for Energy, Carbohydrate, Fiber, Fat, Fatty
        Acids, Cholesterol, Protein and Amino Acids
  - source_sentence: >-
      The IANA Task Force (2021) builds upon previous research suggesting that
      slower gait speed is associated with increased risk of adverse outcomes in
      older adults (Levine et al., 2015; Schoenfeld et al., 2016).
    sentences:
      - >-
        Transdisciplinary research in sustainability science: practice,
        principles, and challenges
      - >-
        Gait speed at usual pace as a predictor of adverse outcomes in
        community-dwelling older people an International Academy on Nutrition
        and Aging (IANA) Task Force
      - Referring to another writer’s idea(s) or position
datasets:
  - Corran/SciTopicTriplets
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: nomic-ai/nomic-embed-text-v1.5
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: SciGen Eval Set
          type: SciGen-Eval-Set
        metrics:
          - type: cosine_accuracy@1
            value: 0.19750889679715303
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5547153024911032
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.81605871886121
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9893238434163701
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.19750889679715303
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1849051008303677
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.16321174377224199
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.098932384341637
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.19750889679715303
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5547153024911032
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.81605871886121
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9893238434163701
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5663698287874538
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.43265442297915546
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.433292401944685
            name: Cosine Map@100

nomic-ai/nomic-embed-text-v1.5

This is a sentence-transformers model finetuned from nomic-ai/nomic-embed-text-v1.5 on the sci_topic_triplets dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: nomic-ai/nomic-embed-text-v1.5
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- sci_topic_triplets
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Corran/SciTopicNomicEmbed")
# Run inference
sentences = [
    'The IANA Task Force (2021) builds upon previous research suggesting that slower gait speed is associated with increased risk of adverse outcomes in older adults (Levine et al., 2015; Schoenfeld et al., 2016).',
    'Gait speed at usual pace as a predictor of adverse outcomes in community-dwelling older people an International Academy on Nutrition and Aging (IANA) Task Force',
    'Referring to another writer’s idea(s) or position',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: SciGen-Eval-Set
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1975
cosine_accuracy@3	0.5547
cosine_accuracy@5	0.8161
cosine_accuracy@10	0.9893
cosine_precision@1	0.1975
cosine_precision@3	0.1849
cosine_precision@5	0.1632
cosine_precision@10	0.0989
cosine_recall@1	0.1975
cosine_recall@3	0.5547
cosine_recall@5	0.8161
cosine_recall@10	0.9893
cosine_ndcg@10	0.5664
cosine_mrr@10	0.4327
cosine_map@100	0.4333

Training Details

Training Dataset

sci_topic_triplets

Dataset: sci_topic_triplets at 8bf9936
Size: 35,964 training samples
Columns: query, positive, and negative

Approximate statistics based on the first 1000 samples:

	query	positive	negative
type	string	string	string
details	min: 17 tokens mean: 40.37 tokens max: 93 tokens	min: 5 tokens mean: 18.75 tokens max: 56 tokens	min: 5 tokens mean: 10.74 tokens max: 23 tokens

Samples:

query	positive	negative
`This study provides comprehensive estimates of life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death and 195 countries and territories from 1980 to 2015, allowing for a detailed understanding of global health trends and patterns over the past four decades.`	`Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: a systematic analysis for the Global Burden of Disease Study 2015`	`Explaining the significance of the current study`
`This paper explores the relationship between the expected value and the volatility of the nominal excess return on stocks using a econometric approach.`	`On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks`	`Stating the focus, aim, or argument of a short paper`
`Despite the increasing attention given to the role of audit committees and board of directors in mitigating earnings management, several studies have reported inconclusive or even negative findings.`	`Audit committee, board of director characteristics, and earnings management`	`General reference to previous research or scholarship: highlighting negative outcomes`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        384,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Evaluation Dataset

sci_topic_triplets

Dataset: sci_topic_triplets at 8bf9936
Size: 4,495 evaluation samples
Columns: query, positive, and negative

Approximate statistics based on the first 1000 samples:

	query	positive	negative
type	string	string	string
details	min: 18 tokens mean: 40.1 tokens max: 87 tokens	min: 5 tokens mean: 18.75 tokens max: 58 tokens	min: 5 tokens mean: 10.74 tokens max: 23 tokens

Samples:

query	positive	negative
`In this cluster-randomised controlled trial, the authors aimed to evaluate the effectiveness of introducing the Medical Emergency Team (MET) system in reducing response times and improving patient outcomes in emergency departments.`	`Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial`	`Some ways of introducing quotations`
`In the data collection phase of our study, we employed both surveys and interviews as research methods. Specifically, we administered surveys to 200 participants and conducted interviews with 10 key industry experts to gather proportional data on various aspects of management science practices.`	`Research Methodology: A Step-by-Step Guide for Beginners`	`Surveys and interviews: Reporting proportions`
`Several density functional theory (DFT) based chemical reactivity indexes, such as the Fukui functions and the electrophilic and nucleophilic indices, are discussed in detail for their ability to predict chemical reactivity.`	`Chemical reactivity indexes in density functional theory`	`General comments on the relevant literature`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        384,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 256
per_device_eval_batch_size: 256
learning_rate: 2e-05
num_train_epochs: 10
warmup_ratio: 0.1
fp16: True
load_best_model_at_end: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 256
per_device_eval_batch_size: 256
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	SciGen-Eval-Set_cosine_ndcg@10
0	0	-	-	0.5454
0.1418	20	4.4872	3.1379	0.5468
0.2837	40	2.241	1.7162	0.5497
0.4255	60	1.5937	1.4834	0.5524
0.5674	80	1.5356	1.3911	0.5541
0.7092	100	1.4106	1.3277	0.5549
0.8511	120	1.2612	1.2919	0.5561
0.9929	140	1.3147	1.2642	0.5572
1.1348	160	1.1527	1.2529	0.5582
1.2766	180	1.2103	1.2388	0.5593
1.4184	200	1.2407	1.2235	0.5598
1.5603	220	1.1356	1.2101	0.5607
1.7021	240	1.1644	1.1938	0.5605
1.8440	260	1.1927	1.1864	0.5612
1.9858	280	1.1909	1.1800	0.5613
2.1277	300	1.0549	1.1785	0.5620
2.2695	320	1.0745	1.1755	0.5630
2.4113	340	1.1485	1.1656	0.5637
2.5532	360	1.1159	1.1654	0.5637
2.6950	380	1.0686	1.1623	0.5640
2.8369	400	1.1436	1.1594	0.5632
2.9787	420	1.0899	1.1534	0.5644
3.1206	440	1.0756	1.1512	0.5647
3.2624	460	1.0203	1.1536	0.5645
3.4043	480	1.1073	1.1564	0.5650
3.5461	500	1.0423	1.1594	0.5651
3.6879	520	1.069	1.1514	0.5652
3.8298	540	1.0101	1.1538	0.5645
3.9716	560	1.0685	1.1647	0.5650
4.1135	580	1.0326	1.1618	0.5653
4.2553	600	1.0729	1.1587	0.5648
4.3972	620	1.0417	1.1515	0.5655
4.5390	640	1.0438	1.1528	0.5657
4.6809	660	1.025	1.1433	0.5660
4.8227	680	1.0526	1.1382	0.5662
4.9645	700	1.0485	1.1392	0.5663
5.1064	720	1.0348	1.1411	0.5665
5.2482	740	1.1001	1.1511	0.5663
5.3901	760	1.0926	1.1625	0.5662
5.5319	780	1.0885	1.1487	0.5662
5.6738	800	1.0942	1.1492	0.5665
5.8156	820	1.0457	1.1465	0.5666
5.9574	840	1.0479	1.1461	0.5664

Framework Versions

Python: 3.11.11
Sentence Transformers: 3.3.1
Transformers: 4.47.1
PyTorch: 2.5.1+cu124
Accelerate: 1.2.1
Datasets: 3.2.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}