File size: 2,773 Bytes
67036b4 6c324ca 3fd2054 acf4948 e50b169 acf4948 a2ba71d acf4948 a698425 acf4948 a698425 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
---
language:
- en
pipeline_tag: sentence-similarity
---
# Model Card for gowitheflow/LASER-cubed-bert-base-unsup
Official model checkpoints of **LA(SER)<sup>3</sup>** (LASER-cubed) from EMNLP 2023 paper "Length is a Curse and a Blessing for Document-level Semantics"
### Model Summary
LASER-cubed-bert-base-unsup is an **unsupervised** model trained on wiki1M dataset. Without needing the training sets to have long texts, it provides surprising generalizability on long document retrieval.
- **Developed by:** Chenghao Xiao, Yizhi Li, G Thomas Hudson, Chenghua Lin, Noura Al-Moubayed
- **Shared by:** Chenghao Xiao
- **Model type:** BERT-base
- **Language(s) (NLP):** English
- **Finetuned from model:** BERT-base-uncased
### Model Sources
- **Github Repo:** https://github.com/gowitheflow-1998/LA-SER-cubed
- **Paper:** https://aclanthology.org/2023.emnlp-main.86/
### Usage
Use the model with Sentence Transformers:
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("gowitheflow/LASER-cubed-bert-base-unsup")
text = "LASER-cubed is a dope model - It generalizes to long texts without needing the training sets to have long texts."
representation = model.encode(text)
```
### Evaluation
Evaluate it with the BEIR framework:
```python
from beir.retrieval import models
from beir.datasets.data_loader import GenericDataLoader
from beir.retrieval.evaluation import EvaluateRetrieval
from beir.retrieval.search.dense import DenseRetrievalExactSearch as DRES
# download the datasets with BEIR original repo youself first
data_path = './datasets/arguana'
corpus, queries, qrels = GenericDataLoader(data_folder=data_path).load(split="test")
model = DRES(models.SentenceBERT("gowitheflow/LASER-cubed-bert-base-unsup"), batch_size=512)
retriever = EvaluateRetrieval(model, score_function="cos_sim")
results = retriever.retrieve(corpus, queries)
ndcg, _map, recall, precision = retriever.evaluate(qrels, results, retriever.k_values)
```
### Downstream Use
Information Retrieval
### Out-of-Scope Use
The model is not for further fine-tuning to do other tasks (such as classification), as it's trained to do representation tasks with similarity matching.
## Training Details
max seq 256, batch size 128, lr 3e-05, 1 epoch, 10% warmup, 1 A100.
### Training Data
wiki 1M
### Training Procedure
Please refer to the paper.
## Evaluation
### Results
**BibTeX:**
```bibtex
@inproceedings{xiao2023length,
title={Length is a Curse and a Blessing for Document-level Semantics},
author={Xiao, Chenghao and Li, Yizhi and Hudson, G and Lin, Chenghua and Al Moubayed, Noura},
booktitle={Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
pages={1385--1396},
year={2023}
}
``` |