|
--- |
|
language: en |
|
tags: |
|
- t5 |
|
- text2text-generation |
|
- openvino |
|
- english |
|
- query-expansion |
|
- int8 |
|
pipeline_tag: text2text-generation |
|
--- |
|
# Query Generation |
|
The T5-base model was trained on the [MS MARCO Passage Dataset](https://github.com/microsoft/MSMARCO-Passage-Ranking), which consists of about 500k real search queries from Bing together with the relevant passage. |
|
|
|
The model can be used for query expansion to learn semantic search models without requiring annotated training data: [Synthetic Query Generation](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/query_generation). |
|
|
|
|
|
## Usage |
|
|
|
```python |
|
from optimum.intel import OVModelForSeq2SeqLM |
|
from transformers import AutoTokenizer, pipeline |
|
|
|
model_id = "SteveTran/T5-small-query-expansion-Q4" |
|
model = OVModelForSeq2SeqLM.from_pretrained(model_id, use_cache=True, use_io_binding=False) |
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
|
|
instruction = "rewrite: " |
|
prompt = "Who lived longer, Nikola Tesla or Milutin Milankovic?" |
|
inputs = tokenizer( |
|
["{} {}".format(instruction, prompt)], |
|
padding=False, |
|
return_tensors="pt", |
|
) |
|
|
|
outputs = model.generate(**inputs, max_new_tokens=24, use_cache=False, temperature=0.6, do_sample=True, top_p=0.95) |
|
print("Answer: ", tokenizer.batch_decode(outputs, skip_special_tokens=True)) |
|
# Nikola Tesla vs Milutin Milankovic lifespan |
|
``` |
