

This is a cross-encoder model trained to predict semantic equivalence of two Russian sentences. 

It classifies text pairs as paraphrases (class 1) or non-paraphrases (class 0). Its scores can be used as a metric of content preservation for paraphrasing or text style transfer.

It is a [sberbank-ai/ruRoberta-large](https://huggingface.co/sberbank-ai/ruRoberta-large) model fine-tuned on a union of 3 datasets: 
1. `RuPAWS`: https://github.com/ivkrotova/rupaws_dataset based on Quora and QQP;
2. `ru_paraphraser`: https://huggingface.co/merionum/ru_paraphraser;
3. Results of the manual check of content preservation for the [RUSSE-2022](https://www.dialog-21.ru/media/5755/dementievadplusetal105.pdf) text detoxification dataset collection  (`content_5.tsv`). 

The task was formulated as binary classification: whether the two sentences have the same meaning (1) or different (0). 

The table shows the training dataset size after duplication (joining `text1 + text2` and `text2 + text1` pairs): 

source \ label	| 0	| 1
-- | -- | --
detox |	1412|	3843
paraphraser	|5539	| 1688
rupaws_qqp	|1112 |	792
rupaws_wiki	|3526	| 2166

The model was trained with Adam optimizer and the following hyperparameters: 

```
learning_rate = 1e-5
batch_size = 8
gradient_accumulation_steps = 4
n_epochs = 3
max_grad_norm = 1.0
```

After training, the model had the following ROC AUC scores on the test sets:
set | ROC AUC
- | -
detox         | 0.857112
paraphraser   | 0.858465
rupaws_qqp    | 0.859195
rupaws_wiki   | 0.906121

Example usage:

```Python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained('SkolkovoInstitute/ruRoberta-large-paraphrase-v1')
tokenizer = AutoTokenizer.from_pretrained('SkolkovoInstitute/ruRoberta-large-paraphrase-v1')

def get_similarity(text1, text2):
    """ Predict the probability that two Russian sentences are paraphrases of each other. """
    with torch.inference_mode():
        batch = tokenizer(
            text1, text2, 
            truncation=True, max_length=model.config.max_position_embeddings, return_tensors='pt',
        ).to(model.device)
        proba = torch.softmax(model(**batch).logits, -1)
    return proba[0][1].item()

print(get_similarity('Я тебя люблю', 'Ты мне нравишься'))  # 0.9798
print(get_similarity('Я тебя люблю', 'Я тебя ненавижу'))   # 0.0008
```
{"language": ["ru"], "tags": ["sentence-similarity", "text-classification"], "datasets": ["merionum/ru_paraphraser", "RuPAWS"]}
transformers pytorch safetensors roberta text-classification sentence-similarity ru dataset:merionum/ru_paraphraser dataset:RuPAWS autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #safetensors #roberta #text-classification #sentence-similarity #ru #dataset-merionum/ru_paraphraser #dataset-RuPAWS #autotrain_compatible #endpoints_compatible #region-us 

This is a cross-encoder model trained to predict semantic equivalence of two Russian sentences.


It classifies text pairs as paraphrases (class 1) or non-paraphrases (class 0). Its scores can be used as a metric of content preservation for paraphrasing or text style transfer.


It is a sberbank-ai/ruRoberta-large model fine-tuned on a union of 3 datasets:


1. 'RuPAWS': URL based on Quora and QQP;
2. 'ru\_paraphraser': URL
3. Results of the manual check of content preservation for the RUSSE-2022 text detoxification dataset collection ('content\_5.tsv').


The task was formulated as binary classification: whether the two sentences have the same meaning (1) or different (0).


The table shows the training dataset size after duplication (joining 'text1 + text2' and 'text2 + text1' pairs):


source \ label: detox, 0: 1412, 1: 3843
source \ label: paraphraser, 0: 5539, 1: 1688
source \ label: rupaws\_qqp, 0: 1112, 1: 792
source \ label: rupaws\_wiki, 0: 3526, 1: 2166


The model was trained with Adam optimizer and the following hyperparameters:


After training, the model had the following ROC AUC scores on the test sets:
set | ROC AUC


* | -
detox | 0.857112
paraphraser | 0.858465
rupaws\_qqp | 0.859195
rupaws\_wiki | 0.906121


Example usage:



TAGS
#transformers #pytorch #safetensors #roberta #text-classification #sentence-similarity #ru #dataset-merionum/ru_paraphraser #dataset-RuPAWS #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #safetensors #roberta #text-classification #sentence-similarity #ru #dataset-merionum/ru_paraphraser #dataset-RuPAWS #autotrain_compatible #endpoints_compatible #region-us 
