This is a small Russian denoising autoencoder. It can be used for restoring corrupted sentences.

This model was produced by fine-tuning the [rut5-small](https://huggingface.co/cointegrated/rut5-small) model on the task of reconstructing a sentence:
* restoring word positions (after slightly shuffling them)
* restoring dropped words and punctuation marks (after dropping some of them randomly)
* restoring inflection of words (after changing their inflection randomly using [natasha](https://github.com/natasha/natasha) and [pymorphy2](https://github.com/kmike/pymorphy2) packages)

The fine-tuning was performed on a [Leipzig web corpus](https://wortschatz.uni-leipzig.de/en/download/Russian) of Russian sentences. 

The model can be applied as follows:
```
# !pip install transformers sentencepiece
import torch
from transformers import T5ForConditionalGeneration, T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained("cointegrated/rut5-small-normalizer")
model = T5ForConditionalGeneration.from_pretrained("cointegrated/rut5-small-normalizer")

text = 'меня тобой не понимать'
inputs = tokenizer(text, return_tensors='pt')
with torch.no_grad():
    hypotheses = model.generate(
        **inputs, 
        do_sample=True, top_p=0.95, 
        num_return_sequences=5, 
        repetition_penalty=2.5,
        max_length=32,
    )
for h in hypotheses:
    print(tokenizer.decode(h, skip_special_tokens=True))
```
A possible output is:
```
# Мне тебя не понимать.
# Если бы ты понимаешь меня?
# Я с тобой не понимаю.
# Я тебя не понимаю.
# Я не понимаю о чем ты.
```
{"language": "ru", "license": "mit", "tags": ["normalization", "denoising autoencoder", "russian"], "widget": [{"text": "\u043c\u0435\u043d\u044f \u0442\u043e\u0431\u043e\u0439 \u043d\u0435 \u043f\u043e\u043d\u0438\u043c\u0430\u0442\u044c"}]}
transformers pytorch jax safetensors t5 text2text-generation normalization denoising autoencoder russian ru license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #jax #safetensors #t5 #text2text-generation #normalization #denoising autoencoder #russian #ru #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

This is a small Russian denoising autoencoder. It can be used for restoring corrupted sentences.

This model was produced by fine-tuning the rut5-small model on the task of reconstructing a sentence:
* restoring word positions (after slightly shuffling them)
* restoring dropped words and punctuation marks (after dropping some of them randomly)
* restoring inflection of words (after changing their inflection randomly using natasha and pymorphy2 packages)

The fine-tuning was performed on a Leipzig web corpus of Russian sentences. 

The model can be applied as follows:

A possible output is:

TAGS
#transformers #pytorch #jax #safetensors #t5 #text2text-generation #normalization #denoising autoencoder #russian #ru #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

passage: TAGS
#transformers #pytorch #jax #safetensors #t5 #text2text-generation #normalization #denoising autoencoder #russian #ru #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
