
# T5-deshuffle  

Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language
However BOW is a lossy compression that eliminates a very important feature of text: order

This model is trained to learn the most probable order of an unordered token sequence,
using a subset of the c4 dataset, and can thus be seen as a "bag-of-words decoder".

Currently, it does not perform well. I'm planning to re-train on a larger subset of c4 later (after may).

How to run:
```python
from transformers import T5ForConditionalGeneration, T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("marksverdhei/t5-deshuffle")
model = T5ForConditionalGeneration.from_pretrained("marksverdhei/t5-deshuffle")

prompt = ' brown dog fox jumped lazy over quick the the '

ids = tokenizer(prompt, return_tensors="pt").input_ids
generated_tokens, = model.generate(ids)
print(tokenizer.decode(generated_tokens, skip_special_tokens=True))
```
{"language": "en", "datasets": ["stas/c4-en-10k"], "widget": [{"text": " brown dog fox jumped lazy over quick the the "}]}
transformers pytorch safetensors t5 text2text-generation en dataset:stas/c4-en-10k autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #safetensors #t5 #text2text-generation #en #dataset-stas/c4-en-10k #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


# T5-deshuffle  

Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language
However BOW is a lossy compression that eliminates a very important feature of text: order

This model is trained to learn the most probable order of an unordered token sequence,
using a subset of the c4 dataset, and can thus be seen as a "bag-of-words decoder".

Currently, it does not perform well. I'm planning to re-train on a larger subset of c4 later (after may).

How to run:

# T5-deshuffle  

Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language
However BOW is a lossy compression that eliminates a very important feature of text: order

This model is trained to learn the most probable order of an unordered token sequence,
using a subset of the c4 dataset, and can thus be seen as a "bag-of-words decoder".

Currently, it does not perform well. I'm planning to re-train on a larger subset of c4 later (after may).

How to run:
TAGS
#transformers #pytorch #safetensors #t5 #text2text-generation #en #dataset-stas/c4-en-10k #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # T5-deshuffle  

Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language
However BOW is a lossy compression that eliminates a very important feature of text: order

This model is trained to learn the most probable order of an unordered token sequence,
using a subset of the c4 dataset, and can thus be seen as a "bag-of-words decoder".

Currently, it does not perform well. I'm planning to re-train on a larger subset of c4 later (after may).

How to run:
passage: TAGS
#transformers #pytorch #safetensors #t5 #text2text-generation #en #dataset-stas/c4-en-10k #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# T5-deshuffle  

Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language
However BOW is a lossy compression that eliminates a very important feature of text: order

This model is trained to learn the most probable order of an unordered token sequence,
using a subset of the c4 dataset, and can thus be seen as a "bag-of-words decoder".

Currently, it does not perform well. I'm planning to re-train on a larger subset of c4 later (after may).

How to run: