pytorch zh transformers license:apache-2.0
---
language: 
  - zh
license: apache-2.0
---

# Abstract 
This is a Chinese transformer-xl model trained on [Wudao dataset](https://resource.wudaoai.cn/home?ind&name=WuDaoCorpora%202.0&id=1394901288847716352) 
and finetuned on a denoise dataset constructed by our team. The denoise task is to reconstruct a fluent and clean text from a noisy input which includes random insertion/swap/deletion/replacement/sentence reordering.

## Usage 

### load model
```python 
from fengshen.models.transfo_xl_denoise.tokenization_transfo_xl_denoise import TransfoXLDenoiseTokenizer
from fengshen.models.transfo_xl_denoise.modeling_transfo_xl_denoise import TransfoXLDenoiseModel

tokenizer = TransfoXLDenoiseTokenizer.from_pretrained('IDEA-CCNL/Bigan-Transformer-XL-denoise-1.1B')
model = TransfoXLDenoiseModel.from_pretrained('IDEA-CCNL/Bigan-Transformer-XL-denoise-1.1B')
```

### generation
```python 
from fengshen.models.transfo_xl_denoise.generate import denoise_generate
input_text = "凡是有成就的人, 都很严肃地对待生命自己的"
res = denoise_generate(model, tokenizer,  input_text)
print(res) # "有成就的人都很严肃地对待自己的生命。"
```

## Citation
If you find the resource is useful, please cite the following website in your paper.
```
@misc{Fengshenbang-LM,
  title={Fengshenbang-LM},
  author={IDEA-CCNL},
  year={2022},
  howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}
```