This model provides a GPT-2 language model trained with SimCTG on the WritingPrompts benchmark [(Fan et al., 2018)](https://arxiv.org/abs/1805.04833) based on our paper [_A Contrastive Framework for Neural Text Generation_](https://arxiv.org/abs/2202.06417).

We provide a detailed tutorial on how to apply SimCTG and Contrastive Search in our [project repo](https://github.com/yxuansu/SimCTG#4-huggingface-style-tutorials-back-to-top). In the following, we illustrate a brief tutorial on how to use our approach to perform text generation.

## 1. Installation of SimCTG:
```yaml
pip install simctg --upgrade
```

## 2. Initialize SimCTG Model:
```python
import torch
# load SimCTG language model
from simctg.simctggpt import SimCTGGPT
model_name = r'cambridgeltl/simctg_writingprompts'
model = SimCTGGPT(model_name)
model.eval()
tokenizer = model.tokenizer
```

## 3. Prepare the Text Prefix:
```python
prefix_text = r"[ WP ] A kid doodling in a math class accidentally creates the world 's first functional magic circle in centuries . <|endoftext|>"
print ('Prefix is: {}'.format(prefix_text))
tokens = tokenizer.tokenize(prefix_text)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
input_ids = torch.LongTensor(input_ids).view(1,-1)
```

## 4. Generate Text with Contrastive Search:
```python
beam_width, alpha, decoding_len = 5, 0.6, 200
output = model.fast_contrastive_search(input_ids=input_ids, beam_width=beam_width, 
                                       alpha=alpha, decoding_len=decoding_len) 
print("Output:\n" + 100 * '-')
print(tokenizer.decode(output))
'''
  Prefix is: [ WP ] A kid doodling in a math class accidentally creates the world 's first functional magic circle in centuries . <|endoftext|>
  Output:
  ----------------------------------------------------------------------------------------------------
  [ WP ] A kid doodling in a math class accidentally creates the world's first functional magic circle in centuries. <|endoftext|> I looked at 
  the circle, it wasn't there. I couldn't see it, and my eyes were watering from the rain that had fallen over the school, the wind howling through 
  the windows and making a wispy noise as it passed through the air. `` What is it? '' I asked, trying to find the source of the noise. `` It's a 
  circle, '' the teacher said in a voice that sounded like it was from an old TV show or something like that. `` You can't make it out of there. '' 
  I looked around the room, there was no one there. It was as if I was in a dream, but no one seemed to notice me. Then I saw a flash of light, and 
  the circle appeared in front of me. I turned around to see what was going on, I had never seen anything like it before in my life. I ran up to the 
  teacher and asked, `` Are you sure this is real?
'''
```

For more details of our work, please refer to our main [project repo](https://github.com/yxuansu/SimCTG).

## 5. Citation:
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!

```bibtex
@article{su2022contrastive,
  title={A Contrastive Framework for Neural Text Generation},
  author={Su, Yixuan and Lan, Tian and Wang, Yan and Yogatama, Dani and Kong, Lingpeng and Collier, Nigel},
  journal={arXiv preprint arXiv:2202.06417},
  year={2022}
}
```



{}
transformers pytorch gpt2 text-generation arxiv:1805.04833 arxiv:2202.06417 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-generation #arxiv-1805.04833 #arxiv-2202.06417 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

This model provides a GPT-2 language model trained with SimCTG on the WritingPrompts benchmark (Fan et al., 2018) based on our paper _A Contrastive Framework for Neural Text Generation_.

We provide a detailed tutorial on how to apply SimCTG and Contrastive Search in our project repo. In the following, we illustrate a brief tutorial on how to use our approach to perform text generation.

## 1. Installation of SimCTG:


## 2. Initialize SimCTG Model:


## 3. Prepare the Text Prefix:


## 4. Generate Text with Contrastive Search:


For more details of our work, please refer to our main project repo.

## 5. Citation:
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!





## 1. Installation of SimCTG: ## 2. Initialize SimCTG Model: ## 3. Prepare the Text Prefix: ## 4. Generate Text with Contrastive Search:


For more details of our work, please refer to our main project repo. ## 5. Citation:
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!
TAGS
#transformers #pytorch #gpt2 #text-generation #arxiv-1805.04833 #arxiv-2202.06417 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ## 1. Installation of SimCTG: ## 2. Initialize SimCTG Model: ## 3. Prepare the Text Prefix: ## 4. Generate Text with Contrastive Search:


For more details of our work, please refer to our main project repo. ## 5. Citation:
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!
passage: TAGS
#transformers #pytorch #gpt2 #text-generation #arxiv-1805.04833 #arxiv-2202.06417 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
## 1. Installation of SimCTG:## 2. Initialize SimCTG Model:## 3. Prepare the Text Prefix:## 4. Generate Text with Contrastive Search:


For more details of our work, please refer to our main project repo.## 5. Citation:
If you find our paper and resources useful, please kindly leave a star and cite our paper. Thanks!