| thon | |
| from transformers import GPT2LMHeadModel, GPT2TokenizerFast | |
| device = "cuda" | |
| model_id = "openai-community/gpt2-large" | |
| model = GPT2LMHeadModel.from_pretrained(model_id).to(device) | |
| tokenizer = GPT2TokenizerFast.from_pretrained(model_id) | |
| We'll load in the WikiText-2 dataset and evaluate the perplexity using a few different sliding-window strategies. |