|
|
--- |
|
|
language: en |
|
|
tags: |
|
|
- gpt |
|
|
- text-generation |
|
|
license: gpl |
|
|
widget: |
|
|
- text: "Once upon a time, " |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
|
|
|
## Keras Implementation of Text generation with a miniature GPT |
|
|
|
|
|
This repo contains the model and the notebook [to this Keras example on Text generation with a miniature GPT](https://keras.io/examples/generative/text_generation_with_miniature_gpt/). |
|
|
|
|
|
Full credits to: [fchollet](https://twitter.com/fchollet) |
|
|
|
|
|
## Background Information |
|
|
This example demonstrates how to implement text generation with a miniature GPT model. The model consists of a single Transformer block with causal masking in its attention layer. |
|
|
|
|
|
## Datasets |
|
|
IMDB sentiment classification dataset for training. The model generates new movie reviews for a given prompt |
|
|
|
|
|
## How to use |
|
|
You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, I |
|
|
set seed for reproducibility: |
|
|
```python |
|
|
>>> from transformers import pipeline, set_seed |
|
|
>>> model = generation= pipeline('text-generation', model='keras-io/text-generation-miniature-gpt', tokenizer='bert-base-uncased') |
|
|
>>> set_seed(20) |
|
|
>>> generation("Once upon a time,", max_length=30, num_return_sequences=5) |
|
|
|
|
|
``` |
