Asking gpt-2 to finish sentence with huggingface transformers
I am currently generating text from left context using the example script run_generation.py of the huggingface transformers library with gpt-2:

$ python transformers/examples/run_generation.py \
  --model_type gpt2 \
  --model_name_or_path gpt2 \
  --prompt "Hi, " --length 5

=== GENERATED SEQUENCE 1 ===
Hi,  could anyone please inform me


I would like to generate short complete sentences. Is there any way to tell the model to finish a sentence before length words?



Note: I don't mind changing model, but would prefer an auto-regressive one.

Unfortunately there is no way to do so. You can set the length parameter to a greater value and then just discard the incomplete part at the end.
Even GPT3 doesn't support completing a sentence before a specific length. GPT3 support &quot;sequences&quot; though. Sequences force the model to stop when certain condition is fulfilled. You can find more information about in thi article
