Saibo-backup commited on
Commit
2f4db23
·
1 Parent(s): 4d5cbc9

put on bold up to truncation

Browse files
Files changed (1) hide show
  1. app.py +5 -2
app.py CHANGED
@@ -72,15 +72,18 @@ if __name__ == "__main__":
72
  gr.Markdown(
73
  """
74
  # 👻 Transformers-CFG JSON Demo
75
- This is a demo of how you can constrain the output of a GPT-2 model to be a valid JSON string(up to truncation).
76
  Here we use a simple JSON grammar to constrain the output of the model.
77
- The grammar is defined in `json_minimal.ebnf` and is written in the Extended Backus-Naur Form (EBNF).
78
 
79
  Internally, it relies on the library [`transformers-cfg`](https://github.com/epfl-dlab/transformers-CFG).
80
  For demo purpose, gpt2-large is used, but you can use much larger models for better performance.
81
 
82
  The inference is a bit slow because of the inference is run on **CPU(~20s for 30 tokens)**.
83
  The constraint itself **doesn't** introduce significant overhead to the inference.
 
 
 
84
  """
85
  )
86
 
 
72
  gr.Markdown(
73
  """
74
  # 👻 Transformers-CFG JSON Demo
75
+ This is a demo of how you can constrain the output of a GPT-2 model to be a *valid* JSON string(*up to max length truncation*).
76
  Here we use a simple JSON grammar to constrain the output of the model.
77
+ The grammar is defined in `json_minimal.ebnf` and is written in the **Extended Backus-Naur Form (EBNF)**.
78
 
79
  Internally, it relies on the library [`transformers-cfg`](https://github.com/epfl-dlab/transformers-CFG).
80
  For demo purpose, gpt2-large is used, but you can use much larger models for better performance.
81
 
82
  The inference is a bit slow because of the inference is run on **CPU(~20s for 30 tokens)**.
83
  The constraint itself **doesn't** introduce significant overhead to the inference.
84
+
85
+ The output may be *truncated* to 30 tokens due to the limitation of the maximum length of the output.
86
+ In practice, with a decent `max_length` parameter, your JSON output will be *complete* and *valid*.
87
  """
88
  )
89