Commit
·
2f4db23
1
Parent(s):
4d5cbc9
put on bold up to truncation
Browse files
app.py
CHANGED
@@ -72,15 +72,18 @@ if __name__ == "__main__":
|
|
72 |
gr.Markdown(
|
73 |
"""
|
74 |
# 👻 Transformers-CFG JSON Demo
|
75 |
-
This is a demo of how you can constrain the output of a GPT-2 model to be a valid JSON string(up to truncation).
|
76 |
Here we use a simple JSON grammar to constrain the output of the model.
|
77 |
-
The grammar is defined in `json_minimal.ebnf` and is written in the Extended Backus-Naur Form (EBNF)
|
78 |
|
79 |
Internally, it relies on the library [`transformers-cfg`](https://github.com/epfl-dlab/transformers-CFG).
|
80 |
For demo purpose, gpt2-large is used, but you can use much larger models for better performance.
|
81 |
|
82 |
The inference is a bit slow because of the inference is run on **CPU(~20s for 30 tokens)**.
|
83 |
The constraint itself **doesn't** introduce significant overhead to the inference.
|
|
|
|
|
|
|
84 |
"""
|
85 |
)
|
86 |
|
|
|
72 |
gr.Markdown(
|
73 |
"""
|
74 |
# 👻 Transformers-CFG JSON Demo
|
75 |
+
This is a demo of how you can constrain the output of a GPT-2 model to be a *valid* JSON string(*up to max length truncation*).
|
76 |
Here we use a simple JSON grammar to constrain the output of the model.
|
77 |
+
The grammar is defined in `json_minimal.ebnf` and is written in the **Extended Backus-Naur Form (EBNF)**.
|
78 |
|
79 |
Internally, it relies on the library [`transformers-cfg`](https://github.com/epfl-dlab/transformers-CFG).
|
80 |
For demo purpose, gpt2-large is used, but you can use much larger models for better performance.
|
81 |
|
82 |
The inference is a bit slow because of the inference is run on **CPU(~20s for 30 tokens)**.
|
83 |
The constraint itself **doesn't** introduce significant overhead to the inference.
|
84 |
+
|
85 |
+
The output may be *truncated* to 30 tokens due to the limitation of the maximum length of the output.
|
86 |
+
In practice, with a decent `max_length` parameter, your JSON output will be *complete* and *valid*.
|
87 |
"""
|
88 |
)
|
89 |
|