Streaming support / batch inference?
#2
by
brainofchild
- opened
For production purposes, these are important. Was curious how the model handles output streaming / batch inference.
For production purposes, these are important. Was curious how the model handles output streaming / batch inference.