Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
This will always wait for 1ms even if the queue is empty, which might not be the
best since you probably want to start doing inference if there's nothing in the queue.