Batch inference on many images
#41
by
yadavsaakash
- opened
I have 1000's of images, how do i make proper use of gpu to do batch inference and reduce my total inference time?
I'd use vllm and send a bunch of async requests so that it can batch them for you