Is there a way for a single GPU and model to run deep learning model prediction/inference in parallel
If your single model uses all 8gb of RAM, then it would not be possible to run another model in parallel using the same resources. You would have to allocate more memory or schedule the second model to run afterwards.
If there's a 8G-RAM GPU, and has loaded a model that takes all the 8G RAM, is it possible to run multiple model prediction/inference in parallel?
or you can only run a prediction at a same time period