I have some things to add regarding your solutions
- As you have mentioned, you can put all the models in the same container and switch between models in the same container. However, you could run into the issue of conflicts, due to the possibility of requiring different versions of the same library.
- Having the scheduler load and unload different containers is definitely feasible.

If you want to be able to run multiple tasks at once where possible, such as when the combined required resources of multiple models are less than what the hardware can handle. Then you will likely need to implement a queue system and a way of checking if multiple tasks in the queue can be run simultaneously. If you implement this, you may also want to implement a priority system, so that important tasks can be run alone to ensure they are run quickly.

There are also preexisting docker orchestration that may prove helpful.
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence