Do you have anything to add to this problem resolving around a scheduler calling multiple dockerized clients that handle AI requests?

ML Parallelity Problem (Docker)

- some models need all the hardware
- we want the same hardware to work on multiple jobs
- -> models need to be switched

Possible Solutions

- Put all models in same container
- Scheduler sends load and unload signal to container
spam lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor creativity violence