Run docker applications within a kubernetes cluster As part of my architecture, I want to create a deployment/service on my kubernetes cluster which reads message queues and open dockerized subscribers and then subsequently relay message to the original subscriber (if its an RPC pattern).

I was wondering what would be the best workflow to ensure minimal latency (esp since some workflows are RPC).


I can create kubernetes JOBs but that would mean i read logs for the output of pods which would be relayed and then subsequently delete the job . But I worry this might be an overkill
I can volume share the docker socket and my service can then run my images using docker client. But I am not if thats the right way to do so given that i would like to have proper tracking/versioning of my docker images esp since nodes on clusters are disposable
Are there existing frameworks to handle my situation.
After reading your Q&amp;A I think that you are looking for a message broker like kafka, rabbitmq or Amazon MQ. Please let me know whether this helps.