How can I push multiple containers, created with docker-compose, to a registry
Just make sure the images you build all have the right names in terms of registry prefix and so on. If not you need to rename them accordingly.

Otherwise this task is pretty easy. To push all Images you build using a docker-compose file just use docker-compose push.

Make sure you are authorised to push to the registry (logged in etc.)