How to migrate huge MySQL databases one by one to new server?
I think I found a solution which is close to what I'm trying to achieve: https://dba.stackexchange.com/a/76198/107793.

The only thing I have to look into is, how to avoid the possible downtime with locking the tables. Since I'm on InnoDB exclusively (apart from the MySQL system DBs) that should be possible.