MariaDB master-slave replication failover and failback
I went with master-master replication. To prevent connections to the "passive" copy of the master I used a script to delete all users (except the DBA and the replication one) which makes sure to prevent writes on the "passive" copy.

As part of the failover process the users will be transferred to the "passive" and dropped from the "active".

This basically achieves what I was looking for.