MySQL failover - Master to Master Replication My company is trying to implement a MySQL failover mechanism, to achieve higher availability in our webservices tier - we commercialize a SaaS solution. To that end we have some low-end VMs scattered through different geographical locations, each containing a MySQL 5.5 server with several DBs, that for the time being are merely slave-replicating from the production server - the objective up until now was just checking the latency and general resilience of MySQL replication.

The plan however is to add a Master-Master replication environment between two servers in two separate locations, and these two instances would handle all the DB writes. The idea wouldn't necessarily imply concurrency; rather the intention is having a single one of the instances handling the writes, and upon a downtime situation using a DNS Failover service to direct the requests to the secondary server. After the primary comes back online, the b-log generated in the meantime in the secondary would be replicated back, and the DNS Failover restored the requests back to the first one. 

I am not an experienced administrator, so I'm asking for your own thoughts and experiences. How wrong is this train of thought? What can obviously go wrong? Are there any much better alternatives? Bash away! 

Thanks!
I'm thinking of doing exactly as you describe using Amazon's Route 53 DNS Failover.  So the idea is to setup 2 databases as master-master

db1.domain.com
db2.domain.com

Then do a Route53 "failover" set on

db.domain.com
-> primary is db1.domain.com => TCP-based health check on port 3306
-> secondary is db2.domain.com  => TCP-based health check on port 3306

I have not gotten to this point yet but that is my plan