Replication Broken, List Out of Sync Tables In a master-slave setup on MySQL 5.5 on RHEL 6.1, when db replication breaks (for whatever reason), what is a good way to:


check (and alert) replication is broken?
get a list of tables that are out of synch across various databases?


I don't have access to server logs, but can query the tables under "mysql" db and "information_schema", in addition to the dbs used by the applications.
With regard to the first point


  check (and alert) replication is broken?


just check the following

MYSQL_CONN="-uroot -ppassword"
SLAVESTATUS_OUTPUT=/tmp/ShowSlaveStatus.txt
mysql ${MYSQL_CONN} -ANe"SHOW SLAVE STATUS\G" &gt; ${SLAVESTATUS_OUTPUT}
IO_RUNNING=`grep "Slave_IO_Running" &lt; ${SLAVESTATUS_OUTPUT} | awk '{print $2}'`
SQL_RUNNING=`grep "Slave_SQL_Running" &lt; ${SLAVESTATUS_OUTPUT} | awk '{print $2}'`


Just check for both IO_RUNNING and SQL_RUNNING being Yes.

If either value is No, check the error message.

You could check for other things such as Seconds_Behind_Master ( Does `Seconds_Behind_Master` show exact Slave Lag from Master? )

With regard to the second point


  get a list of tables that are out of synch across various databases?


The best tool in the world for this is pt-table-checksum. Please read the documentation on it because the output will be all the tables in the specified databases. You will have to parse the text output to compare the checksum values between Master and Slave.

I wrote a part post about using this tool:


In a MySQL Master/Slave replication, what would happen if I write to the Slave?
Fixing tables out of sync as reported by pt-table-checksum, pt-table-sync is not working (This one has shell scripting  I made to do the parsing for you)