MySQL utf8mb4 Migration - Replication Considerations
First make sure any latin1 columns have not been messed up.  Find an accented letter and do SELECT HEX(col) ... -- any accented letter should show as 2 bytes.  For example é is latin1 hex E9 and utf8mb4 hex C3A9.

If you see C383C2A9, you have "double encoding" and you need to worry about that now.

As for Master/Slave,...

If it is Master-Master (for ease of failing over), then consider doing them one at a time, thereby avoiding more than a brief downtime (to failover).  Be sure to use the SET to disable replication for the session.

Are you using ALTER TABLE ... CONVERT TO CHARACTER SET ...?  (That is probably the only valid way to be doing the conversion.)

With Master-Slave and plenty of downtime, that ALTER can simple go through replication.