Impact of replication compression & encryption on performance, traffic?
I like this presentation by Giuseppe Bianchi. Starting on page 5 it contains a desctiption of the TLS protocol - segment size, header size, HMAC overhead. As for the handshake, the impact on replication should be negligible. It will only occur on connection, and there may be a key exchange going on every hour, depending on the configuration.
As for the compression, I'd go with what the book says. Or you can try gzipping your binlog file, as the ratio will likely be application-specific. (This may not be accurate if the compression is initialized per packet, not once for the replication stream.)
On performance - type "openssl speed" in the command line, you'll get a benchmark of the throughput of different ciphers on your computer. Find the relevant ones and calculate the utilization given the throughput you expect from the replication stream.