How does a telephone hybrid transformer work?
On the other hand, I don't understand how a signal coming from the
left phone would only travel through the lower pair.

I've concluded that some of the signal from the telephone (Ron's voice) does reach the transmit amplifier (the source of Kim's voice) BUT this is just not that a big deal because, the important thing is that Kim's voice (port 2) does not get transmitted to port 3 (The receive path for Ron's voice).
Here is a better diagram showing the impedances not shown above: -

Also here is what wiki says about hybrid transformers: -


Double transformer hybrid
When both the 2-wire and the 4-wire circuits must be balanced, double
transformer hybrids are used, as shown at right. Signal into port W
splits between X and Z, but due to reversed connection to the
windings, cancel at port Y. Signal into port X goes to W and Y. But
due to reversed connection to ports W and Y, Z gets no signal. Thus
the pairs, W &amp; Y, X &amp; Z, are conjugates.

Note 1 - Port Y is the balancing impedance shown as Z\$_B\$ in the top diagram in my answer.
Note 2 - Port W is the telephone line
Note 3 - Ports X or Z are interchangeable as the transmit output or the receiver input.