A question about digital scope input and common mode noise
Always examine source impedance of interference at some frequency either by R, L or C and compare with the receiver impedance with some  impedance for transmission line and load.  


then examine the CM gain or attenuation ratio vs the Differential Mode (DM) gain to the load whether
the source and load impedance  whether it is 50 ohms or 10 M makes big difference and also whether the receiver is earth grounded. 
Earth Grounding both ends is a choice that depends on shared paths for stray current such as lightning or AC ground faults, but generally only the Receiver of high impedance signal measurements is earth grounded.


a high CMMR receiver cannot null a CM noise if the source impedance of that stray signal is also not balanced.  
When CM Z is not balanced such as with electrodes , active guarding is used such as for EEG, ECG signals with right leg drive obtained from the CM signal amplifier.
when it is not perfectly balanced for RF such as Ethernet up to 100M or 1GHz then a CM choke is needed, also used for TV and Radio Tx in addition to a differential transformer which is harder to balance over a wide spectrum with different winding paths for primary and secondary so this is still not sufficient for RF.
when no source is connected to coax source but terminated by 1M at DC, the coax impedance is open circuit different mode (DM) and common mode (CM) and at 50Hz has some DM impedance that is not 50 Ohms but at 100pF/m for LF it looks like ~31 Mohm DM but still fentofarads to stray power lines, so still no signal, even though not "balanced" the DM impedance but the coax is earth grounded, so this shunts the CM stray impedance by the shield capacitance like Ohm's Law at 50 Hz.
but as soon as you use pair wires now they are higher impedance ( depends on gap and turns/m) but now exposed to more chance of differential stray coupling but far better than untwisted pairs.  Shielded differential pairs are even better.



Now you have some cable that that picks up 50 Hz with high impedance input and load , so what is the differential impedance, if high then the attenuation degrades and more DM signal can occur from the stray noise inbalance and wire inbalance.
  - you will notice that all power lines around the world are rotated every km for a reason to balance the stray coupling from solar flares and magnetic storms. This is a form of balanced cables and even floating Delta transformers are not enough to reject unbalanced power lines from stray magnetic noise because the floating Delta transformers are not perfectly balanced with secondary and primary windings at a different radius. But it helps for long distance transmission.  


Once it gets to distribution transformers , the secondaries are Star-connected 3 phase with neutral earth-grounded to lower the CM impedance to 0 at the transformer.
now getting to your situation, it is unclear what you used and whether the Rx is earth grounded or not makes a big difference.  Often wire signals enter a shielded box with what are called "feedthru" caps that shunt the high impedance signal somewhat to common earth ground with the conductor thru the via, then used aferrite beads for RF and CM SMT transformer for even better balancing of differential impedance with much higher inductance than the wires and  shunt caps to earth to provide attenuation ratio of the LF signal here.  
However the differential source impedance must be much lower than these CM shunt caps to yield enough DM bandwidth.
each scenario for EM compatibility or EMC desired to maximize the signal to noise ratio (SNR) and  must consider the bandwidth, impedance Z and inbalanced Z and signal level of BOTH the CM and DM interference and the differential mode signal to determine what works best. So there are a lot of combinations to consider which is why there is no one fix for all. It is impossible for even CM chokes to work over more than a 2 or 3 decades of bandwidth effectively and sometimes 2 CM chokes may be cascaded or choosing better INstrument Amps (INA) or choosing better electrodes to lower source impedance or choosing better coax to reject UHF noise.  This subject takes at least 1 good book on EMC to discuss properly. 


When the source is floating with high CM noise, scope users know to use Differential probes or two single ended 10:1 probes with A-B display for better balanced signals and CM rejection.