Square Wave pulses generating EMI
Difficult to assess this without going round the circuit with a scope probe looking for spikes, including on the power supply; but there are further cheap tricks that can be applied to mitigate EMI even without understanding exactly where it's coming from:


small resistors in series with TX/RX; try 33 ohm to start with, up to maybe 100 ohm. Place near the side driving that signal. This mitigates "ringing" in the line.
decorate power supplies with decoupling capacitors to ground, 1-10 uf. Place near device using power.
small ferrite beads/chokes in line with power supplies
shield the RX/TX lines not just the pickup (it doesn't mention this in the diagram)


You mention a USB cable but I can't work out where it would have gone in the diagram? USB is both balanced and properly terminated, which gives it better EMI properties.