Power coming from unknown source
Most of new  ICs (analog or digital) have ESD protection diodes which are diodes connected between the input pin and the supply rails, their main job is to clamp the input pin voltage between the supply rails in the event of a voltage surge, see 

How Diode Clamping Works

these diodes have low current rating so if you continuously load them with enough current they can be damaged,

Since you haven't supplied voltage to the circuit, your circuit can be turned on
through the ESD diodes on the data lines, the simplest way to protect these diodes and also fixing your problem is placing a resistor on your digital lines,
if you don't have high baud rates 1kohm should be a good starting value