Is it right to connect Serial module directly?
Maybe the serial module has a serial cable attached to it which gives a voltage.

I think it can hurt, since you have two voltage sources, one from the Arduino and one from the Serial cable. Not using the power wire from the serial cable should fix the problem.

A resistor will not help because you still have two power sources in case you have both voltage from the Arduino (adapter/USB) and serial.