Ground Loop RS232 between PC and Breadboard
A ground is, in your case, the reference point in an electrical circuit from which voltages are measured. This is defined as 0V but in reality there is no absolute 0V. It is relative. You don't have +5V you have +5V relative to the ground of a certain system.

If you now have two grounds they will represent two different 0 because you generated it with two different power supplies. They each represent 0 but in reality the two grounds can be really different or just a bit. It is this difference that can harm and form a ground loop as you will have a electrical flow. 

If your system works then you are lucky but to make sure that there is no problem I would use a optical isolated RS-232. Those a RS-232 cables designed specifically to be optical isolated and thus you can work with two different grounds.