Something I don't understand with electricity and my circuit
Just connect the usb-to-serial converter ground and the ATmega ground together. Since the two circuits are powered by two different sources there is no guarantee that their references are at the same potential, so the communication do not happen properly. When you touch the ATmega board you actually stabilize the reference in some ways so the circuit works.

Not connecting grounds together is a common mistake and apart from not working circuits can lead to more catastrophic failures. The rule is: always connect ground first, and if you want to unpower a part of your circuit just leave the ground connection there, removing only the "hot" power is faster, easier and safer.