How to safely connect the Arduino 5V Pin with a 5V wall wart and USB TTL converter for data transfer?
Do not tie all 5V connections together. Only GND is required to be connected since it is the common reference for all voltages; each device will receive power separately from their corresponding source. The 5V wall wart should be connected to the 5V pin of the Arduino and every device other than the USB-UART bridge.

You should be able to connect the wall wart first and then the bridge, but if you want to be extra careful you can use a 74LVC2G240 powered by the bridge's 5V output; LVC devices support Ioff, which basically means that all inputs and outputs are high-Z when no power is supplied.

Ground loops aren't usually a problem with 2-prong wall warts, but if you use a 3-prong wall wart then you should verify that its GND does not connect to earth ground; this will eliminate the possibility of a ground loop through the bridge. Star distribution will handle the rest.