TriState Connection to Atmega328 and FT232R
As far as I can see, with the OE# pin held high, the buffer outputs should be tri-stated and not cause the problem you are seeing. So I cannot help you with that.

HOWEVER, I see other problems when the buffer is enabled. First, one of your buffers is reversed! BUF2 is feeding to the ATMEGA TX pin which is also an output. In fact, you don't need BUF2 (there). you can just feed the TX pin to both the FT232R and the external device's RX pins.

Second, You need another buffer to tri-state the TX output of the FT232R. Otherwise, when BUF1 is enabled, it will fight with it. This buffer needs the OE# pin inverted.







simulate this circuit &ndash; Schematic created using CircuitLab

Note that the NOT gate is not required as a discrete part. Instead, you could use two GPIOs, one to enable each buffer. Just make sure that you done enable both at once.