ATMEGA UART transmition with flow control
Sounds like you have two basic problems, a badly designed external device and that you are trying to make inappropriate hardware do a job it is not intended for.

First, does this other device really require absolutely no more bytes after RTS is asserted?  That would be very unusual.  It is very common for transmitters to empty a small buffer before obeying RTS, so unless the other engineer didn't know what he was doing the device should be able to tolerate a few more bytes.  Maybe it can't handle the "full featured" case of 16 more bytes, but not allowing 2-4 more bytes is just bad design.

Second, given that you need to stop transmitting faster than the size of the output buffer, you picked the wrong hardware.  I am not familiar with that line of micros, but surely this is in the datasheet.  I see three options:

Get a micro that does have a RTS input and that stops on the next byte boundary when it is asserted.  Most UARTs in micros don't have this extra feature, but there are certainly some that do.  I know some PIC 33F can do this because I've chosen them in part for this feature.  Look around the Atmel product line.  They may have a few micros with this feature too.

Use a external UART that does hardware flow control on the byte level.

Use the existing UART with more software intervention.  You won't be able to use the internal buffer, and will have to wait until one character is sent, check RTS, then write the next character to the UART.  This will be facilitated if the UART can interrupt when empty, not just when there is room in the output buffer.



None of these tradeoffs may be what you want, but that's the cost of bad architecture.  Fix it next rev, or respin now if it's really important.  The world doesn't always have a magic answer just because your boss wants it now or because you painted yourself into a corner.