What would cause a delay when using serial between the Arduino IDE & external clients?
If your data input rate is higher than, or even close to, your serial output rate (baudRate/10 char/s), your system will be unable to keep up.

With an interrupt driven system, make sure your interrupt function executes and returns as quickly as possible, i.e., does as little as needed to not lose data. Let the background process (the main sketch code) deal with the incoming data as it appears. Make sure the both processes are short enough to deal with the worst-case input-data bursts without losing any. For the background process, use as high a baud rate as it is capable of - and as it's downstream neighbor (Storage device? Another MCU or PC?) is capable of - keeping up with.