AVR ADC performance: interrupts vs. manual conversion
What you're probably measuring is the overhead of entering and exiting the ISR in software. Because you haven't decorated the ISR it is saving the entire register state and restoring it on return. Your ISR can be declared to be naked and you can manage the preservation of the SREG and other critical registers yourself (e.g. in inline assembly) and that will get you where you want to be.

You don't really want to be calling functions from the ISR either. Just move bytes into a global buffer and do stuff with the buffered data from your main loop (like send them out over the UART).

From the avr-gcc manual:


  define ISR_NAKED 
  
  ISR is created with no prologue or epilogue code. The user code is responsible for preservation of the machine state
  including the SREG register, as well as placing a reti() at the end of
  the interrupt routine.