What's the fastest way to periodically communicate via serial? I will be using an Arduino as a bridge between a computer high-level Java client and some low-level hardware computing units (e.g. other Arduinos). I need to find a solution to periodically communicate in both ways. It should be possible to send digital, analog inputs to the PC and receive digital, analog outputs from the PC via serial.

I have very specific requirements for my project to work: 


Lowest possible latency in both ways. High latency would cause
instability of controlled systems.
Extremely constant, possibly user-defined update period. Varying
sample rate would cause inaccuracies in the control.


I need a very fast solution possible for update frequency of 100 Hz and higher.

So far I have tried to send data via Firmata to my java client using time interrupt.

ISR(TIMER1_COMPA_vect) {
  Firmata.sendAnalog(analog, analogRead(analog));
}


The messages however don't come at a very constant rate and are sometimes held up for twice the sample period or more which is unnaceptable. For higher frequencies this happens even more often. I suspect some kind of a buffer. Do you have any idea where should I look for bottlenecks? Would it be benefiting to design my own communication protocol and ditch Firmata?

EDIT: Look at the attached diagram . Blue lines are "inputs", green lines "outputs". Right now I am trying to design the "Arduino (Bridge)"
Lowest possible latency in both ways. High latency would cause instability of controlled systems.


the fact that you are using a not-real-time OS introduce a lot of unpredictable latency, the use of a visrtual machine over that os then add a bit more.
Fast communication can be obtained increasing the baudrate, that will decrease the latency between the sample and the elaboration. 
Also sending RAW data instead of string will be faster (no convertion AND less byte send), and finally decreasing the overhead caused by the protocol.

For example Firmata by default use a low baudrate AND will add a lot of overhead to send WHAT kind of operation is doing. Writing your own communication protocol is better.

For example: analogRead use 10bit, so you may send a raw int, 2 byte, 16bit, so 6 bit are lost.. or you can "compress" with a bit of bitwise operation 4 read (40 bit) into 5 byte (40bit, no bit loss) instead of 8(int size * 4)!

AnalogRead is a really slow operation by default on arduino (~200us), but can be faster if you set up a lower prescaler (attention, lower prescaler means less ADC precision. at prescaler 16 guaranteed precision is 8bit, if my memory is good)


  Extremely constant, possibly user-defined update period. Varying sample rate would cause inaccuracies in the control.


On the PC side, if you need micro/nano second precision you need a real time os, newest linux kernel add a new special schedule but i think is still hard to find documentation. If less precision is ok, just use a timer.

On the arduino side, using a hardware timer is the best way, see LeOS for a nice scheduler on arduino

Please note HardwareSerial on the arduino side use interrupt to send data, so using it inside an ISR (like you are doing now or with LeOS) is really usafe and deadlock prone.
Also ISR will "slow down" each others, as they cannnot be executed parallel; and analogRead is a realtivly slow operation..
I would use timer to set up a flag, and in the loop if i see the flag, start the analogread, Serial.write and finally Serial.flush (to be sure to not fill Serial buffer, causing a lot of trouble on the arduino reprogrammation reset)