What is it about our brain that breaks when we hear ourselves with a delay?
This phenomenon is called delayed auditory feedback. Why is still an open question, but two theories are:


Servomotor hypothesis: delayed auditory feedback creates an error signal, which the motor systems uses to attempt to correct the motor output.
Distruptive rhythm hypothesis: Speech is too fast for error correcting feedback (i.e. speech is feedforward), delayed auditory feedback disrupts the rhythm of motor output.