Does our brain constantly ifft to hear?
No. Why do you think it would? First of all, the human brain works very different then any human constructed computer (to date); so the assumption that it runs mathematical "algorithms" is somewhat dicey..

Here is roughly how it works:


Sound wiggles the air drum
That vibration is transferred by the ossicles to the cochlea. The ossicles  act mainly as a mechanical transformer. 
Inside the cochlea the vibration induces a bending wave on the basilar membrane. Due to its shape certain frequencies wiggle preferably at different spots. That's how the frequency selectivity is created. In essence this is a mechanical way of doing a short term Fourier transform.
There are special type of neurons (nerve cells) called hair cells or cilia in the cochlea. When the basilar membrane wiggles it shears the hairs and induces the neurons to fire electrical impulses.
Interestingly enough about 20% of the neurons go from the brain back to the cochlea. This implements an active feedback mechanisms that greatly enhances the frequency selectivity. If that feedback loop goes out of whack you can "hear your ears ringing". That's one of the main causes of tinnitus. 
Neurons are inherently somewhat digital. They are amplitude discrete (i.e. they fire or not) but they are time continuous. The more stimulus is there, the more frequent they fire, but the size of the impulse stays always the same.
From there on upward it all stays in the neural domain. Everything that can be observed are firing patterns of active cells. Every layer of the brain is connected to the next layer by a huge number of connections which transform one firing pattern into a different one. Certain firing patterns mean that you actually "hear" something and are related to conscious perception.
The human brain has a huge number of active neurons and glia. There are more than 100 billion cells in your head that can process information. Needless to say this is a very complicated process. 
However, there is no need for the brain to reconstruct the original time domain waveform. It actually has no good way of representing a fast analog waveform. Neurons are pretty slow with a maximum firing rate of about 100 times per second. That's really not a good match for sound signals.