Audio compression with STFT Let STFT[frame, freqbin] be the short-time Fourier transform of some audio data.
(here frame is the index in time, and  freqbin is the index in frequency).

Very often, STFT[k, f0] will be close to STFT[k+1, f0]...



... then it could be interesting to store, for each frequency bin f0, only the differences :

STFT[0, f0], STFT[1, f0] - STFT[0, f0], STFT[2, f0] - STFT[1, f0], STFT[3, f0] - STFT[2, f0], etc.


(such processing is known as delta encoding) ; these differences will be small on average, and this could lead to a reduction of the average bit usage for storage of the audio data.

Can this idea be turned into an efficient audio data compression ?

(I don't speak here about psychoacoustic effects that could lead to use different levels of quantizations for each frequency / or frequency masking ; I know that such ideas are used in MP3 compression).

Edit: as mentionned by @pichenettes, the graph above showing continuity between adjacent frames is the "magnitude plot". The "phase / angle plot" is a bit less continuous :
I see two big drawbacks to your method:


The image you posted is not the STFT, but its magnitude. The difference between the magnitude of adjacent STFT frames is small; so for your scheme to work you would have to store the magnitude and phase separately. But then, there would be no compression at all on the phase data that would still represent 50% of the data and would be required for reconstructing the signal. Try plotting the phase (angle) of the STFT to see how ugly it is.
Assuming the deltas are encoded using variable-length integers or Golomb codes (I assume that's the idea behind it...), your method would compress large and temporally stable values in the spectrum (the red "lines" on the spectrogram image). It's obvious that these represent only a small fraction of the values. There would be no gain for the background "noise" which amounts for a very large portion of the spectrogram magnitude data.


A much more efficient way of getting rid of these stable/predictable peaks in the spectrogram is to whiten the signal and code the residual+the filter coefficients. This is (among other things) the principle of the FLAC lossless codec.