Using STFT as an input to a Neural Net I'm trying to use the STFT as an input to a neural network. After flattening, there are over 4,000 features for a few seconds of audio. Is there a recommended way of summarising these to be a more manageable/useful feature set? All I can think of is using the stats like mean, variance and perhaps the higher order stats, but I can't help but feel there must be some better approach. Even with these stats, would it be better to take them per SFTF in isolation, or calculate them across all segments?
(As an aside, since this is related and someone coming across this question might also be interested, I am also looking into normalisation: https://danielsdiscoveries.wordpress.com/2017/09/29/spectrogram-input-normalisation-for-neural-networks/)
Given a $M \times N$ STFT (spectrogram), use this as the input to a convolutional neural network. Do not flatten the spectrogram. Since your spectrogram will be complex, then you can use the magnitude spectrogram or phase spectrogram or both. However, PyTorch recently released support for complex numbers, so you might be able to train the convolutional neural network using the spectrogram directly.