CNN High Variance across multiple trained models, what does it mean?
Firstly if you're doing spectrogram classification, you'd probably want to use a loss function like cross entropy. That would give you less "high-variance" like results when you're calculating your evaluation metrics. 

About your ideas:


Dropout and early stopping would be enough
Data augmentation with speech is quite helpful, and there are a few libraries to help you with volume/speed/noise augmentation. Volume/speed augmentation is actually really important and you'd want to do it anyway for decent results in production. Speech processing is sometimes a bit tricky, in the sense that what we hear may not be what ends up in the spectrogram and parameters like noise, recording setup, encoding, accent end up causing a domain-mismatch (covariate shift)
In general, a smaller model (less parameters) helps with high variance. Start with small and simple models first, make sure eval metrics look ok, and then try larger models. 
Random initialization doesn't really affect the results in a systematic way. You can indeed ensemble predictions of models trained with different initialization; that does help with variance but don't expect a lot of difference. The gain is not worth the computation cost for models applied in practice (ie not kaggle)


Other ideas:


Check that you shuffle data properly in train/test, and between folds 
Do some error analysis on the mis-predicted test samples, do they have something in common. Check the label distribution in your dev/test set.
If the test set was captured in a process different from your dev set (total blind), don't expect a robust performance. In general, speech recognition is really data hungry with a decent "generic* model needing hundreds of hours of recording from different domains (tv, news, movies, telephone, lecture, echo, etc.).