What are the fundamental differences between speech and music in frequency domain?
One way is to look at the energy below 80 Hz. Unless you have a recording of James Earl Jones, there will be almost none for speech and any music with bass and/or drums is going to have a lot down there.
Note that there is huge variability in music recording and a lot in speech. Creating an algorithm that can reliably distinguish between the two for the majority all recordings is a fairly complicated. Looking at the &quot;average spectrum&quot; isn't that useful since the any individual instance can (and does) look completely different than the &quot;average spectrum&quot;. This is actually a good candidate for machine learning although many deterministic algorithms do exist as well.