Difference between speech and music signal
Not really an answer, but usually when people speak there are pauses between words, sentences, thoughts, etc. Music (with multiple instruments) is fairly continuous. Even if one instruments stops another is probably still playing. So there would be less lulls in the audio stream. Just a guess though.

Another thing you could do is look at frequencies, voices are typically in the 300Hz-4kHz range (this is what telephones originally used as bandwidth) whereas musical instruments are usually much broader. A frequency outside the human range could indicate instruments  I randomly found that image here