The need for a Comparator
There's a lot you're not telling us. In fact we hardly know anything:  


What's the audio signal (level, DC bias)?  
What type is the opamp?  
How is the LED connected to the opamp (to ground or Vcc)?  
What type is the comparator?  
What's the comparator's threshold?  
How's the opamp connected to the comparator (+ or - input)?  
How is the LED connected to the comparator (to ground or Vcc)?  


I'll assume the audio signal is pure AC, without DC bias. That means that you'll only amplify the positive part of the signal, so what you hear through the speaker will be very much distorted. If the input signal is at line level (about 500mV RMS) anything above 50mV will be clipped to Vcc, and you'll have little of your signal left. Add to this that a typical opamp only supplies 20mA, then you'll have 3mW in an 8\$\Omega\$ speaker. That's not much, but should be audible if the signal didn't clip like that. Anyway, if your output is most of the time at 5V a LED connected to 5V will only light very faintly.
When you add a comparator you'll be above the threshold almost all the time, so it makes sense that the LED lights up.  

If the input signal is from a microphone it will be only a few mV, amplifying 100-fold will give a few 100mV at the output. That's low enough to light the LED if it's connected to Vcc, but too low to light it if connected to ground. If your comparator threshold is low enough (0V?), again you'll be above that for much of the time, and the LED will light.  

So depending on your setting it can be explained that the LED doesn't (or hardly) light at the opamp, but does when connected to the comparator. Answer the questions and we'll be able to tell you more. In the mean time I'll vote to close the question until you're more specific.