Why am I measuring a voltage on the output of an audio amplifier with no input signal?
According to the schematic you included, it is an amplifier with differential output. You have to connect the loudspeaker between pins 5 and 8, not to ground. Both outputs will probably carry a signal half way between 0 and Vcc, with one output going down and the other going up in the rythm of the music. Output 5 and 8 are each others mirror.

As with the  speaker, to measure output voltage, you again have to measure between pins 5 and 8, and not with reference to ground. Now with a multimeter, this is pretty easy to accomplish, but with a scope this is probably a different story.

The problem with a scope is that the scope ground may very well be attached to the circuit ground for safety. Either through its own power supply, or the attached music player('s power supply) or any other attached device.

If this is the case, then attaching scope ground to one of the pins6 or 8 will short circuit the amplifier which is ... bad. Most oscilloscopes with at least two channels have the option to add or subtract the two input channels. Use the subtract mode to measure the voltage difference on both outputs of this amp.