Common anode 7 segment display
This is how the output of most logic chips works.  When they are high, they are connected to the 5 volt supply, and when they are low, they are connected to ground.  You saw this yourself; you just didn't believe it.  

Equally important is why the LED is off when the output is high.  When the cathode and anode are both at 5 volts, there is no voltage difference between them, and the LED does not light.  Many beginners have trouble understanding the nature of voltage.