What exactly am I hearing from my 5DMkII audio out jack?
You may be getting just a little bit of drive from the A/V Output connection.  The 5D Mark ii has no headphone connector.  All it has is a TRRS (tip, ring, ring, sleeve) connector that happens to be the same size as a headphone plug.  It is designed to feed an analog video feed out of the camera and in to a TV.

While there would be a drastic impedance difference, the part of the headphones that happened to connect to one of the audio channels may have been sufficient to minimally drive your headphones.  The other ear piece likely got connected to the video feed which wouldn't do much except perhaps produce noise, quite probably in-audibly.

I would avoid trying it as sending the wrong kind of signal through your headphones may potentially cause damage, though you could use the A/V cable to output the audio and then wire it up to a small headphone amplifier to monitor the audio output from the A/V cable safely.

More information about the 5D Mark ii's A/V output connector can be found on page 139 of the manual.