Would a laptop charger at incorrect voltage fail to supply power?
At first, your laptop power source may be really weird one (not likely). Some chargers do provide only lower voltage, until device that's going to be charged requests higher voltage. Good thumb of rule: if there's only two connectors (typically round one, outer one is ground), then it should provide correct voltage all the time.

If that's not the case, then either your multimeter is showing wrong readings or your power adapter is broken.

You can check your multimeter with some known voltage, for example new 9V battery.

Typically laptop power adapters provide overvoltage without any load. It's not unusual to get for example 23V from 19V AC adapter, when it's not loaded. 

Your laptop charger circuit regulates whatever is coming from AC adapter to more appropriate voltage for charging battery. In the same way, battery voltage is regulated down to 3.3V, 5V and probably also 12V for computer internals (processor, memory, motherboard, display and so on).