Why does MAP show ambient pressure when engine is shut off?
The intake manifold is where the air flows from outside into the engine, so obviously it must be open to the outside. So when everything is calm, the pressure quickly equalizes and the manifold pressure is equal to the ambient pressure.

The non-obvious thing is actually why it differs when the engine is running.

When the engine is running, it sucks air in. The volume of air per unit of time is proportional to its rotational speed: higher RPM means more strokes per minute and each stroke has the same displacement.

Now without any restriction in the intake manifold, the air would still enter at the ambient pressure (speaking of normally aspirated engine). And with fully open throttle, the restriction is small enough that the manifold pressure is indeed only a little below ambient.

It is only as you close the throttle that you create significant restriction, which reduces the manifold pressure.

But the reduction is still proportional to the engine RPM. With engine not running there is no suction, so the pressure can't decrease and the gauge still shows ambient pressure.

This dependence on the RPM also means that it is not actually good indicator of the power the engine produces. When you increase RPM by adjusting the mixture, the engine power will increase—because it takes more power to sustain higher RPM—but the manifold pressure will decrease—because there is more suction. The same also happens when you increase RPM by advancing propeller pitch selector.

What manifold pressure is good indicator of though, and the reason it exists, is the engine wear. The amount of air, and fuel, are proportional to the manifold pressure, and therefore so is the peek pressure when the air is compressed and the fuel burnt. And the higher the peek pressure, the more wear on the components. That is why there is absolute maximum you should never exceed and continuous power limit that you should only exceed for limited time on take-off or in emergency.

Note that the relations are further complicated in turbocharged engines where higher engine RPM also mean higher turbocharger RPM and that increases the pressure gain on the turbocharger that adds to the manifold pressure.