What does maintenance mode do?
The main functionality of maintenance mode is that it allows the site to use the theme system without requiring the database.

It does almost exactly what you had guessed, it locks out all users that do not have the 'use site in maintenance mode' permissions or administrator roles.  A site in maintenance mode will use a different template file than the live site, normally maintenance-page.tpl.php. 

Maintenance mode also kicks in in other situations:


Site installation
Updates
Any time the database fails