What happens when the mirroring server goes offline (for a short while)? Let's say we have a principal server, a mirroring server (the one that serves as the failover backup) and a witness server. 

Does it matter if I were to bring the mirroring server (not the principal) offline for a short while (five minutes, let's say). Is this going to break the mirror? Is there anything I need to do (such as turning the mirroring off temporarily on the mirrored databases) before I take the server offline?

There are plenty of resources online to explain what needs to happen when the principal goes down, but nothing I can find about this scenario.
I'd do a set partner suspend, then take the mirror down.

Some additional links:

Mirror backup &amp; restore

Mirror backup script