Can I first open an account on an exchange then later on transfer to my wallet?
If I open an account on, say, mtgox, can I immediately buy bitcoins and if I can, can I then transfer the very bitcoins I just bought to my own personal wallet (the one that I'll generate later on offline and put its private part in a safe).


Yes, that's a basic Bitcoin withdrawal.


  As a side-question: where are the bitcoins that I buy on mtgox stored? In a wallet belonging to mtgox?


Yes, exactly.