Re-installing Android Mobile phone technology has been growing rapidly. Nowadays, we have mobile phones with the same configuration as a Pentium-1,Pentium-2 processor + 1 GB Ram  20GB Hard disk as we had a few years ago for Desktops.

For eg: I have just ordered for Galaxy S-III, which as 1 GHz CPU, 1 GB RAM and 16 GB Permanent storage.

So, my question is : can we format (same thing as we do on a Desktop) such mobile devices and install whatever OS (Android only for now) we like ? If so, I would like to install latest Android version on my mobile. Similar to the CD-Rom drive that can hold the OS that we are going to install on our fresh Desktop, for mobile devices can't we use any USB storage drives that has the latest Android (installation source) ?
It depends on the device you are using. Some devices have a locked bootloader, meaning you cannot boot into a mode which allows you to overwrite the part of the persistent memory which holds the operating system image.

The steps you usually take are the following:


Flash an advanced Recovery Image
Use that recovery so you can upload an alternative Android version to your ROM


Let me clear up some of the terms:


ROM is the part that contains the operating system and cannot be written to while the device is running. You have to override it by flashing over USB or using an aforementioned Recovery Image
Recovery is an alternative mode you can boot into which enables you to create backups of your system or write to the ROM to install a different operating system


Every alternative Android version comes with tons of tutorials and help. So just search for your device model in conjunction with "cyanogenmod" or "custom ROM".

Edit:

I just saw that you have ordered a Samsung Galaxy S3, so you will have little trouble installing an alternative Android.