Where the apps from google play get downloaded? If I am installing an application in windows, I will be downloading the exe and clicking the exe will install the application. Even if I uninstall the application, I can reinstall the application from the downloaded exe.

Now, if I am downloading an app from play store, it is downloading and installing. Will the downloaded apps will be saved in some other space and then installed, if yes where will be the application while downloading? I know the installed application will be in data/app/. I am asking while downloading, and what happens once installation, will the downloaded app be deleted.
In older Android OS versions the apk files downloaded by Google Play Store were usually stored in either /cache/download or /data/local directories.

Now the temporary location is determined by the Download Provider service, and is usually found in /data/data/com.android.providers.downloads/cache directory.  Again, this may be different for various Android OS versions.  It can also vary if manufacturers modified the OS (e.g. Samsung's TouchWiz or HTC's Sense.)