How to update Bitcoin client? Today I downloaded and installed the new version of Bitcoin (0.5.0)
I extracted the file and it created a folder. I followed the instructions and clicked bitcoin-qt.exe and the program started. No installation!

Is the new version a portable version?

What should I do with the old one? Because when I click it, it opens the old version?
Should I simply uninstall it? Or should I backup the wallet from the old version?
It is always advised to backup your wallet.dat before doing anything with the client, and also to keep a general backup just in case.

As the file saving protocol in Bitcoin client rarely changes, one can possibly run a couple different versions on the same .dat files without an issue. However, running an older version after an update is not advised, as unexpected behaviour can happen every now and then.

So you should backup your wallet.dat, uninstall the old version, install the new version, check if everything is working okay, and then delete the temporary .dat backup.