Keeping the blockchain index after Bitcoin-Qt upgrade Can this be done or do everyone updating their client everytime a new version is announced on bitcoin.org has to re-do the whole process over again?

On OS X, (probably with the other two OS's as well) blockchain data is kept on a seperate directory in ~/Library/Application Support/Bitcoin ; however a new version of the client needs to reindex the whole thing over again- which is a super time-consuming process..
Version 0.8 of the reference client changed the database format completely, in order to scale better. That meant a one-time rebuild that everyone had to go through, as converting the old database would have been error-prone.

With that change, the transaction index was made optional, and off by default. This is because for normal operation (verifying and processing blocks and transactions, and maintaining a wallet), such an index is not necessary.

An option -txindex (or txindex=1 in bitcoin.conf) was introduced to re-enable this transaction index (which is necessary for the getrawtransaction RPC call). It has to be specified when the database is created though, so at first run of a post-0.8 version, or when explicitly rebuilding the index (with -reindex).

This type of database format change is not expected to happen frequently.