Do miners have to hold all the blocks in memory? Running a miner node, one would have to verify transactions. This includes checking all the inputs of all the transactions.

Does this mean that a miner must hold all the block-chain in memory (currently about 8.7 GB)? If not, how can a miner find previous transaction (a transaction input does not have an indication to the block containing it)?
No, miners do not need the entire block chain to be accessible. Technically, they don't even need it at all. The blocks themself are only needed for rescanning wallets, reorganisations, and serving blocks to other nodes. That is why pruning them away will likely become viable in the future.

What you need for validating blocks and transactions (a fundamental demand from miners), is the database of unspent transaction outputs. 0.8+ clients maintain this database explicitly in a compact form (around 200 MB now, as of May 2013).

In addition to that, a miner needs to maintain a memory pool: a set of transactions that are validated against the current best chain, to choose transactions for the next candidate block from.