Will I need to keep track of my old private keys forever?
I think the core of the problem is that we call base58-encoded public key hashes "addresses", no matter whether people are willing to accept payments to them.

In my opinion, they are just identifiers for a public key - something internal in Bitcoin's technology. Humans shouldn't ever see them, unless they are debugging.

Satoshi originally didn't intend to use the current base58 addresses as "payment destination" - he thought pay-to-IP (where the public key was negotiated on the spot) would be the common way of doing transactions. The pay-to-pubkeyhash-via-P2P was only added later for cases where the receiver was offline. Pay-to-IP had several problems, and became obsolete, but it had a lot of advantages compared to what we have now. I think the solution is going back to something like that, like a pay-to-URI where again keys are negotiated at the time of the transaction. It means you're always sure the key used is still active, private keys don't need to be retained forever, no key is ever reused (better for Bitcoin's privacy model) and you instantly get support for attaching messages or refund information to transactions.

Of course, people will probably keep the private keys anyway, for example in an archive wallet like mentioned in other answers to this questions. But in my opinion, it shouldn't be a disaster when people don't.