Will I need to keep track of my old private keys forever? Let's say someone (person A) starts using Bitcoin and in a period of time eagerly publishes some of their randomly generated (eg. by the Satoshi client) receiving Bitcoin addresses on formus, sends some other addresses to some people etc. 

The old keys are now associated with person A, and in a couple of years, some people might want to send person A some funds; they might find one of these old addresses (on an old forum post or in an old private Bitcoin address book) and use them.

But by now, A has moved on to using another wallet instead of the original Satoshi client, and moved all their funds there.

Doesn't this mean that A will forever have to keep track of all the original (non-deterministically generated) private keys of all the addresses that he might have sent, or made available, to other people, and keep importing them into new wallets whenever he moves to a new wallet system, and keep backing them up (safely), etc.? This seems very tedious and error-prone.

I know it's recommended to use a new address for each new transaction, but that's in no way enforced, and more importantly, it's out of person A's control -- it's all the other people that might want to send them money.

This problem will be less important in the future (eg. with deterministic key generation that requires only the seed to be backed up), but someone using the Satoshi client extensively and freely in the past might never be able to make a clean cut.

So is this a real problem or do I understand things incorrectly? Maybe there's a simple (and safe) workaround, or maybe this is not so cumbersome after all?
I think the core of the problem is that we call base58-encoded public key hashes "addresses", no matter whether people are willing to accept payments to them.

In my opinion, they are just identifiers for a public key - something internal in Bitcoin's technology. Humans shouldn't ever see them, unless they are debugging.

Satoshi originally didn't intend to use the current base58 addresses as "payment destination" - he thought pay-to-IP (where the public key was negotiated on the spot) would be the common way of doing transactions. The pay-to-pubkeyhash-via-P2P was only added later for cases where the receiver was offline. Pay-to-IP had several problems, and became obsolete, but it had a lot of advantages compared to what we have now. I think the solution is going back to something like that, like a pay-to-URI where again keys are negotiated at the time of the transaction. It means you're always sure the key used is still active, private keys don't need to be retained forever, no key is ever reused (better for Bitcoin's privacy model) and you instantly get support for attaching messages or refund information to transactions.

Of course, people will probably keep the private keys anyway, for example in an archive wallet like mentioned in other answers to this questions. But in my opinion, it shouldn't be a disaster when people don't.