Self-expiring symmetric keys, or: cryptography in absence of secure deletion
I don't believe there is any way to achieve what you want, within your constraints.

You stated as a requirement that a user must be able to get all of their data from the server, without storing anything other than their private key.  If an attacker learns the private key, the attacker then knows everything that the user does, so of course the attacker can decrypt too.

A random beacon (a continuous, public stream of random numbers) doesn't seem to help, since an attacker can record a copy of that entire stream.  Also, it would rely upon on some external trusted party to generate the stream; if you are willing to trust a third party, there are simpler solutions.  An example of such a solution: store $H$ on the trusted third party and return it only upon request to users who prove knowledge of the private key, and only if the expiration date has not yet passed.

A different solution is to have the user change their private key every so often.  To ease this, they can generate a long-lived keypair which is used only for signing, and then generate short-lived keypairs every so often.  The long-lived key is used to sign each short-lived public key and publish this certificate.  To send a message to this user, you look up the current short-lived public key, check its corresponding cert, then encrypt to the short-lived public key.  The user can then securely delete their short-lived keypair when it expires. (I think you might also be able to build sophisticated schemes that eliminate the need for a lookup, but I'm not 100% certain about that.)