NACL with public key encryption and authentication
If I understand properly, you want to use ephemeral keys, and verify the identity of the sender.

A HMAC would not be enough to verify the identity of the sender, because the recipient is able to compute a valid authentication tag as well.

Signing the message before encrypting it solves this. Note that in that case, the message can simply be a random session key. So, for most applications, the cost is pretty low: signing is only required once, during key exchange. If this is acceptable for your application, this is probably the easiest route to go.

Otherwise, the Noise framework can be used to build secure key exchange protocols, according to what each participant knows about its peer. I would highly recommend using these patterns instead of inventing your own.

Do both the client and the server know about each other's public keys? The Noise KK pattern is likely to be the one you are interested in. Or if you have a single client, NNpsk0 is faster as it requires a single DH operation.