DH and PKI for KeyExchange
Key exchange is notoriously hard to get right, and I strongly recommend not to do your own (unless your security requirements are really minimal). For example, what you propose does not provide forward security, which is generally considered of great importance in key exchange protocols. The good news is that there is a paper doing exactly what you need; look here. I haven't looked at the details of it, but by the abstract it sounds very efficient (albeit, not as efficient as your proposal, but giving you provable security with properties like forward security as well).