Diffie-Hellman over several non-secured HTTP requests - will it work? This problem is about setting up cryptographic keys on an IoT device. I don't want my devices to be set up with any cryptographic keys pre-baked, but actual communication have to be based on TLS. Now I need a way to inject an SSL certificate and its corresponding private key from the device the user is using. Can this be done by using several HTTP requests to perform a Diffie-Hellman key exchange?

The protocol:


The user presses a button on the device, key setup and the unencryted HTTP channel allowed.
Client to Server: GET /setup/dh.wib.
Server to Client: DH initial parameters, and a new random session cookie.
Client to Server: POST /setup/dh.wib with client's computed DH intermediates and session cookie.
Server to Client: Server's computed DH intermediates.


Up to this point, a cryptographic key that is suitable for AES encryption should be established. Then:


Client to Server: POST /setup/auth.wib with the the device's serial number (as printed on the back of the device) encrypted using the established key, a SHA512-HMAC of the serial number using the key in plain, and the session cookie.
Server to Client: HTTP status code 204 if serial number matched with the record stored inside the device, otherwise HTTP status code 401 and the session cookie removed and invalidated.
Client to Server: POST /setup/key.wib with the generated certificate, and its corresponding key bundled encrypted using the established key, a SHA512-HMAC of the aforementioned bundle using the key in plain and the valid, authenticated session cookie.
Server to Client: If the certificate and key pair worked then HTTP 204, otherwise HTTP 400. Session cookie is invalidated no matter what here. Key setup and unencrypted HTTP access disabled here.
First, I'd like to say: Don't roll your own crypto.

Your first protocol is basically "your own crypto". The easiest / standard solution to this problem is to use TLS. You can use TLS (with server-authentication only) as a replacement for your first part. I'd recommend though, using DNSSEC / DANE as well (if possible) so you don't have to have a (complex) root-certificate store.



What you do next is up to you. I'll first analyze your proposal and then provide another.




If an attacker (within your threat model) can get hands on the
serial number he can pretend being that device and thereby may break
security guarantees, because the server doesn't know who he's
talking to.
In your third step you say that "Client to Server" sends the key. I think you meant it the other way around? Note that if you send the private key over the wire, the key is only as secure as the connection (so if you use a "bad" DH private key the key could still be compromised even though it's high entropy).


Otherwise this looks fine and pretty standard.




You generate your own private key on the device along with a CSR.
You send the CSR to your server along with some "secret" that identifies the device in question and that can not easily be retrieved for attackers in your threat model.
The server will create the certificate from the CSR and send it back to the client.
The client can now use said certificate for client-authentication.




As for the "how" of the generation of the private key:

You stated that you have a somewhat reliable entropy source on the device.
If you can guess any private key generated on the device there's no chance that you can get a private key securely to the device as the transport key could be broken (because it's weak).
Assuming you can scratch enough entropy together (i.e. 128 bits of entropy distributed over an unspecified amount of bytes) you can run your "random" bytes through a key based key derivation function (e.g. HKDF), to get a key and an IV for the stream cipher of your choice (e.g. AES-CTR or ChaCha20 or Salsa20).
You can now use this stream cipher to generate a pseudo random stream of bytes that you can use as a replacement for your system RNG to get a good private RSA key. Note that 128-bit entropy is good enough for a 3072-bit RSA key as it anyways only provides a 128-bit security margin.