Is it possible to find the key of an HMAC-SHA-512 given the input and the hash output? Is it possible to find the key of an HMAC-SHA-512 given the input and the hash output?


If so, what algorithm if any prevents this?

I'm working on a simple E2E communication system between two self-managed servers, and my idea is to store a static key on both servers, and it is critical that the stored key cannot be reverse-engineered from the sent requests. The requests would contain both an unencrypted and encrypted version of a string, the encrypted one being hashed with the stored key. This does sound unorthodox, but the maths checks out if the mentioned reverse-engineering operation is impossible.

EDIT: Clients will not be able to use an old authentication string to authenticate a new message.
HMAC

$\operatorname{HMAC}$, in extended form keyed-hash message authentication code or hash-based message authentication code, is a MAC that includes a cryptographic hash function and a secret key.

With these properties, it can be used for data integrity and authentication.

With the used hash function $H$, $\operatorname{HMAC}$ defined as;

\begin{align}
  \operatorname{HMAC}(K, m) &amp;= \operatorname{H}\Bigl(\bigl(K' \oplus opad\bigr) \parallel 
\operatorname{H} \bigl(\left(K' \oplus ipad\right) \parallel m\bigr)\Bigr) \\
                         K' &amp;= \begin{cases}
                                 \operatorname{H}\left(K\right) &amp; K\text{ is larger than block size} \\
                                 K                              &amp; \text{otherwise}
                                \end{cases}
\end{align}

HMAC is usually written as HM$\operatorname{HMAC-X}$ where the $\operatorname{X}$ represents the used hash function and in your case it is $\operatorname{HMAC-SHA512}$

Security;

Security of $\operatorname{HMAC}$ depends on the deployed key size and hash output. The most common attack is a brute force attack. In your case, the attacker will try all possible keys to match the input and the output. If your key size is above 128 you will be fine for single target attacks. If possible prefer larger key sizes like the 256-bit key size. With SHA-512 one can use key sizes up to 1024 since the limit is determined by the block size of the used hash function.

As proven by Bellare 2006 that if the compression is PRF than HMAC-SHA512 is a PRF. As a result, HMAC doesn't suffer the weakness found in MD5 - the MD5 collision attack. 

php hash_hmac supports various hash algorithms, the current list contains 43.



Note 1: HMAC-SHA512 uses the underlying hash function two times to countermeasure against length extension attacks. For the SHA-3 series, there is KMAC that uses SHA-3 directly and SHA-3 has resistance to length extension attacks by design.

Note 2: As of today (21/05/2020), I didn't see a PHP implementation of KMAC.