Would this authentication on a Diffie-Hellmann-key-exchanged channel work? Before I ask my question, let me say that I don't know the math behind OTR, so I don't know if the process of authentication in OTR is any similar to this.




Alice and Bob have exchanged keys using Diffie-Hellman key exchange
There's a man-in-the-middle - Eve.
This is the encryption "circuit": Alice -- e_alice -- Eve -- e_bob -- Bob
Both Alice and Bob use e to express the encryption key they use.
Thus in conditions where there is no MitM, e_alice == e_bob 
Alice wants to verify that there is no MitM, so uses a verification question (and an answer) (like OTR in XMPP):


Q: Do you have a cat?

A: Yes

Note: Obviously use strong values for A. Don't ask about the presence of a cat, but rather cat's name (ideally if it's a unique name containing numbers and special characters) :P


Alice sends Q to Bob
Bob sends hash(e_bob+A). hash is any safe hashing algorithm, e_bob is the encryption key Bob uses, and A is the answer, salt in this context.
Alice compares received == hash(e_alice+A). In case this is true, there is no MitM.




In this context:


The hashing algorithm for zero-knowledge-proof
The encryption keys are a pre-shared (by the key exchange) secret
The answer is used as salt (while being another pre-shared secret), so Eve can't think: "Oh look, that's a sha1 hash. Let's try to crack it. Oh look it's our encryption key -- that's a no-go"
Possibly add additional encoding like base64 to minimize chance of hashed contents being in wordlists. (hash(base64(e_bob+A)))




Question: Are there any serious vulnerabilities, or would this work for authentication for insecure encrypted channels?
In the question as it stands now, nothing prevents Eve to relay Do you have a cat? to Bob, get the answer A = Yes by testing A for Yes or No against hash(e_bob+A) obtained from Bob, then send hash(e_alice+A) to Alice and defeat the protocol.

In a now-gone comment, it was added that we


  should use "strong values" for A; only Q is sent, plain A is never sent.


Notice that then, in order for Alice to perform a check, A must be known to Alice in advance. If this "strong value" (or even the convention to use it the way described) is known to Alice and Bob, but not Eve, then the technique works.

If only the "strong value" is unknown to Eve, it acts as a secret key in an ad-hoc protocol (for something academic we'd want to use this "strong value" as key to HMAC of message e)

If the convention is unknown to Eve, that's not cryptography in the modern sense of Kerckhoffs's second principle; and that's brittle, and can't be generalized.