How Can I decrypt this VNC Authentication handshake? I'm trying to analyze a mobile application and I believe it is is using DES.  Here is a sample of the authentication handshake.  

{""id":"handshake","authentication":"vnc","authentication-challenge":"A489C9790FB7C3CE2E56868C788641FC"}

{"id":"handshake","authentication-response":"D33BAD35B07EF500A82329749CEE9192"}

{"id":"handshake","authentication-passed":true}


For simplicity, the password is set to AAAAAAAA.

For authentication, it is identified as "VNC".  This leads me to believe that they are using VNC authentication, as defined by RFC 6143.

https://tools.ietf.org/html/rfc6143#section-7.2.2

The documentation states:

The client encrypts the challenge with DES, using a password supplied
by the user as the key.  To form the key, the password is truncated
to eight characters, or padded with null bytes on the right.  The
client then sends the resulting 16-byte response


So if  I understand correctly, If I were to encrypt "0xA489C9790FB7C3CE2E56868C788641FC" with the key of "AAAAAAAA" it should equal "0xD33BAD35B07EF500A82329749CEE9192".  Is that correct?

Challenge contains:

$ cat  ~/Desktop/challange | xxd -p
a489c9790fb7c3ce2e56868c788641fc


I try to run:

$ openssl des -in ~/Desktop/challange  | xxd -p
enter des-cbc encryption password:
Verifying - enter des-cbc encryption password:
53616c7465645f5f3f72b55c62bf2c9da805942f64f028b090cecc652dde
8cdade63763a253c1d33


Which doesn't prepuce what I am hoping for, which is D33BAD35B07EF500A82329749CEE9192.  What am I doing wrong? 

EDIT:

I found this online.  

http://www.vidarholen.net/contents/junk/vnc.html

I believe this might explain why I'm having issues.
challenge":"318E90D502A379A7B1583C35DB772138"}
{"id":"handshake","authentication-response":"1D214EF554123712A0DC9BCF003C27F4"}
{"id":"handshake","authentication-passed":true}


Ok, I got it.  Just answering for anyone else who stumbles here.  While not documented, RFB mirrors each bit.  Using the key of all B's, I was able to get my result, as B is a palindrome.