DIY over-encryption with a related key I know next to nothing about cryptography, which is bad because I'm a programmer. I was exploring some code the other day and the encryption part of it seemed a little strange to me. I'd like the name/terminology/links to this algorithm/concept so I can learn more about it, and I'd also like to know if it's secure or a waste of processing. Also please bear with me, I don't know any of your terminology u.u

The code included some stuff that sounds pretty good: it uses CBC with AES/Rijndael (using 256 bit block size) with null byte padding (the data is supposed to end with a certain character so null bytes works). The IV is randomly generated with each encryption and is then prepended to the encrypted data (second question: does this practice have a name?). But then as a wrapper around that it has another layer of "encryption"? - I'll try to explain it the best I can:

First, the key that was used in the AES/Rijndael is passed through a hash function. I'll call this the key hash? This same value is used in both encrypting and decrypting. Then the encrypted data (from the AES CBC, and with the IV in front) is broken into chunks the same length of the key hash (blocks?). Each byte of the key hash is added to the encrypted byte, and then modulus-ed(?) (if > 255 then subtract 255). Currently the hash function used is MD5, but I'd also like to know about this in a more general sense too.

Is this whole method secure? What is it called? Is it a waste of processing? Can it be done more efficiently/better? Why did the developer choose to do it like this? Etc... So many questions!

Not important but: The developer then proceeds to store/send the encrpyted data over the network as... doubly base64 encoded data, and then inside xml. e.g. xml(base64(base64(encrypted data)))... Gross!!! And the original data is so verbose and redundant too.
The key that was used in the AES/Rijndael for encryption is passed through a hash function


(strikethrough/strikeout mine)

That's OK if the hash function is secure and not vulnerable to timing attacks. Obviously MD5 is not a great option as it has been proven to be insecure. For this specific function however - similar to key derivation - it's OK. You should not be able to retrieve the key value as a hash is a one way function.


  I'll call this the key hash? This same value is used in both encrypting and decrypting. Then the encrypted data (from the AES CBC, and with the IV in front) is broken into chunks the same length of the key hash (blocks?). Each byte of the key hash is added to the encrypted byte, and then modulus-ed(?) (if > 255 then subtract 255).


This is basically a one-time pad where the key is repeated. Now normally it would be relatively easy to find the "hash" key with a known plaintext attack. With the ciphertext being properly randomized that is not possible. If however some part of the original AES/CBC ciphertext is ever leaked then the hashed key becomes known and the cipher can be reversed.

Basically this scheme doesn't add much security. It also shows a very limited understanding of cryptographic techniques and encoding methods. Because of this it should be seen as a risk, even if the scheme cannot be attacked directly. If MD5 is ever reversed you're in trouble. For now it is indeed a waste of bandwidth and processing time.

Furthermore, if you'd ever want to use hardware based encryption then the encryption key value will not be directly available (as the encryption operation is performed in the hardware itself). So it's probably not possible to perform MD5 over the key, making this scheme incompatible with hardware based security.

Finally, there seems a lot to be going on, but this scheme in all its needless complexity seems to miss an authentication tag. This means that an attacker may send you corrupted data that's still successfully decrypted. The attacker won't be able to simply inject any data, but it is possible to at least let the receiving party process garbage. In that sense the hash function could have been put to better use in a HMAC construction. In that case a SHA-2 hash such as SHA-256 should be preferred of course.