Encrypting twice with different modes? I was using a SOCKS5 proxy today (tunneled through SSH) and I began to wonder what the effect of encrypting twice (TLS over encrypted SOCKS5 in this case) would have on the security of the transmitted data. The proxy was using 256-bit AES in CTR mode, while the page I was viewing was using 128-bit AES in GCM mode. While I am not worried that this will be harmful at all, I am curious as to how much more difficult a scenario like this would make it for an attacker to break the encryption (assuming they could somehow bruteforce 128/256-bit keys).
When you are e.g. sending TLS encrypted data over a SSH tunnel, there are two things in particular that should be noted:


The TLS handshake will only commence, once the SSH connection has been established.
The bulk encryption keys of TLS will be completely independent of the SSH encryption keys.


Since the handshakes and keys are completely independent, there is normally no reason to worry about any bad interaction between the two layers. The TLS data is no different from any other kind of data, from the perspective of SSH. The SSH connection is no different from any other kind of serialized physical connection from the perspective of TLS.

That said, in some corner cases there might still be security issues with such combinations, but only if you e.g are using the same broken random generator for the pre master secrets, etc.


  I am curious as to how much more difficult a scenario like this would make it for an attacker to break the encryption (assuming they could somehow bruteforce 128/256-bit keys).


Most likely, not at all. If you picture yourself being targeted by an adversary with the ability to brute force 256 bit keys (which of course is not realistic), adding a second layer with 128 bit security will not increase the hardness in any significant way. This is (if not for any other reason) because the outer layer might be attacked separately. The TLS handshake is not encrypted, so a theoretical brute force attack on the SSH transport encryption will still only require a $2^{256}$ work factor.