Playfair Cipher + Vigenere Cipher? I'm researching cryptography for a school project, came across the above two ciphers, and something occurred to me. Would combining these two schemes give a considerably stronger encryption than either one individually?

Consider two encryption schemes: one in which only the vigenere cipher is used (which I'll call $E_1$), and one in which both the vigenere and playfair cipher is used ($E_2$).

When discussing $E_2$, assume all references to it's key will be referring to the key of it's vigenere cipher unless explicitly stated otherwise.

When trying to decrypt $E_1$, the first step is looking for repeated sequences of characters and recording their distance apart to help determine the length of the key, which is likely to be a factor of many of these recorded distances.

However, with an $E_2$ cipher on the same ciphertext, this won't work nearly as well. If E1 showed repetitions of odd length n, $E_2$ would show those same reptitions of length $n-1$ with either the final character or the first character missing, while even length repetitions would either be reproduced at length n or have both the end and the beginning cut off for length $n-2$. In addition, any given sequence will become one of two entirely different sequences based on where the beginning of the sequence falls at the start of a playfair pair or not.

This means two things:


Repeated sequences aren't quite as clear - particularly shorter ones (which are the most common): sequences of length 3 (notably the word 'the') for example, become one of two sequences of length two, which is considerably easier to form by chance (which is why repeated sequences of length 2 are often not used in vigenere cryptanalysis). Four letter repetitions suffer a similar fate - half the time, the playfair pass will instead make them into repeated sequences of length two as well.
If $E_1(P)$ produces n repeated sequences (a single repetition) of average distance m apart, $E_2(P)$ will, on average, produce 2 different repeated sequences, each with $n/2$ repetitions that are $2*n*$ distance apart. This means that the distances between the repetitions are significantly larger and could have significantly more factors, which would mean the length of the key is less certain to the cryptanalyst.
The short answer is yes. It would be considerably more secure.

But nowadays, classical encryption methods like Playfair and Vigenère are so easily broken by computer analysis that they offer next to no security whatsoever. Aiming for something "considerably more secure" than either of these is really setting the bar very low.

Specifically, although the combination of Playfair and Vigenère does reduce the frequency of repeated sequences, it doesn't get rid of them. So you still haven't eliminated the primary weakness of vanilla Vigenère.

For example, if your Vigenère key has length $n$, just one repetition of a word at an interval of $2kn$ places provides enough information to get started on an attack. You obviously don't know what the value of $k$ is, but there's nothing to stop you testing for key lengths corresponding to every divisor of $2kn$.

Even if your plaintext contains no repeated words whatsoever, you're still vulnerable to the same attack if the same word appears in more than one message. (Unless, of course, you use a different set of keys every time, in which case you may as well use a one-time pad.)

This system would also fail according to Kerckhoffs's principle:


  A cryptosystem should be secure even if everything about the system, except the key, is public knowledge.