What is wrong if you use a key in a Caesar cipher that is greater than 26? TL;DR- why can't we have a key cipher greater than 25 in the Caesar cipher?

Let's suppose the key is K = 30.
The message I want to send is "H E L L O W O R L D"

Now converting each letter to a number from 0-25, we get 7-4-11-11-14 22-14-17-11-3.

Adding 30, we get 37-34-41-41-44 52-44-47-41-33.
But as the cipher text ranges only from 0-25, we now have to use the modulo function on every letter by 25.

We end up getting 12-9-16-16-19 2-19-22-16-7

Now on receiving this message, all one has to do is subtract the key 30 from each number, add 25 and then convert back each number to its corresponding alphabet. 

So why do we have to restrict the key-space of a caesar cipher to 0-25?
Basically the Caesar and Vigenére ciphers perform operations on groups. In this case the group consists of the values $\{0, 1, 2 ... 25\}$ in that particular order and the modular addition $+$. Now within a group it doesn't make sense to add a number outside of the group domain - in this case 0 to 25 - to the value.

So as we are using modular addition we can see that $12 + 30 \equiv 12 + 4 \mod 26$ as $42 \bmod 26 = 16$ and $16 \bmod 26 = 16$.

We can use this fact to call the values $0$ to $25$ representants because they actually just represent an infinite amount of integers: all integers which are the representant plus some multiple of $26$. So with this in mind $30$ and $4$ have the same representant $4$ (in theory you could use special notation to make the difference between numbers and representants for infinite amounts of numbers more clear but we will not do so for the sake of alignment with established standards).

So adding 30 only makes sense if we extend the group to at least 31 values. Within these ciphers that basically means extending the alphabet: the letters that get encrypted (or duplicating characters, to make it more difficult to analyze the resulting ciphertext).



You could accept $30$ as key, but it would not really extend the key space because $30$ would be fully equivalent (that's the $\equiv$ symbol) to the key value $4$. So the final key strength would still consist of 26 possible keys (about 4-something bits) as well.