Is there a limit on the size of the block header?
The only valid header is 80 bytes, no more, no less. 

The other stated length of 128 bytes is part of the internal SHA256 hashing algorithm (its input must be a fixed multiple of 64 bytes), it's not represented anywhere in this format. Likewise, the null padding added when hashing the header is not ever considered to be part of it.