Why do all inter-halogen compounds contain an even number of atoms?
Yes neutral $\ce{IF4}$ would be unstable because it would have one lone pair and one unpaired electron.  The extra lone pair is not a problem because Iodine is large enough to exceed an octet, but the unpaired electron makes it a free radical.