What determines if a small protein / large peptide is immunogenic?
Edited to delete off-topic section: 

Most (all?) proteins are ultimately immunogenic. You can't really design your way around immunogenicity, which is a good thing, because otherwise we'd all be dead from viruses. 

Obviously not all proteins are equally immunogenic, but basically all protein therapeutics are immunogenic at some rate. One thing that seems positively associated with immunogenicity in that review is quaternary structure, but it's quantitative not qualitative.

Some of the commonly used synthetic antigens used in biochemistry such as HA C-terminus (24 residue) or FLAG (8 residue) are quite small. 

More generally, this study finds that even 3-5 residue peptides can be sufficient to raise immune responses. So likely any polypeptide is potentially immunogenic.

Such small peptides are unlikely to have meaningful secondary/tertiary structure. My guess is that that you would have to have a pretty structurally strange protein to avoid any potential immune response. However if you had an insoluble or somehow sequestered protein you might be able to avoid it, but I'm not an expert so I can't say how.