Acid strengths for HBrO4 vs HIO4, and HBr vs HI
"We know that HI is a stronger acid that HBr because I is a much larger atom"


That is only half the story. It is about the strength of the H-Br versus H-I bond. Since I is larger than Br, the H-I bond is longer, weaker, and therefore more prone to dissociation. See here for more in-depth discussion.


  $\ce{HBrO4}$ and $\ce{HIO4}$


both have the acidic proton bound to oxygen:



The H-O bond is approximately of equal strength in both acids and therefore you turn to electronegativity arguments.