Why is shielding effectiveness different for various hole geometries?
Antenna efficiency decreases as it gets smaller relative to a half wave. This means shielding effectiveness (SE) improves as the radiator shrinks relative to its 1/2 wave dimension. 

We know that slots, gaps and apertures radiate unintended RF proportional to these fractional wavelengths well below 1/4,1/2 wave.  But apertures needed for intentional radiation antenna also leak out unintended radiation.

These experiments prove the average SE follows this well-known behaviour.  If there must be an aperture for an internal antenna, a circle is an optimal shape for unintended emissions as the diameter is constant and a square or rectangle.

Slots have always had higher impedance for grounds, cabinets and doors than circles which degrades unintended efficiency by raising the impedance in the air gap surrounded by ground. This is why they are attached at least every 1/2 wavelength or PCB dual ground planes attached by micro vias every 1/10th wavelength.

It also verified that internal shields can be coupled to small apertures and increase radiation coupling by orders of magnitude either unintended or intended by Patch antenna.

This is just my experience, not a theoretical answer.