Why do modern fighter jets use 20mm guns?
20mm is the smallest caliber that is practical to load with a bursting charge. Some fighter jets use 23mm, 27mm, or 30mm guns.

The reason 20mm has become one of the most common aircraft gun calibers is that it produces the lightest gun that meets the minimum requirements to be useful in air combat: enough rate of fire to hit a maneuvering jet, good enough ballistics for meaningful range, enough bursting charge capacity for meaningful damage.  

The majority of modern combat jets do not carry armor, but they are built with considerable redundancy and internal subdivision. A bursting charge (explosive payload) is needed to do meaningful damage to a modern combat aircraft. 

Smaller calibers can cause some damage that will require repairs, but not enough to rely on them to bring down a jet in combat. With missiles prioritized, most air forces want the lightest gun that's still effective.

Larger calibers have a cost in design. A large caliber rapid-firing gun will be much heavier, its propellant gases (muzzle exhaust) can cause flameouts in the engine, and firing will throw the aircraft's aim off center. A slower-firing one can be suitably light, but still has the same recoil force. Another issue with large calibers is collateral damage - 30mm+ rounds send fragments a couple hundred meters, far enough to damage friendly aircraft in air combat maneuvering.

The 20mm rounds used by combat aircraft have limited armor-piercing capability anyway (comparable to .50 BMG). Armored aircraft are rare, and some exceptions include CAS aircraft such as the A10 and to a lesser extent the Su-25 and the Su-34. This armor is mostly designed to protect them against ground fire, not enemy fighters.