Wrong load capacitance for 20 MHz, 20 pf crystal
To get the specified 20 pF load for a crystal, you need two 40 pF caps, theoretically. However, these 40 pF caps in reality do include package pin capacitance (2-3 pF each pin), plus parasitic capacitance of traces and pads under the IC and under XTAL, which typically are about 5-6pF. Therefore in your case you should use about 33 pF caps. Together with pin and trace capacitance it will form the desired 40 pF.

The parameter you seem to be concerned with is called "crystal pullability", see this publication for brief explanation and typical dependence. Deviation of +- 5pF from the designated 20pF load will give you at most +- 100ppm, which is about +- 2 kHz for a 20 MHz crystal.