How does harmonic distortion occur in ADCs?
There are many distortion mechanisms in ADC but the most important ones are:


"static" distortion due to INL
Track-and-hold distortion


The first one is caused by the fact that the voltage-in-code-out relationship is not perfectly linear (INL/DNL).

An approximation of harmonic distortion based on DNL is given by (Lee, JSSCC 12/2017):

$$
\mathrm{HD3} = -20\log\left( \frac{4}{3\sqrt{3}} \frac{2^B}{\mathrm{INL}_{\mathrm{max}}} \right)
$$

The second one is caused by track nonlinearity. The sampling switch is for example a weakly nonlinear CMOS switch. Its on-resistance is given by (using the simple square law model):

$$
R_{on} = \frac{1}{\mu C_{ox} \frac{W}{L} (V_{DD}-V_{in}-V_t)}
$$

From which is can be seen that the resistance depends on the input signal (see for example Razavi, Data Conversion System Design, page 16).

This distortion can be approximated as

$$
\mathrm{HD3} = 20\log\left(\frac{1}{4} \left(\frac{A}{V_{GS}-V_t}\right)^2 2\pi f_{in} R_{on} C \right)
$$

For a simple CMOS switch this is by far the dominant distortion source and is practically overcome by using a bootstrap sampling switch. However, also this technique has limitations.