When to use buffer/ low-pass filter on ADC input?
The output impedance of the potentiometer (with both ends connected to low impedances) varies over the pot's revolution.  The greatest is at halfway.  The impedance is the parallel combination of half the pot's resistances, or one-fourth of the potentiometer's resistance.

Each ADC will give a specification for maximum driving impedance.  If this is greater than the pot's output impedance, then accuracy will be affected.  Some ADC's care more than others about input impedance.  Some datasheets will give you an equivalent circuit to the ADC's input impedance.  You can use this to analyze the error you are likely to see, either by hand or with a Spice simulator.

In particular, watch out for some PIC microcontrollers which have a surprisingly low input impedance and maximum driving impedance spec.

The low-pass is for one of several reasons:

1) There is a low-pass filter for removing frequencies above the Nyquist frequency.  But a hand-operated potentiometer just won't generate enough high frequencies for aliasing to be a problem.

2) The capacitor is right at the ADC input pin and is intended to reduce the driving impedance, at least at high frequencies.  I've had mixed success with this method.  It doesn't work if there is DC leakage at the ADC input.

3) The capacitor is across the potentiometer's wiper.  The common failure mode for potentiometers is that the track gets dirty and makes intermittent contact. You have experienced this problem if you've ever had an old radio that made loud crackling sounds when you touched the volume knob.    With a capacitor across the wiper, the wiper voltage doesn't change if it loses contact with the track intermittently.  (This trick only works for DC signals.)

Remember, op-amps are pretty cheap.  I would recommend always buffering the signal before ADC unless the volumes are high enough that the cost savings are worth it.