How does the ELPF and XTAL inputs on the ADV7180 recommended circuit work?
The capacitors for the crystal are crystal load capacitors. The actual value will vary across crystal manufacturers, but the suggested values are a good start. The datasheet is light on details of the required load capacitance for the internal oscillator.

The ELPF components are part of the loop filter for the internal phase locked loop.

I have extensive experience with this specific range of decoders, and if your application has any real temperature variation, the use of C0G / NPO capacitors in the ELPF is not merely recommended, but just about mandatory for stable circuit performance.

Note that ADI does not mention this; I found it out the hard way.

The component values in the ELPF are set by Analog Devices, and I was never told just why those specific values were used despite my best efforts to find out (my application was a bit unusual, so understanding all these things was rather necessary).