Strange problem with proximity sensor on Nexus 5X
Answering my own question. As I didn't have much to loose, I opened the phone up and found HUGE amounts of dust inside, including around the proximity sensor.  The whole sensor was just clogged with dust. I took out all the tiny parts, gaskets, etc., carefully cleaned everything with a soft brush and re-assembled the phone. Voila! Everything works correctly now.

I suspect the dust was getting in the way of the light in the proximity sensor - but it wasn't fully closed due to the dust moving around there.