Why does this comparator exhibit non-binary behaviour?
This kind of high-speed chip does not work on a breadboard.
After placing it upside down dead-bug style (thanks @peufeu) on a bit of scrap PCB, with the signal source next to it (and the copper foil connected to GND), I am seeing the expected output.
Connections need to be as short as possible. Even after placing it I still observed a lot of "linear", non-binary behaviour until I connected it directly to the signal source with a short wire. Prior to that I had been using a piece of audio cable about 10cm long with a phone jack connected to my phone.
Below is the component connected to the signal source (an MC44144 PLL) and the output, 0-5V at around 4.43MHz.