Crystal oscillator behaving strangely
I don't see anything inherently wrong with your circuit or your constant-current drive output circuit, except that you MUST connect unused inputs on the chip to a valid logic level (ground or +5 in this case). In particular, pin 12 (RESET) must be grounded or it will float around with leakage and capacitive pickup.
It's also possible the 470K is too high and your oscillator is barely able to maintain oscillation.