Reference clock output conditioning?
Turns out this was a red herring. When I finally got around to looking at it on a proper 50 ohm terminated scope with 50 ohm coax, the signal looks fine. It's around 1.5v from low to high and 10-90% rise and fall times are around 5 ns.