Do I need buffers prior to a comparator?
Typical noise from a 24V zener at optimum current (much higher than you are running them at) is less than 200mV RMS (maybe 1 or 1.5 volts p-p). 

A 24V zener might have 5% tolerance, meaning that your two zeners may be mismatched by +/-2.4V, meaning no signal at all. 

You could consider replacing one of the zeners with an RC low-pass filter connected to the remaining zener so that it settles near the average voltage. 

If you're depending on this being truly random you will want to make sure the PSD is fairly flat.