How is jitter of an oscillator measured and quantified in practice?
Jitter is the time domain word for this characteristic. The frequency domain characteristic would be the Phase noise.

Check wikipedia for a quick overview: https://en.wikipedia.org/wiki/Phase_noise

Phase noise can be measured different ways:


Spectrum Analyzer. The spectrum analyzer must have a significantly better oscillator than the one you want to measure!
Special Phase noise measurement equipment. There are certain specialized ways to measure it.
Take an oscilloscope and measure the jitter in time domain (Usable only for lower frequencies that can be well measured with an oscilloscope) 


Here is a presentation from keysight explaining the measurement: https://www.keysight.com/upload/cmc_upload/All/PhaseNoise_webcast19Jul12.pdf

Since you didn't specify the sort of oscillator (low frequency, high frequency etc, sine, square...) it is difficult to give an exact advise on how to measure it.



Edit: The spectrum of phase noise can be described with Leeson's formula
$$ L(f_m) = 10 \log \bigg[ \frac{1}{2} \bigg( \bigg(\frac{f_0}{2 Q_l f_m}\bigg)^2 + 1\bigg)\bigg(\frac{f_c}{f_m} + 1\bigg)\bigg(\frac{FkT}{P_s}\bigg) \bigg] $$

If you measure the phase noise spectrum you can calculated back the parameters inside the formula, like the quality factor Q of the oscillator. It can also be used to qualify the oscillator's performance.