Cheap options for high precision distance measurement?
I think you should get to the bottom of what is causing your errors. Either you are missing steps or your mechanical setup is too flexible or has backlash in the nuts/bearings. Your homing switches may also have poor repeatability if you are not using an edge finder to locate the tool precisely. Or perhaps it's a combination. 

Things like backlash and flexibility are very difficult to compensate for with software. For example, depending on the direction of cut you may start off with the tool in the right position but as soon as it bites in kerchunk and you've dug into the work as the cutter pulls itself in. Or you may be doing climb milling and the cutter runs well outside of the desired path if the gantry and slides are too flexible. 

Anyway, glass scales are a sort of mid-priced way to measure a few microns down to 1um resolution. Accuracy over the full scale might be 10-15um over 1m for a cheap one. They typically have a quadrature 5V digital output (incremental), some may have quadrature sinusoidal signals. But each axis will probably cost about as much as you've put into this so far, and there is no guarantee you will be able to do much better in part accuracy. Photo from this page



If you win the lottery you can consider Renishaw and Heidenhain encoders, which can reach resolutions orders of magnitude less than a wavelength of light and do it with absolute measurement.