How to remove gain and offset errors from ADC logic?
A old but good application note from TI: Understanding Data Converters
https://www.ti.com/lit/an/slaa013/slaa013.pdf
They show the dominant errors in ADC's.
Most of them you can measure and compensate with a Lookup Table, Interpolation or simple multiply/add operations... but you will loose full scale range in most/all cases, so I guess your fun with correcting a 3bit Converter will be limited.