Instrument amplifier gain setting and REF pin setting on INA232
The INA232, when powered from 5V, has a minimum input voltage of 0.55V, so if you apply 50mV input you're operating the part out of it's specification.