What will this OP_AMP output if it's VCC is off?
For pretty much any IC design (including Op Amps), the design is only valid for properly biased power supply voltages. After all, an IC is a solid block of semiconductor material, with carefully controlled trace amounts of dopant materials implanted. So unlike a circuit built on a breadboad, where everything is isolated unless explicitly connected, within an IC everything is connected unless explicitly isolated. Every internal transistor is made of several P-type and N-type regions; and every PN junction behaves like a diode. Normally the unwanted PN junctions are reverse-biased, but when the power supply pins are unconnected and voltage or current are also applied at the inputs or outputs, those unwanted (parasitic) PN junctions can conduct. It's somewhat unpredictable. You can't even count on inputs and outputs being inputs and outputs.

Notice that on page 2 at the top of the Electrical Characteristics table, it says in bold letters, Vs = 2.7V to 5.5V. Meaning that those specifications are only valid when the supply voltage is within that range. Above 5.5V or below 2.7V the manufacturer makes no claim about what the product will do.

Some ICs are designed with an shutdown or enable control signal, which is intended to "power off" the IC to a low quiescent current state, while keeping the power supply signals correctly biased. If you're powering off an IC by removing the power supply voltage, all of its inputs and outputs should also be unpowered. Otherwise it may present something like a nonlinear diode sort of load. (That's consistent with the 500mV measurement.)

Part of what's happening in this particular application (driving the input of an ADC) is a phenomenon called charge injection. Internally the ADC has a sample/hold circuit which is equivalent to a capacitor. When the ADC enters "track" mode, the sampling capacitor is connected to the input signal. When the ADC starts a conversion, the track/hold switches to "hold" mode and the sampling capacitor is isolated from the input. A side effect of the conversion is that the charge on that sampling capacitor gets reset. So when the ADC conversion is finished and the track/hold switches back to "track" mode, the sampling capacitor is connected to the input again -- and at this point, a small glitch may be measurable as some charge enters or leaves the sampling capacitor. The ADC input briefly acts like an output, injecting a small amount of charge into the input. This would be avoidable if only the ADC could somehow reset the sampling capacitor to match the input voltage before entering track mode... but there's no way for it to know what the input voltage is without measuring it, and measuring it is what causes the disruption. The effect is more noticeable with a high-impedance source, and it's worse for some ADC architectures. As long as the sampling capacitor gets charged up to the input voltage within the allotted track time interval, all is well.