Ground loop problem in op amp circuit
There is no ground loop in your question since there are no current shown to cause noise. So we can assume this is induced line noise.


Common Design error.


Mismatched source and Return impedances.


The whole point of using an INA is to use the differential inputs and have balanced impedances from source to output.  
INA Impedance, Differential \$10^{12}~Ω // 1 pF\$



Your external to IC impedance on IN- = 81MΩ and IN+ =0 . Thus to see 1mV of noise all you need is \$I=V/R=\dfrac{10^{-3}~V}{81 ~MΩ }= 12~ pA\$

If you balance your sensor, wires and input impedance, if this is induced line noise, this should correct the problem. 


The Vbat/2 may need noise suppression cap. 
the cable may need to be twin ax shielded differential pair.
you may need a Pi filter Balun = CM audio choke