Op Amp different supply voltage than signal voltage
You will have no problem as long as there is a return path from the op-amp circuit to the sensor circuit. Typically this return path is through the "ground" node:



As you suspect, if there is no common ground you will have problems. Even if you use a differential sensor signal to provide a return path, without a common ground, the sensor signal can easily drift out of the common mode input range of the op-amp.