Floating analog input value
Sounds like the sensor's output is noisy. Put a capacitor (100nF or so) at the sensor output to reduce the noise.

Or 
in the coding side, you can take \$N\$ samples of sensor value and then average it. Use this average value for further calculations. 

You may also want to calibrate the sensor.

Calibration


Find the minimum ADC reading given by the sensor - \$S_{min}\$
Find the maximum ADC reading as well - \$S_{max}\$
Your sensor therefore gives value \$x\$ in range \$(S_{min} - S_{max})\$, which you can map to \$x_m\$ in the range \$(0 - 1023)\$ mathematically in the code using the relation:


$$\frac{x-S_{min}}{S_{max}-S_{min}}=\frac{x_m-0}{1023-0}$$
Use \$x_m\$ for further calculations.