LM324n weird output when used as buffer
This behavior is called phase inversion.

It happens in certain op-amps when you exceed the input common mode range limits of the op-amp (for example, when the input voltage is below the most-negative supply voltage).

It's explained at the circuit level in a TI presentation I found (warning: Powerpoint link), and also in an EE Times article.

Notice the input common mode minimum voltage for your op-amp is equal to the most negative supply voltage, so you should not use a 0 V VEE with this op-amp if you will have negative inputs. An easy solution in your case might be to use the op-amp with split supplies to handle the velocity input.