Using ps2 mouse (PAW3402 / PAW3412) as a distance sensor for arduino
Assuming under normal conditions a relative pointing device such as a optical mouse always reports relative position changes with out regard to speed.  If true, then consider that the speed of the robot is faster than the mouse can process.  Also, consider that the mouse's optics is out of focus with respect to the surface it is supposedly taking its positioning cues from.  Lastly, consider the surface may not be ideal for an optical mouse.  Attempt the same experiment using a sheet of paper or a "not-shiny" wood surface.  

All surfaces have to be flat.  For example, a ceramic tile floor will likely not work as the surface will come in and out of focus during travel.  Make sure the elevation of the robot with respect to the surface is constant.  Inspect the wheels to make sure they are perfectly round and rotating around their center point.

Check and make sure you are not suffering from a integer calculation or type size problem.  Examine your code and make sure you are handling all cases including the expected maximum and minimum values properly.  And that your variables are large enough to handle these expected values.