Mouse Wheel Rotary Encoder ... encoding wrong
Mice encoder wheels are normally quadrature encoded.  The mechanical indents may or may not correspond to a change in the encoded signal.

A quadrature encoded signal normally involves 2 individual outputs.  Let us arbitrarily label them A and B.  When rotated in one direction A will change state followed by B.  After further rotation A will again change state followed by B.  If rotated in the opposite direction, B will change state followed by A.  Then, after further rotation B will again change state followed by A.



The sensor which drives output A should be 90 degrees out of phase with the sensor which drives output B.  However as long as the two sensors are at different locations (angles) this quadrature encoding scheme should work.

Knowing this, we can devise a logic circuit (or the equivalent in software) to determine in which direction the encoder wheel is rotating and how far it rotated.