Rotary Encoder - Unexpected output
You have, most likely, connected pin A or B to ground instead of C.

Figure 1. Encoder pinout. (Click to enlarge.)

Figure 2. (1) Encoder ABC. (2) Push-button pins.
Now that C is the centre pin.



simulate this circuit &ndash; Schematic created using CircuitLab
Figure 3. In (a) 'A' requires both switches to close before it will pull low. This will occur 25% of the time. 'B' just requires SW2 to be closed to pull low. This will occur 50% of the time.