wheel encoder to find direction of wheel
This isn't the usual rotary encoder pattern, but it should still work. See this wikipedia section for the normal gray-code implementation.

Basically, you will need to:


Store the current state of the encoders to a register
Wait for some length of time. (This must be short enough that only one transition occurs)
Compare the new encoder state with the old encoder state to see if they are different. 
Based on your previous state, and the new state, decode the direction. (This step/algorithm would be easier if you used a typical implementation.)


The disadvantage of your setup is that it takes two transmitters, probably has a more complicated pattern drawn, and there's no way to identify if the encoder has gone 2 steps instead of 1. Additionally, only one bit changes with the gray code count, so the comparison routine is easier and not a look-up table.