Beagle Bone Black interrupts for optical encoder
BBB run on Sitara AM335x which already has the interface for 3 encoders, it called eQEP. The pins are probably used as GPIOs, what you would need is to locate pins of your board, then search for suitable linux driver that can use/setup those GPIOs as eQEP, then you don't need an interrupt - this method is deprecated, it was used with MCUs without a time process unit, nowdays MCUs have all those intefaces like encoder input, PWM generator, timers,...already implemented in hardware, they are very fast.
I have done a fast internet search and this might help you: https://github.com/Teknoman117/beaglebot/tree/master/encoders/patches