Encoder direction change during index pulse
It sounds like your index reset is happening in hardware and you can't control what edge(s) to trigger on. You could always move it out into a software interrupt instead.
A concept I ran into only after programming quad encoders for FPGAs and STM32s (they have a A-B counter but the index and all the nitty gritty features must be done in software with interrupts), was found in the dsPIC hardware Quadrature Encoder Interface:

http://ww1.microchip.com/downloads/en/DeviceDoc/70000601c.pdf
The feature is called &quot;Index Match&quot; although most of the time it is referred to simply as &quot;match&quot; in the document so don't try and only search &quot;Index Match&quot; or you will miss the bulk of the important material.
Basically, it does not responds to an index pulse alone. It only responds to the index pulse when when the A and B channels are a particular state. It uses this so that the exact same point is used for a reset regardless of the direction you are approaching the index pulse from.
You can manually select ahead of time what you want the A and B states to be, or record them when the first index pulse is received. The catch with this is that it is a lot easier to implement in hardware than in software since (I think) the optimal way is to search for the correct A-B state while the index pulse is the appropriate level. That's super easy to do in hardware but would bog down your processor in software.
In software it's much less expensive to use interrupts but those are edge triggered. Obviously, the bad way to do it is to furiously hunt for the proper A-B state while the index is active, and the worst way is to furiously hunt for the index whenever the the A-B state is correct. You could perhaps have an edge-triggered interrupt on the index pulse that disables itself after enabling an edge-triggered interrupt on the A and/or B channels to check the A-B state. Then after it finds it, it can disable itself and re-enable the index interrupt.

Alternatively, you could set the index interrupt to trigger on both rising and falling edges, inside the interrupt, only reset the counter if the type of edge matches the direction of rotation (there would be two valid combinations, and the other two would be rejected). That would effectively turn your index pulse into an index-edge. Sounds like a lot less work than what I described above.