STM32 Rotary Encoder with Hardware Interrupts
That's not a great algorithm in your handler.  You should have ZERO ifs. No decisions.

Store your AB state, i.e., 00 or 01,  then append your next state,  i.e 0001 means AB went from 00 to 01,thus B changed from 0 to 1.   Make this  a +1.  If starting from 00, and you change to 10, then call this a -1.  Build a 16 element array of all possible transitions holding the number that needs to be added to your count if it occurs, noting that some are illegal and need to be handled.

0000       0    0  no transition


0001       1   +1

0010       2   -1

0011       3    0   Illegal, two transitions,  and so on.


Index into this array on every transition, watching for illegal events and dealing with them as you see fit. Add the result to the count.  Shift the new values to the old value spot in the index number, and repeat forever

In Pseudocode

 signed int8 add_subt[16] = { 0, 1 ,-1 ,  ....};
 unsigned int8 idx;
 signed int32 pos_count;

main() {
% initialize idx
idx = readA &lt;&lt;3 + readB&lt;&lt;2 + readA&lt;&lt;1 + readB;
while(1){}
}

interrupt_on_any_change(){
idx=idx&lt;&lt;2 &amp; 0x0F + readA&lt;&lt;1 + readB;
pos_count=pos_count+add_subt[idx];
}


You could maintain err_idx to help you flag bad transitions

Doesn't that look a hair simpler??