PIC18 Robot - Travelling exact distance straight using two wheel encoders problems
Sounds like you are missing encoder ticks when your CPU is busy with other tasks.

Your Robot is executing code in a loop and the loop times are approximately constant so you are missing the same number of ticks (roughly) in each loop. You've calibrated this error out of your system. 

However, occasionally, you get unlucky and the loop runs longer and you miss more ticks than average.

It is impossible to decode/debug/double-check your firmware with the snippet provided since the register-level configuration of the hardware input capture unit is omitted. I'd look there.