avcodec/motion_est: Fix some undefined shifts

Fixes: asan_heap-oob_1dd60fd_1049_cov_4200102444_P4250048.MOV

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
