avformat/mov: Change the type of the r/g/b variables

Fixes integer overflow
Fixes: unknown_unknown_31b_795_cov_1818643045_raybauduc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
