tiff: fix overflows when calling av_reduce

The arguments of av_reduce are signed, so the cast to uint64_t is misleading.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
