qdm2: Check data block size for bytes to bits overflow.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit dac56d9ce01eb9963f28f26b97a81db5cbd46c1c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
