svq3: Check for any negative return value from ff_h264_check_intra_pred_mode

Also pass on any returned error code.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
