avcodec/xsubdec: Check that RLE coded image and colors fit in the buffer

Fixes: Timeout
Fixes: 1747/clusterfuzz-testcase-minimized-6035451213250560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
