avformat/rtmppkt: Check for packet size mismatches

Fixes out of array access

Found-by: Paul Cher <paulcher@icloud.com>
Reviewed-by: Paul Cher <paulcher@icloud.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>