lavc/imgconvert: fix check on av_image_check_size() return code in avpicture_get_size()

The documentation states that av_image_check_size() will return a
negative value in case of error, while the check is done on ret != 0.

Also return a proper error code rather than -1 in case the check fails.
