lavc/videotoolboxenc: Fix crash when closing codec after error

Fixes crash in #5352. VTCompressionSessionInvalidate() crashes if the
internal encoder hasn't completed, but hasn't experienced an error.
The function call isn't needed since the encoder is invalidated when
the reference count reaches 0 anyway.

Signed-off-by: Rick Kern <kernrj@gmail.com>
Signed-off-by: wm4 <nfxjfg@googlemail.com>
