Don't fill in frame gaps with copied refs after flush

The filled in refs cause corruptions in the video frame
for a long time after it should have recovered.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
