1. An apparatus for encoding a picture in a video sequence, comprising: a patch library creator for creating a first patch library from an original version of said picture and a second patch library from a reconstructed version of said picture, each of said first patch library and said second patch library including a plurality of high resolution replacement patches for replacing one or more pruned blocks during a recovery of a pruned version of said picture; and a pruner for generating said pruned version of said picture from said first patch library, and for deciding whether to respectively replace one or more original blocks from said original version of said picture with said one or more pruned blocks responsive to a comparison between a first rate-distortion estimate and a second rate-distortion estimate, said first rate-distortion estimate and said second rate-distortion estimate being based on both a distortion component and a bitrate component; a metadata generator for generating metadata from said second patch library, said metadata for recovering said pruned version of said picture; and an encoder for encoding said pruned version of said picture and said metadata.