Video compression - why not just sending the changed parts?
You could do that, in theory, if you knew "what are the parts" (or objects) frame-to-frame in a video. But:


objects move, change scale, rotate, have color changes, they even can be occluded.
the segmentation is a complicated task in general.


MPEG 4, at least, allows to manipulate objects instead of pixels, see MPEG-4 Natural Video Coding - An overview. If objects and backgrounds are already decoupled, like filmed with a green screen:



then the object contents can be processed separately. Otherwise, what is more important is that, visually, the representation is accurate enough. Thus, a block from some object in a frame  can be cast to another block of a different object in a different frame, as long as the result is satisfactory. Thus, in a lot of cases, the block matching is sufficient, and more importantly, relatively fast to compute.