1. A computer program product comprising a computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: receive a video sequence having a plurality of frames; detect a feature in a first frame from the plurality of frames and a first position of the feature in the first frame; estimate the position of the feature in a second frame from the plurality of frames to determine a second position; compute a displacement vector between the first position and the second position; determine a plurality of content characteristics for the first frame and the second frame; spatially diffuse the displacement vector with a spatial filter over a frame from the plurality of frames to generate a spatially diffused displacement vector field, the spatial filter utilizing the plurality of content characteristics; and temporally diffuse over a video volume, with a temporal filter, the spatially diffused displacement vector field to generate a spatiotemporal vector field, the temporal filter utilizing the plurality of content characteristics.