1. A method comprising: obtaining video data comprising a video stream of a person; classifying pixels in the video stream at a given time instance during a time period as one of a foreground pixel and a background pixel; generating a data entry in a data structure corresponding to each pixel, wherein the data structure comprises data indicating foreground history values for each of a plurality of time instances of the video stream and data indicating a time period value; when classifying indicates that a first pixel of the video stream is a foreground pixel, evaluating the data structure associated with the first pixel to determine whether or not to update the data representing a foreground history value associated with the first pixel at the given time instance; and generating a motion gradient vector for the video stream based on the foreground history value associated with the first pixel and foreground history values associated with other pixels over the time period.