1. A method for temporally segmenting a video, the method comprising: selecting sample video frames from a plurality of decoded video frames of the video; annotating each of the sample video frames; segmenting the selected sample video frames into a plurality of segments for each entity of the video based on the annotation of the selected sample video frames, a segment for an entity of the video representing a semantically meaningful spatial-temporal region of the video; and generating an overall temporal segmentation of the video based on the plurality of segments of each entity of the video.