1. A computer-implemented method comprising: identifying, in a plurality of digital videos, a plurality of candidate volumes representing spatio-temporal segments of the digital videos, wherein each of the candidate volumes corresponds to a contiguous sequence of spatial portions of the video frames having a starting time and an ending time, and potentially represents a discrete object or action within the video frames; determining, for each of the identified candidate volumes, features characterizing the candidate volume, wherein the features are determined from visual properties of the spatial portions of the video frames contained in the candidate volumes; and assigning a verified label to each volume of a plurality of the identified candidate volumes using the determined features, the verified label indicating a particular object or action represented by the volume to which the label is assigned.