1. A moving object detection apparatus which detects a moving object in video by performing segmentation to identify an entire or partial region of the moving object in the video, the moving object detection apparatus comprising: an image input unit configured to receive at least three pictures included in the video; a trajectory calculating unit configured to (i) extract, from among the at least three pictures, image subsets each including at least two of the pictures, and (ii) calculate, for each of the image subsets, trajectories each of which is a trajectory of motion of a block, between the pictures included in the image subset, the block including one or more pixels and being included in each of the pictures; a subclass classification unit configured to classify, for each of the image subsets, the trajectories into subclasses, the trajectories being included in the image subset and calculated by the trajectory calculating unit; an inter-subclass similarity calculating unit configured to (i) calculate a trajectory share ratio which indicates a degree of sharing a same trajectory between two of the subclasses, and (ii) calculate a similarity between the two of the subclasses based on the trajectory share ratio; and a moving object detection unit configured to detect the moving object in the video by classifying the subclasses into classes such that the subclasses between which the similarity calculated by the inter-subclass similarity calculating unit is higher are more likely to be classified into the same class, to identify, as the region of a moving object, the blocks which correspond to the trajectories included in a same class.