1. A moving object detection apparatus, comprising: motion vector calculation means for calculating motion vectors from an input image; motion vector removal means for removing a motion vector having high randomness from said calculated motion vectors; motion vector accumulation means for temporally accumulating each said motion vector not removed by said motion vector removal means, and for calculating an accumulated number of occurrences and an accumulated value of each motion vector; and moving object detection means for determining, based on said calculated accumulated value and calculated accumulated number of occurrences of each motion vector, whether each said motion vector corresponds to a moving object.