1. An image processing apparatus comprising: a conversion unit configured to reduce an input image by a predetermined number of times to generate an image group containing the input image; an extraction unit configured to extract a plurality of feature points and a local feature amount of each feature point from the image group; a determination unit configured to compare a first image of a first image group generated from a first input image by the conversion unit with a second image of a second image group generated from a second input image by the conversion unit, and to determine a relation between the first image and the second image in which a size of an object of the first image and that of the second image get closer; and a calculation unit configured to assign high weights to a combination of feature points extracted from images having the relation determined by the determination unit to calculate a similarity degree between the first input image and the second input image.