1. An image processing apparatus comprising: a conversion unit configured to reduce an input image by a predetermined number of times to generate an image group containing the input image; an extraction unit configured to extract a plurality of feature points and a local feature amount of each feature point from the image group; a first determination unit configure to compare a first image that a first image group generated from a first input image by the conversion unit includes, and a second image that a second image group generated from a second input image by the conversion unit includes, and to combine a feature point of the first image group and a feature point of the second image group; a second determination unit configured to determine a relation between the first image and the second image in which a size of an object of the first image and that of the second image get closer; and a calculation unit configured to assign high weights to a combination of feature points extracted from an image having the relation determined by the second determination unit to calculate a similarity degree between the first input image and the second input image.