1. An information processing apparatus comprises: a first calculation unit configured to calculate a relative position and orientation with respect to the object relative to the imaging unit for each of the plurality of view points based on a plurality of images captured by imaging an object from a plurality of view points by the imaging unit; a lacking position and orientation calculation unit configured to calculate a lacking position and orientation of the imaging unit from which imaging by the imaging unit is acking based on the calculated plurality of relative positions and orientations; and a lacking view-point image generation unit configured to generate an image used for display of the calculated lacking position and orientation on a display unit.