1. An image processing apparatus comprising: a calculation unit configured to calculate a difference between a feature quantity of input video data and a feature quantity of a model representing a background; a determination unit configured to determine whether a partial area in the input video data is a foreground or a background based on the difference; a detection unit configured to detect a subject area from an area determined to be a foreground; a first update unit configured to update appearance information that represents an appearance state of a background relating to the subject area; and a second update unit configured to update the model based on the appearance information.