1. A computer-readable medium carrying one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to fit an input image to an object model by performing the steps comprising: [a] using a object model fitting system comprising a set of weights, at least one set of manifold constraints, and a three-dimensional (3D) object active appearance model (AAM) comprising a 3D object shape model that receives shape parameter values and pose parameter values and an object appearance model comprising a set of object subcomponent appearance models that receive appearance parameter values, to generate a first image and a second image, the first image being generated using the 3D object shape model, the input image, the shape parameter values, and pose parameter values to generate the first image from the input image; and the second image being generated using the object appearance model and appearance parameter values to generate the second image; [b] calculating a set of fitting errors representing a difference between the first image and the second image; [c] responsive to a stop condition not being satisfied: updating at least some of weights in the set of weights using at least some of the shape and pose parameter values; updating at least some of the shape, pose, and appearance parameter values using at least some of the set of errors and the at least one set of manifold constraints; and returning to step [a].