1. A three dimensional shape reconstitution device comprising: an observation matrix generating unit configured for inputting an image sequence of a captured target object, for tracking a plurality of feature points preset to the target object on an image plane, and for generating an observation matrix including time-sequential arrangement of positions of the feature points; a factorizing unit configured for factorizing the generated observation matrix in the observation matrix generating unit, and for generating a shape matrix showing a three dimensional shape of the target object and a motion matrix showing a motion of the target object; and a reliability evaluating unit configured for generating a position reliability evaluation value showing reliability of the position of each feature point, wherein the factorizing unit executes factorization with emphasis on the feature point having the high position reliability evaluation value.