1. A method of tracking a human subject within a scene, the method comprising: receiving from a source an observed depth image of the scene including the human subject; supplying the observed depth image to a pose tracking pipeline; receiving an indication of a problem state from the pose tracking pipeline; supplying an identification of the problem state to the pose tracking pipeline; and receiving from the pose tracking pipeline a virtual skeleton including a plurality of skeletal points defined in three-dimensions, the pose tracking pipeline selecting a three-dimensional position of at least one of the plurality of skeletal points in accordance with the identification of the problem state supplied to the pose-tracking pipeline.