1. A computer-implemented method for estimating a state-space controller from a set of video frames, comprising: receiving the set of video frames, wherein the video frames depict a motion of an entity; initializing, via one or more processors, the state-space controller, wherein the state-space controller includes states corresponding to an atomic controller and transitions between the states; and incrementally optimizing parameters of the state-space controller and changing a structure of the state-space controller to approximate through simulation the motion as depicted in an initial subset of the video frames and in expanded subsets of the video frames, wherein parameters of a last one or more states of the state-space controller are optimized at each stage of the incremental optimization.