14. A computer-readable storage device containing instructions executable by a computing device to enable a natural input device to control a legacy gaming system, the instructions being executable to perform a method comprising: receiving a depth image from a depth camera of a natural input device distinct from a legacy controller device of the legacy gaming system, the depth image including depth values for objects in a field of view of the depth camera; processing the depth image to identify a natural user input, the natural user input simulating a game motion that corresponds to a motion a legacy game input represents; selecting a gesture library from a plurality of gesture libraries based on a legacy gaming program on the legacy gaming system; associating the natural user input with a first gesture from the gesture library selected; mapping the associated first gesture to the corresponding input of the legacy controller device; sending the corresponding input of the legacy controller device to the legacy gaming system in response to receiving the natural user input; and adding a second gesture to a gesture library in the plurality of gesture libraries.
16. The computer-readable storage device of claim 14 , further comprising instructions executable for: removing at least one background element from the depth image; segmenting at least one player of the legacy gaming system within the depth image; mapping a skeleton to the at least one segmented player; associating a change in the skeleton with the first gesture; and identifying the natural user input based on the first gesture.