1. A method for managing a set of states associated with users of a gesture-based system, comprising: setting a state of the gesture-based system to an unbound and detected state in which a first user is not bound to a first virtual port of the gesture-based system based on determining that the first user is present in a first depth image; after setting the state for the first user to an unbound and detected state, binding the first user to the first virtual port and setting the state for the first user to a bound and detected state; processing a motion or pose of the first user captured in a second depth image to invoke a command to the gesture-based system after setting the state for the first user to the bound and detected state; and reassigning a second user from a second virtual port to the first virtual port after the first user becomes unbound from the first virtual port, based at least on input received at least at one of the first virtual port or the second virtual port.