1. A threshold learning control system for learning a robot controller of a robot, the threshold learning control system comprising: a threshold learning module executing on a data processing apparatus and receiving sensor inputs from a sensor system of the robot; a regime classifier executing on the data processing apparatus and receiving sensor inputs from the sensor system of the robot and determining a control regime based on the received sensor inputs, the regime classifier communicating the control regime to the threshold learning module, the regime classifier receiving at least one state-action map, each state-action map having control regimes arranged contiguously with boundaries therebetween, and each control regime providing a state-action space of possible robot states and robot actions in a corresponding control space; an exploratory controller executing on the data processing apparatus and receiving sensor inputs from the sensor system of the robot and control parameters from the threshold learning module; and a control arbiter executing on the data processing apparatus and receiving exploratory commands from the exploratory controller and limits from the threshold learning module, the control arbiter issuing modified exploratory commands based on the received limits to the robot controller, wherein the threshold learning module learns the boundaries between control regimes within the state-action space of the at least one state-action map using at least one of the received sensor inputs, control regime classifications of the regime classifier, anchor points of the at least one state-action map, and feedback of the modified exploratory commands issued by the control arbiter.