1. A method for building a model to control a first device, comprising: receiving, as input, demonstration data from a human operating a second device to perform a demonstration and environmental sensory data associated with the demonstration data; determining device settings of the second device, as operated by the human, from the demonstration data; extracting, from the sensory data, feature vectors describing at least locations of objects in the environment; training, based on the determined device settings and the extracted feature vectors, a regressor which takes additional feature vectors as input and outputs planned device settings for operating the first device; and instructing the first device to attain the planned device settings output by the trained regressor.