15. A method implemented by one or more processors, comprising: identifying a current image captured by a vision sensor associated with a robot; generating, over a grasp convolutional neural network based on application of the current image to the grasp convolutional neural network: a measure of successful grasp, by a grasping end effector of the robot, of an object captured in the current image, and spatial transformation parameters; generating, over a spatial transformer network, a spatial transformation based on the spatial transformation parameters, the spatial transformation being of the current image or an additional image captured by the vision sensor; applying the spatial transformation as input to a semantic convolutional neural network; generating, over the semantic convolutional neural network based on the spatial transformation, an additional measure that indicates whether a desired object semantic feature is present in the spatial transformation; generating an end effector command based on the measure and the additional measure; and providing the end effector command to one or more actuators of the robot.
16. The method of claim 15 , wherein generating the measure of successful grasp and the spatial transformation parameters over the grasp convolutional neural network is further based on application of a candidate end effector motion vector to the grasp convolutional neural network.