A robotic arm trying to pick up a block in a MujoCo simulator Google-owned DeepMind uses sophisticated computer simulations for computers to teach themselves how to accomplish certain tasks. The simulated training, known as reinforcement learning, involves the computer trying out thousands (or millions) of different things until it manages to figure out what to do. Using this approach combined with deep learning, the London-based artificial intelligence research unit is teaching computers how to beat the world's best Go players and training robots how to move around in the world. A tiny Berkeley, California-based AI startup, Bonsai, has invented a trick to beat DeepMind in this game. The trick -- the company is calling it "concept networks" -- massively increases the efficiency of reinforcement learning. In a recently published paper, Bonsai's AI researchers describe how concept networks function by breaking out an objective into distinct problem areas. To teach a robot how to pick up and stack a block, for example, Bonsai has broken the task out into five concepts -- reach, orient, grasp, move and stack. The robot needs to learn how to do all five well in order to complete the task at hand. Bonsai solves for each five concepts individually and then combines them all in the end to complete the task. By breaking out individual concepts as their own problem, the robotic arm has a much simpler set of goals to solve for. It needs to, for example, simply figure out how to grasp a block. Also, some concepts such as reach and move are already highly optimized using classical controllers (not deep neural networks), said Bonsai CEO and founder Mark Hammond. As a result, reach and move don't need to be trained all over again. Since the reach and move tasks on the robotic arm don't need to be retrained, the computer can cut down on simulation time. The other tasks that aren't well optimized yet -- orient, grasp and stack -- can be trained through simulation using neural networks. “If the robot already knows how to grasp and stack things, you don’t need to relearn those things over again,” Hammond said. DeepMind's paper describing its reinforcement learning approach takes on a similar grasping and stacking task with a robotic arm, but Bonsai's concept networks makes for a hugely more efficient system. It took DeepMind's system one million cycles to learn how to grasp and stack a block (meaning it took the robotic arm one million attempts to complete the task in a simulated environment). Meanwhile, Bonsai's system took only 22,000 cycles to figure out the task. Bonsai's AI team who worked on the paper includes Aditya Gudimella, Ross Story, Matineh Shaker, Ruofan Kong, Matthew Brown, Victor Shnayder and Marcos Campos, who previously served as head of applied machine learning at Uber. This may all seem like just a neat science experiment, but deep reinforcement learning is an important method for teaching AI systems how to interact with the world. Google, for example, used DeepMind's reinforcement learning techniques on its data center cooling to cut energy usage by 40%. Bonsai focuses on big industrial systems, such a building's HVAC controls and wind farms, where these techniques could be applied to massively increase efficiency. "I see a wide variety of problems this could solve," said Hammond. "This is the cutting edge of applying AI to controller and optimization problems."
California-based start-up Bonsai has worked out how to beat Google-owned DeepMind at its own game - it's designed a way to teach artificial intelligence systems how to interact with the world. The technique, which the company has called "concept networks", substantially increases the efficiency of reinforcement learning, the process of training computers to learn through repeated actions that underpins DeepMind. While it took DeepMind's system one million cycles to learn how to grasp and stack a block (meaning it took a robotic arm a million attempts to complete the task in a simulated environment), Bonsai's took only 22,000 cycles.