Scalable Co-Optimization of Morphology and Control in Embodied Machines
  Evolution sculpts both the body plans and nervous systems of agents together
over time. In contrast, in AI and robotics, a robot's body plan is usually
designed by hand, and control policies are then optimized for that fixed
design. The task of simultaneously co-optimizing the morphology and controller
of an embodied robot has remained a challenge. In psychology, the theory of
embodied cognition posits that behavior arises from a close coupling between
body plan and sensorimotor control, which suggests why co-optimizing these two
subsystems is so difficult: most evolutionary changes to morphology tend to
adversely impact sensorimotor control, leading to an overall decrease in
behavioral performance. Here, we further examine this hypothesis and
demonstrate a technique for "morphological innovation protection", which
temporarily reduces selection pressure on recently morphologically-changed
individuals, thus enabling evolution some time to "readapt" to the new
morphology with subsequent control policy mutations. We show the potential for
this method to avoid local optima and converge to similar highly fit
morphologies across widely varying initial conditions, while sustaining fitness
improvements further into optimization. While this technique is admittedly only
the first of many steps that must be taken to achieve scalable optimization of
embodied machines, we hope that theoretical insight into the cause of
evolutionary stagnation in current methods will help to enable the automation
of robot design and behavioral training -- while simultaneously providing a
testbed to investigate the theory of embodied cognition.
