Embodied intelligence via learning and evolution

The authors propose a new framework, deep evolutionary reinforcement learning, evolves agents with diverse morphologies to learn hard locomotion and manipulation tasks in complex environments, and reveals insights into relations between environmental physics, embodied intelligence, and the evolution of rapid learning. The intertwined processes of learning and evolution in complex environmental niches have resulted in a remarkable diversity of morphological forms. Moreover, many aspects of animal intelligence are deeply embodied in these evolved morphologies. However, the principles governing relations between environmental complexity, evolved morphology, and the learnability of intelligent control, remain elusive, because performing large-scale in silico experiments on evolution and learning is challenging. Here, we introduce Deep Evolutionary Reinforcement Learning (DERL): a computational framework which can evolve diverse agent morphologies to learn challenging locomotion and manipulation tasks in complex environments. Leveraging DERL we demonstrate several relations between environmental complexity, morphological intelligence and the learnability of control. First, environmental complexity fosters the evolution of morphological intelligence as quantified by the ability of a morphology to facilitate the learning of novel tasks. Second, we demonstrate a morphological Baldwin effect i.e., in our simulations evolution rapidly selects morphologies that learn faster, thereby enabling behaviors learned late in the lifetime of early ancestors to be expressed early in the descendants lifetime. Third, we suggest a mechanistic basis for the above relationships through the evolution of morphologies that are more physically stable and energy efficient, and can therefore facilitate learning and control.

Automated summary

The new framework of deep evolutionary reinforcement learning evolves agents with diverse morphologies to learn difficult tasks in complex environments, revealing insights into the relationships between environmental physics, embodied intelligence, and rapid learning evolution. Research indicates that environmental complexity fosters the evolution of morphological intelligence, and that more physically stable and energy efficient morphologies facilitate learning and control. Additionally, a morphological Baldwin effect is demonstrated, where evolution rapidly selects morphologies that learn faster, impacting the behaviors learned and expressed throughout generations.