Reinforcement Learning-Based Cascade Motion Policy Design for Robust 3D Bipedal Locomotion

This paper presents a novel reinforcement learning (RL) framework to design cascade feedback control policies for 3D bipedal locomotion. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint or task space trajectories. Unlike these studies, we propose a policy structure that decouples the bipedal locomotion problem into two modules that incorporate the physical insights from the nature of the walking dynamics and the well-established Hybrid Zero Dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, sample efficiency, and less dependence on prior knowledge. The proposed solution learns stable and robust walking gaits from scratch and allows the controller to realize omnidirectional walking with accurate tracking of the desired velocity and heading angle. The learned policies also perform robustly against various adversarial forces applied to the torso and walking blindly on a series of challenging and unstructured terrains. These results demonstrate that the proposed cascade feedback control policy is suitable for navigation of 3D bipedal robots in indoor and outdoor environments.

Automated summary

This paper presents a novel reinforcement learning framework for designing cascade feedback control policies for 3D bipedal locomotion. The proposed solution decouples the locomotion problem into two modules, incorporating physical insights from walking dynamics and the Hybrid Zero Dynamics approach. The framework offers advantages such as lightweight network structure, sample efficiency, and reduced dependence on prior knowledge.