SuperTrack: Motion Tracking for Physically Simulated Characters using Supervised Learning

In this paper we show how the task of motion tracking for physically simulated characters can be solved using supervised learning and optimizing a policy directly via back-propagation. To achieve this we make use of a world model trained to approximate a specific subset of the environment's transition function, effectively acting as a differentiable physics simulator through which the policy can be optimized to minimize the tracking error. Compared to popular model-free methods of physically simulated character control which primarily make use of Proximal Policy Optimization (PPO) we find direct optimization of the policy via our approach consistently achieves a higher quality of control in a shorter training time, with a reduced sensitivity to the rate of experience gathering, dataset size, and distribution.

Automated summary

This paper demonstrates a method for motion tracking of physically simulated characters using supervised learning and optimizing the policy directly. By training a world model to approximate a specific subset of the environment's transition function, the policy can be optimized to minimize tracking error. Compared to popular model-free methods, this approach consistently achieves higher quality control in a shorter training time with reduced sensitivity to experience gathering rate, dataset size, and distribution.