PE-DLS: a novel method for performing real-time full-body motion reconstruction in VR based on Vive trackers

Real-time full-body motion capture (MoCap) is becoming necessary for enabling natural interactions and creating deeper immersion in virtual reality (VR). To reduce the cost and complexity of MoCap systems, some studies attempt to track only the joint data of root and end effectors and reconstruct full-body motion by solving inverse kinematics (IK) problems. However, ensuring the accuracy of full-body motion reconstruction in real-time is challenging because the problem is inherently under-constrained. In this paper, we propose PE-DLS, a novel method to perform full-body motion reconstruction in two stages: pose estimation (PE) and damped least squares (DLS) optimization. First, we use analytical IK solvers to estimate the spine and limbs in sequence. To further improve model accuracy, we use the DLS method to optimize the results obtained from PE. To evaluate the model performance, we compare it with other methods in terms of the reconstruction error and computational time of full-body reconstruction via testing on publicly available datasets. These results indicate that PE-DLS outperforms other methods in terms of the mean per joint position error (2.11 cm) and mean per joint rotation error (10.75 degrees) with low time cost (1.65 ms per frame). Furthermore, we implement a full-body MoCap system based on an HTC Vive headset and five Vive trackers. Live demos and qualitative comparisons show that our system achieves comparable quality to the commercial MoCap system. With high accuracy and low time cost, PE-DLS contributes to construct a real-time MoCap system in VR.

Automated summary

This paper proposes a method called PE-DLS for full-body motion reconstruction in two stages: pose estimation (PE) and damped least squares (DLS) optimization. The method outperforms others in terms of reconstruction error and computational time, and a full-body MoCap system based on Vive devices is implemented.