UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture

We present UnrealEgo, i.e., a new large-scale naturalistic dataset for egocentric 3D human pose estimation. UnrealEgo is based on an advanced concept of eyeglasses equipped with two fisheye cameras that can be used in unconstrained environments. We design their virtual prototype and attach them to 3D human models for stereo view capture. We next generate a large corpus of human motions. As a consequence, UnrealEgo is the first dataset to provide in-the-wild stereo images with the largest variety of motions among existing egocentric datasets. Furthermore, we propose a new benchmark method with a simple but effective idea of devising a 2D keypoint estimation module for stereo inputs to improve 3D human pose estimation. The extensive experiments show that our approach outperforms the previous state-of-the-art methods qualitatively and quantitatively. UnrealEgo and our source codes are available on our project web page (https://4dqvanpiinf.mpg.def/UnrealEgo/).

Automated summary

UnrealEgo is a new large-scale naturalistic dataset for egocentric 3D human pose estimation in stereo environments. It utilizes an innovative concept of eyeglasses equipped with fisheye cameras and provides the widest variety of human motions among existing egocentric datasets. Additionally, a novel benchmark method involving a 2D keypoint estimation module for stereo inputs is proposed to enhance 3D human pose estimation performance.