Three-dimensional cameras and skeleton pose tracking for physical function assessment: A review of uses, validity, current developments and Kinect alternatives

Background: Three-dimensional camera systems that integrate depth assessment with traditional two-dimensional images, such as the Microsoft Kinect, Intel Realsense, StereoLabs Zed and Orbecc, hold great promise as physical function assessment tools. When combined with point cloud and skeleton pose tracking software they can be used to assess many different aspects of physical function and anatomy. These assessments have received great interest over the past decade, and will likely receive further study as the integration of depth sensing and augmented reality smartphone cameras occurs more in everyday life. Research Question: The aim of this review is to discuss how these devices work, what options are available, the best methods for performing assessments and how they can be used in the future. Methods: Firstly, a review of the Microsoft Kinect devices and associated artificial intelligence, automated skeleton tracking algorithms is provided. This includes a narrative critique of the validity and clinical utility of these devices for assessing different aspects of physical function including spatiotemporal, kinematic and inverse dynamics data derived from gait and balance trials, and anatomical assessments performed using the depth sensor information. Methods for improving the accuracy of data are examined, including multiple-camera systems and sensor fusion with inertial monitoring units, model fitting, and marker tracking. Secondly, alternative hardware, including other structured light and time of flight methods, stereoscopic cameras and augmented reality leveraging smartphone and tablet cameras to perform measurements in three-dimensional space are summarised. Software options related to depth sensing cameras are then discussed, focussing on recent advances such as OpenPose and web-based methods such as PoseNet. Results and Significance: The clinical and non-laboratory utility of these devices holds great promise for physical function assessment, and recent developments could strengthen their ability to provide important and impactful health-related data.