Individual versus Group Calibration of Machine Learning Models for Physical Activity Assessment Using Body-Worn Accelerometers

Modeling approaches for translating accelerometer data into physical activity metrics are often developed using a group calibration approach. However, it is unknown if models developed for specific individuals will improve measurement accuracy. Purpose: Wesought to determine if individually calibratedmachine learningmodels yielded higher accuracy than a group calibration approach for physical activity intensity assessment. Methods: Participants (n = 48) wore accelerometers on the right hip and nondominant wrist while performing activities of daily living in a semistructured laboratory and/or free-living setting. Criterion measures of activity intensity (sedentary, light, moderate, vigorous) were determined using direct observation. Data were reintegrated into 30-s epochs, and eight random forest models were created to determine physical activity intensity by using all possible conditions of training data (individual vs group), protocol (laboratory vs free-living), and placement (hip vswrist). A2 x 2 x 2 repeated-measures analysis of variance was used to compare epoch-level accuracy statistics (% accuracy, kappa [kappa]) of the models when used to determine activity intensity in an independent sample of free-living participants. Results: Main effects were significant for the type of training data (group: accuracy = 80%, kappa = 0.59; individual: accuracy = 74% [P = 0.02], kappa = 0.50 [P = 0.01]) and protocol (free-living: accuracy = 81%, kappa = 0.63; laboratory: accuracy = 74% [P = 0.04], kappa = 0.47 [P < 0.01]). Main effects were not significant for placement (hip: accuracy = 79%, kappa = 0.58; wrist: accuracy = 75% [P = 0.18]; kappa = 0.52 [P = 0.18]). Point estimates for mean absolute error were generally lowest for the group training, free-living protocol, and hip placement. Conclusions: Contrary to expectations, individually calibrated machine learning models yielded poorer accuracy than a traditional group approach. In addition, models should be developed in free-living settings when possible to optimize predictive accuracy.

Automated summary

The study found that individually calibrated machine learning models yielded poorer accuracy compared to traditional group approaches, and models should be developed in free-living settings when possible to optimize predictive accuracy.