Assessment of the unified model of performance: accuracy of group-average and individualised alertness predictions

To be effective as a key component of fatigue-management systems, biomathematical models that predict alertness impairment as a function of time of day, sleep history, and caffeine consumption must demonstrate the ability to make accurate predictions across a range of sleep-loss and caffeine schedules. Here, we assessed the ability of the previously reported unified model of performance (UMP) to predict alertness impairment at the group-average and individualised levels in a comprehensive set of 12 studies, including 22 sleep and caffeine conditions, for a total of 301 unique subjects. Given sleep and caffeine schedules, the UMP predicted alertness impairment based on the psychomotor vigilance test (PVT) for the duration of the schedule. To quantify prediction performance, we computed the root mean square error (RMSE) between model predictions and PVT data, and the fraction of measured PVTs that fell within the models' prediction intervals (PIs). For the group-average model predictions, the overall RMSE was 43 ms (range 15-74 ms) and the fraction of PVTs within the PIs was 80% (range 41%-100%). At the individualised level, the UMP could predict alertness for 81% of the subjects, with an overall average RMSE of 64 ms (range 32-147 ms) and fraction of PVTs within the PIs conservatively estimated as 71% (range 41%-100%). Altogether, these results suggest that, for the group-average model and 81% of the individualised models, in three out of four PVT measurements we cannot distinguish between study data and model predictions.

Automated summary

This study assessed the ability of biomathematical models to predict alertness impairment and found that these models have high accuracy in predicting alertness decline at both the group-average and individual levels. The results indicate that in three out of four psychomotor vigilance tests, it is difficult to distinguish between study data and model predictions.