A Multi-Stage, Multi-Feature Machine Learning Approach to Detect Driver Sleepiness in Naturalistic Road Driving Conditions

Driver fatigue is a contributing factor in about 20% of all fatal road crashes worldwide. Countermeasures are urgently needed and one of the most promising and currently available approaches for that are in-vehicle systems for driver fatigue detection. The main objective of this paper is to present a video-based driver sleepiness detection system set up as a two-stage model with (1) a generic deep feature extraction module combined with (2) a personalised sleepiness detection module. The approach was designed and evaluated using data from 13 drivers, collected during naturalistic driving conditions on a motorway in Sweden. Each driver performed one 90-minute driving session during daytime (low sleepiness condition) and one session during night-time (high sleepiness condition). The sleepiness detection model outputs a continuous output representing the Karolinska Sleepiness Scale (KSS) scale from 1-9 or a binary decision as alert (defined as KSS 1-6) or sleepy (defined as KSS 7-9). Continuous output modelling resulted in a mean absolute error (MAE) of 0.54 KSS units. Binary classification of alert or sleepy showed an accuracy of 92% (sensitivity = 91.7%, specificity = 92.3%, F1 score = 90.4%). Without personalisation, the corresponding accuracy was 72%, while a standard fatigue detection PERCLOS-based baseline method reached an accuracy of 68% on the same dataset. The developed real-time sleepiness detection model can be used in the management of sleepiness/fatigue by detecting precursors of severe fatigue, and ultimately reduce sleepiness-related road crashes by alerting drivers before high levels of fatigue are reached.

Automated summary

A two-stage model for video-based driver sleepiness detection achieved a sensitivity of 91.7%, specificity of 92.3%, and F1 score of 90.4%. In comparison, the accuracy of the method without personalisation was only 72%.