Comprehensive Drowsiness Level Detection Model Combining Multimodal Information

This paper presents a drowsiness detection model that is capable of sensing the entire range of stages of drowsiness, from weak to strong. The key assumption underlying our approach is that the sitting posture-related index can indicate weak drowsiness that drivers themselves do not notice. We first determined the sensitivity of the posture index and conventional indices for the stages of drowsiness. Then, we designed a drowsiness detection model combining several indices sensitive to weak drowsiness and to strong drowsiness, to cover all drowsiness stages. Subsequently, the model was trained and evaluated on a dataset comprised of data collected from approximately 50 drivers in simulated driving experiments. The results indicated that posture information improved the accuracy of weak drowsiness detection, and our proposed model using the driver's blink and posture information covered all stages of drowsiness (F1-score 53.6%, root mean square error 0.620). Future applications of this model include not only warning systems for dangerously drowsy drivers but also systems which can take action before their drivers become drowsy. Since measuring the information requires no restrictive equipment such as on-body electrodes, the model presented here based on blink and posture information can be used in several practical applications.