Multimodal Emotion Recognition Empowers Fatigue Driving Detection

Abstract

The probability of traffic accidents caused by fatigue is not small, so the reliable fatigue reminder device is an important way to ensure driving safety. Therefore, this paper introduces six kinds of multimodal fatigue driving detection techniques with research significance: (1) a method based on heart rate and Percentage of Eyelid Closure(PERCLOS); (2) a method based on hybrid electroencephalography(EEG) and eye tracking; (3) attention-based approach; (4) a method based on multimodal feature coupled mode; (5) a method based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) architecture; (6) a non-invasive method based on heart rate, eye and facial features. Instead of simply listing the existing methods, this paper systematically analyzes the performance of these methods in terms of accuracy, robustness, hardware cost and practical deployment feasibility. The results show that the accuracy of most models is more than 96%. In addition, this paper also critically discusses the main limitations of current methods, and proposes corresponding improvement directions. Finally, this article discusses the application prospects of these technologies in the automotive and transportation fields and clarifies the advantages and disadvantages of these six technologies. This article provides a reference for designing low-cost and easy-to-use in vehicle fatigue driving detection solutions in the field of automotive, bus, aircraft and commercial vehicle fleet. It is helpful to promote the practical application of fatigue detection technology in vehicles and aircraft, thereby improving road safety and reducing the occurrence of fatigued driving accidents.