Detection and Analysis of Human Mental State Based on Multimodal Information - Luo, Jiahao; Wang, Zicheng (Highlights in Science, Engineering and Technology, 2026)
With the continuous evolution and refinement of intelligent sensing technologies, the development and application of multimodal intelligent information in detecting human mental states have become a key direction in the advancement of intelligent technologies, surpassing traditional unimodal detection methods. Multimodal detection technology that integrates multiple information sources to reflect human mental states has been widely applied in fields such as medical care and interrogation. However, existing detection methods generally have disadvantages such as low precision and detection equipment being susceptible to environmental influences. Based on this background, this paper explores a framework for detecting and analyzing human mental states based on multimodal information. This framework integrates multi-source data, including non-contact sensing, contact-based physiological signal acquisition, and emerging dataset construction. This framework combines feature level, decision-making-level level and end-to-end multimodal fusion methods to achieve the deep synergy of cross-modal features. It enables the coordinated utilization of multiple human signal technologies, establishing a comprehensive data analysis framework for multimodal feature fusion. The rational use of this framework can significantly improve the robustness of the detection method. This study proposes innovative improvement solutions for applications in scenarios such as medical monitoring and judicial interrogations.