基于面部图像和心率变异性的多模态融合情感识别算法OA

To address the challenges of ineffective emotion recognition in virtual sports,this study proposes an al-gorithm of Multimodal Fusion Emotion Recognition based on Face Image and Heart Rate Variability(MFER-FI-HRV).By capturing and analyzing data from both facial image and heart rate variability modalities,the proposed approach enables a fine-grained perception of users'emotional changes,thereby facilitating a personalized Human-Computer Interaction(HCI)experience tailored to their current emotional state.The algorithm first employs a multi-modal fusion Transformer to achieve complementary learning between facial images and HRV data.Then,a multimo-dal feature fusion strategy is utilized to concatenate fused features with the original feature representations.Addition-ally,a lightweight self-attention mechanism is introduced to capture high-level semantic representations within the multimodal data.Extensive experiments on two public datasets demonstrate the superior performance of the proposed method,confirming its effectiveness and robustness.These findings provide valuable insights for the design and de-velopment of user experience systems.