融合多子空间及通道注意力的多模态情感分析OA

Current multimodal sentiment analysis primarily relies on complex techniques for fusing multimodal features.However,due to the significant distribution differences among various modal features,direct fusion yields poor results.To address this issue,this paper proposes an interactive learning network model that integrates a multi-subspace framework and channel attention.Firstly,a hybrid neural network is utilized to extract features from each modality,and a stacked bidirectional long short-term memory network is employed to represent the utterance sequence at the linguistic level.Fixed-size utterance vectors are mapped into two different representations:modal-invariant and modal-specific,with the latter undergoing bimodal interaction using a temporal convolutional network.Subsequently,channel attention is leveraged to extract more meaningful information,and a cross-modal interactive bidirectional gated recurrent neural network and a bimodal interactive attention mechanism are proposed for deeper interaction among the extracted modal-invariant representation vectors.Loss optimization is then performed using a loss function.Finally,a multi-head attention mechanism based on Transformer is executed to obtain a joint vector,and a fully connected layer is utilized to predict the final result.Experiments conducted on the CMU-MOSI and CMU-MOSEI datasets demonstrate that this method can effectively eliminate multimodal differences and achieve multimodal fusion.