基于图像特征优化与自监督学习的视觉问答模型OA

To tackle the challenges of insufficient image feature understanding and data bias in current visual question ans-wering(VQA)models,this paper proposes a novel VQA model featuring a dual visual feature encoder,text encoder,multi-modal feature fusion,and multimodal feature decoder,where the dual visual encoders extract multi-granularity features during question reasoning to address inadequate visual feature comprehension.In training,masked image reconstruction is employed to strengthen the correlation between local features and global structures,while generating highly irrelevant negative samples forces the model to learn true semantic associations between images and questions.The two-stage self-supervised training guides the model to deepen feature understanding from"pixel-level reconstruction"to"semantic-level alignment",optimizing image feature extraction and bias suppression.Experiments demonstrate that the model achieves 65.74%,67.50%,and 61.86%accuracies on VQA-CPv2,VQAv2,and OK-VQA datasets,surpassing state-of-the-art methods,with ablation studies and visual analyses validating the effectiveness of its modules.This work provides an effective approach for enhancing VQA models'generalization through rational architectural design and self-supervised training.