知识增强与协同推理的视觉常识推理方法OA

Visual Commonsense Reasoning(VCR)aims to enable models to perform human-like cognitive reasoning based on images.Although existing methods primarily rely on large-scale cross-modal pre-trained models for implicit cross-modal modeling,they still fall short in terms of external knowledge incorporation,contextual modeling,and the consistency between answers and rationales.To address these limitations,the paper propose a knowledge-augmented collaborative reasoning framework for VCR.The proposed frame-work introduces a dual-layer knowledge injection mechanism comprising scene commonsense graphs and re-gional factual descriptions.By explicitly integrating scene-level commonsense with entity-level visual facts,this mechanism mitigates knowledge deficiency and semantic ambiguity.Meanwhile,a dual-task collabora-tive learning framework based on large language models is designed to jointly optimize question answering and rationale inference.A cognitive alignment loss is further employed to enhance the logical consistency be-tween the two tasks in the latent space.Experimental results on the VCR benchmark demonstrate that the proposed framework effectively improves model reasoning performance.Notably,it outperforms state-of-the-art baselines by 4.2 percentage points on the joint Q→AR task,thereby validating the effectiveness of the knowledge-augmented collaborative reasoning strategy.