首页|期刊导航|现代信息科技|融合双线性二部图注意力的场景图问答方法研究

融合双线性二部图注意力的场景图问答方法研究OA

Research on a Scene Graph Question Answering Method Integrating Bilinear Bipartite Graph Attention

中文摘要英文摘要

场景图将图像表示为结构化语义图,其中节点表示对象及其属性,边表示对象间关系,从而丰富视觉信息表达.场景图生成研究已取得显著进展,但基于场景图的问答任务仍待深入探索.为此,构建一种双线性二部图注意力网络(BB-GAN)框架,能够根据问题内容从场景图中筛选最相关节点.首先将问题编码为全连接图,并利用图注意力网络进行信息迭代更新;随后构建语义结构图与场景图的二部图,通过 BB-GAN 实现节点匹配,并引入模块化注意力机制增强模态对齐;自适应融合全局场景特征、关键词节点特征和问题特征进行答案预测.在 GQA 数据集上的实验表明,本方法在二元问题上显著优于现有方法,消融实验进一步验证了BB-GAN的有效性.

Scene graphs represent images as structured semantic graphs,where nodes denote objects and their attributes and edges denote relationships between objects,thereby enriching visual information representation.Research on scene graph generation shows remarkable progress,but question answering based on scene graphs still requires further exploration.To this end,this study constructs a Bilinear Bipartite Graph Attention Network(BB-GAN)framework that selects the most relevant nodes from the scene graph based on the question content.First,the question is encoded into a fully connected graph,and a graph attention network is used to iteratively update the information.Subsequently,a bipartite graph between the semantic structure graph and the scene graph is constructed,BB-GAN is used for node matching,and a modular attention mechanism is introduced to enhance modality alignment.Global scene features,keyword node features,and question features are adaptively fused for answer prediction.Experiments on the GQA dataset demonstrate that the proposed method significantly outperforms existing approaches on binary questions.Ablation studies further validate the effectiveness of BB-GAN.

加东东;薛羽馨;张兆馨;李天宇

延安大学 物理与电子信息学院,陕西 延安 716000||陕西省能源大数据智能处理省市共建重点实验室,陕西 延安 716000延安大学 物理与电子信息学院,陕西 延安 716000||陕西省能源大数据智能处理省市共建重点实验室,陕西 延安 716000延安大学 物理与电子信息学院,陕西 延安 716000延安大学 物理与电子信息学院,陕西 延安 716000

信息技术与安全科学

场景图问答双线性二部图注意力网络图注意力网络模态对齐

scene graph question answeringbilinear bipartite graph attention networkgraph attention networkmodality alignment

《现代信息科技》 2026 (15)

34-38,42,6

10.19850/j.cnki.2096-4706.2026.15.007

评论