DGS-CapNet:基于空间-频率感知的SAR图像描述模型OA
DGS-CapNet:A Spatial-frequency-aware Model for SAR Image Captioning
合成孔径雷达(SAR)作为一种主动微波遥感系统,具有"全天时、全天候"的观测能力,在灾害监测、城市管理及军事侦察等领域发挥着重要应用价值.尽管深度学习技术已推动SAR图像解译取得了显著进展,但现有目标识别与检测方法多聚焦于局部特征提取与单一目标判别,难以全面刻画复杂场景的整体语义结构与多目标关系,且解译流程仍高度依赖专业人员,自动化水平有限.SAR图像描述旨在将视觉信息转化为自然语言,是从"感知目标"向"认知场景"跨越的关键技术,对于提升SAR图像解译的自动化与智能化水平具有重要意义.然而,SAR图像固有的相干斑噪声干扰、纹理细节匮乏、语义鸿沟显著进一步加剧了跨模态理解的难度.针对上述问题,该文提出一种基于空间-频率感知的SAR图像描述方法(DGS-CapNet).首先,构建空间-频域感知模块,利用离散余弦变换(DCT)掩码注意力机制对频谱成分加权以抑制噪声并强化结构特征,同时结合Gabor多尺度纹理增强模块提升对方向与边缘细节的感知能力;其次,设计跨模态语义增强损失函数,通过双向对比损失与最大互信息损失,有效缩减视觉特征与自然语言间的语义鸿沟.此外,我们还构建了包含72400条高质量图文对的大规模细粒度SAR图像描述数据集FSAR-Cap.实验结果表明,该方法在SARLANG和FSAR-Cap数据集上的CIDEr指标分别达到151.00和95.14.定性分析表明,该模型有效抑制了幻觉,并准确捕捉了细粒度的空间纹理细节,显著优于主流方法.
Synthetic Aperture Radar(SAR),as an active microwave remote sensing system,offers all-weather,all-day observation capabilities and has considerable application value in disaster monitoring,urban management,and military reconnaissance.Although deep learning techniques have achieved remarkable progress in interpreting SAR images,existing methods for target recognition and detection primarily focus on local feature extraction and single-target discrimination.They struggle to comprehensively characterize the global semantic structure and multitarget relationships in complex scenes,and the interpretation process remains highly dependent on human expertise with limited automation.SAR image captioning aims to translate visual information into natural language,serving as a key technology to bridge the gap between"perceiving targets"and"cognizing scenes,"which is of great importance for enhancing the automation and intelligence of SAR image interpretation.However,the inherent speckle noise,the scarcity of textural details,and the substantial semantic gap in SAR images further exacerbate the difficulty of cross-modal understanding.To address these challenges,this paper proposes a spatial-frequency aware model for SAR image captioning.First,a spatial-frequency aware module is constructed.It employs a Discrete Cosine Transform(DCT)mask attention mechanism to reweight spectral components for noise suppression and structure enhancement,combined with a Gabor multiscale texture enhancement submodule to improve sensitivity to directional and edge details.Second,a cross-modal semantic enhancement loss function is designed to bridge the semantic gap between visual features and natural language through bidirectional image-text alignment and mutual information maximization.Furthermore,a large-scale fine-grained SAR image captioning dataset,FSAR-Cap,containing 72400 high-quality image-text pairs,is constructed.The experimental results demonstrate that the proposed method achieves CIDEr scores of 151.00 and 95.14 on the SARLANG and FSAR-Cap datasets,respectively.Qualitatively,the model effectively suppresses hallucinations and accurately captures fine-grained spatial-textural details,considerably outperforming mainstream methods.
张金琪;庄迪;张腊梅;邹斌;董洪伟;司凌宇;孟庆彪;吴有明
哈尔滨工业大学电子与信息工程学院 哈尔滨 150001哈尔滨工业大学电子与信息工程学院 哈尔滨 150001哈尔滨工业大学电子与信息工程学院 哈尔滨 150001哈尔滨工业大学电子与信息工程学院 哈尔滨 150001中国科学院软件研究所天基综合信息系统全国重点实验室 北京 100190中国科学院软件研究所天基综合信息系统全国重点实验室 北京 100190中国科学院空天信息创新研究院 北京 100190中国科学院空天信息创新研究院 北京 100190
信息技术与安全科学
SAR图像描述空间-频域感知DCT掩码注意力多尺度纹理增强跨模态对齐图像-文本数据集
SAR image captioningSpatial-frequency-awarenessDCT mask attentionMulti-scale texture enhancementCross-modal alignmentImage-text dataset
《雷达学报》 2026 (2)
441-462,22
国家自然科学基金(62271172)The National Natural Science Foundation of China(62271172)
评论