基于自监督辅助CLIP与Transformer自注意力的提示引导式弱监督语义分割方法OA

To address the problem that traditional weakly supervised semantic segmentation methods relying on image-level annotations of target datasets still fail to obtain fine-grained spatial supervision signals,this paper pro-poses a self-supervised contrastive language-image pre-training prompt-guided method that requires no target do-main annotations.This paper mainly tackles three key issues:First,the lack of pixel-level spatial constraints in tra-ditional supervision methods;second,the limitation of initial pseudo-labels due to the propagation bias of classifica-tion features;third,the semantic mismatch between pre-trained features and segmentation tasks.In response to these problems,three corresponding solutions are proposed.1)A self-supervised prompt learning mechanism is de-signed to replace traditional fixed text templates.By using cross-modal attention to dynamically generate learnable prompt vectors containing image context.2)A dynamic clipping strategy based on loss response gradients is put forward.It filters out high-gradient noise regions by utilizing the"early memory effect"of deep neural networks,thus optimizing the boundary accuracy of pseudo-labels.3)A class-aware affinity graph based on Transformer self-attention is constructed to enhance the spatial consistency of segmentation results.Experimental results on PAS-CAL VOC 2012 and MS COCO 2014 datasets show that,under the setting without target domain annotations,the proposed method outperforms existing label-free weakly supervised semantic segmentation methods in terms of mI-oU metric,and has significant advantages in boundary preservation and background suppression.