细粒度语义增强的民航图文跨模态检索方法OA

In the fields of low-altitude economy and civil aviation,security assurance tasks heavily rely on the efficient correlation of cross-modal information such as images and texts.However,while mainstream cross-modal retrieval models perform well on general datasets,they underperform in these areas,which require high levels of fine-grained semantic understanding.Based on existing civil aviation datasets,a cross-modal retrieval approach with fine-grained semantic augmentation is suggested as a solution to this problem,creating a whole pipeline that includes data processing,model development,and training.First,text descriptions are optimized and enhanced based on a large multimodal model to construct a cross-modal retrieval dataset containing rich semantic information.Second,a model is created using a popular cross-modal retrieval framework.To improve the model's ability to express fine-grained semantic features,techniques such class supervision,key semantic information masking,and a fine-grained feature extraction module are introduced.Experimental results on two datasets verify the effectiveness of the proposed method,providing a reference technical path for cross-modal retrieval in low-altitude economy scenarios.