基于多层Transformer的细粒度图像-文本检索算法OA

Most traditional deep learning-based image-text retrieval algorithms use CNN and RNN networks to encode image and text data.Such methods cannot fully exploit the fine-grained interaction information across modalities.To address this problem,a Transformer-based image-text retrieval network is proposed.It models the fine-grained interactions within and between modalities through two multilayer Transformer modules respectively,while using a CCA-based feature fusion method to obtain a higher quality feature representation.Finally it maximizes the relevance of paired samples through a feature similarity matrix.Comparative experi-ments conducted on two benchmark datasets,Wikipedia and Pascal Sentence,validate the effectiveness of the paper's approach.