基于广义指代表达分割的缺陷检测和行为识别方法研究OA

In the field of power vision,existing defect detection methods based on deep learning usually take a single image modality as input,resulting in poor accuracy for complex defect types.Additionally,most of the existing defect detection methods based on image-text fusion target one or several specific types of defects and cannot be effectively promoted.This paper proposes a defect detection problem modeling method based on generalized referring expression segmentation.It forms a complete defect detection dataset construction process by building an ontology library,a professional knowledge base,and the mapping relationship between the two.Furthermore,the feasibility of this method is verified on the annotated dataset through an image-text fusion model based on relationship modeling.Experiments show that the image-text fusion model trained under the generalized referring expression segmentation framework achieves an average precision and recall rate of 80%for the recognition of 4 typical targets in the ladder operation scenario,which is significantly better than the visual model trained under the target detection framework.Moreover,the method proposed in this paper also has significant advantages in aspects such as fine-grained detection and interpretability.