适用于内容安全审核的图像文本识别质量评估方法OA

A quality assessment method based on the consistency confidence(CC)mechanism was proposed to ad-dress the challenge of image text recognition quality assessment in content security auditing scenarios.Significant deficiencies were identified in existing evaluation methods.Traditional methods were found to rely on manual an-notation and be cost-prohibitive.The visual language model as judge(VLM-as-Judge)approach was observed to suffer from logical paradoxes and biases.Uncertainty quantification methods were shown to require access to inter-nal model parameters.With the proposed CC-based method,unsupervised quality assessment was achieved through multi-model consistency analysis.Edit distance was utilized to quantify the degree of consistency,and dynamic weighting with threshold gating mechanisms was designed to identify low-quality outputs.This method possesses three major advantages:black-box evaluation,complete unsupervisedness,and model-agnostic property.Experi-mental results demonstrated that the quality verification F1 score reached 49.9%～51.61%,which represented a 42.76%improvement over baseline methods.The best performance was exhibited in challenging scenarios such as adversarial text and complex scenes.Through model ensemble,low-cost open-source models were enabled to sur-pass high-cost closed-source models,with a performance improvement of 32 points.