反向最大化学习:反蒸馏场景下的暗知识清除OA

Knowledge distillation is a technique that enhances the performance of a student model by transferring the rep-resentational capabilities learnt by a teacher model.However,with the development of data-free knowledge distillation,this technology may also be exploited for unauthorized model replication or enhancement,posing risks of intellectual property infringement and challenges to model security.To address this issue,anti-distillation techniques have emerged,aiming to remove the dark knowledge from the teacher model,thereby preventing potential adversaries from extracting the feature representation capabilities of the teacher model through knowledge distillation.Current research mainly focuses on generating multi-peak outputs to confuse the student model during knowledge distillation,thereby mitigating infringe-ment in knowledge distillation.However,this approach may introduce noise during the training of anti-distillation models,potentially degrading their performance.To tackle this problem,this paper proposes a novel method termed reversed maxi-mization learning(RML),which aims to train an anti-distillation model that preserves the original model's performance while reducing the potential risks associated with knowledge distillation.The proposed method decouples the positive and negative classes in the output of original model through binary probability mechanism.Meanwhile,it employs reverse-ranking module to learn the inverted negative-class outputs of the original model,thereby eliminating confidence differences among negative classes in the anti-distillation model to disrupt knowledge distillation,while preserving the confi-dence advantage of the positive class to maintain the original performance.Extensive experiments on datasets such as Cifar-100 and ImageNet200,as well as across various model architectures,demonstrate that the proposed method achieves significant effectiveness in both data-driven and data-free knowledge distillation scenarios,outperforming comparison approaches.