基于正则化5×2交叉验证投票的对抗验证方法OA

In machine learning,an adversarial validation method typically employs cross-validation to estimate the performance of an ad-versarial classification model and compares it with random guessing for inferring the distribution equivalence between the training and the test sets.On the basis of the prediction probability estimators with regard to adversarial labels obtained from cross-validation,the method then selects a subset of samples from the training set that closely matches the distribution of the test set to serve as a validation set,thereby enhancing the generalization ability of a machine learning model on the test set.Thus,cross-validation estimation plays a core role in an adversarial validation method.However,the existing adversarial validation methods use an averaged aggregation to produce cross-validated estimators.Although the averaged aggregation can reduce the variance in the cross-validated estimator of the performance of an adversarial classification model,it is incapable to enlarge the difference between the expected error rates of an adversarial classification model and random guessing,and thus it constrains the performance of an adversarial validation in the inference of the distribution equivalence and the selection of a validation set.Therefore,we propose a regularized 5×2 cross-validated voting estimator and develop a corresponding McNemar's test statistic as well as several types of prediction probability estimation to form a novel adversarial validation method.Experimental results on 5 commonly-used classification algorithms and 18 datasets demonstrate that the proposed method achieves promising performance in both distribution equivalence inference and validation set selection.