可微分神经网络架构搜索综述OA

Neural Architecture Search(NAS)was an interdisciplinary study in the field of deep learning,which aimed to automate the design of neural network structures.NAS required repeated training and evaluation of a large number of candidate networks,which was computationally expensive.Differentiable Neural Architecture Search(DNAS)transformed the discrete architecture search problem into a differentiable continuous optimization problem,which reduced the computational cost.Firstly,a search algorithm framework for differentiable network architecture was constructed from three aspects:search space,search strategy and performance evaluation strategy.Secondly,the performance estimation bias,architecture overfitting and search stability problems of parameterization operation,as well as the improvement strategies of optimizing search space and improving efficiency were analyzed,compared and summarized.Then,the error rate,parameter quantity,search time and experimental hardware conditions of typical DNAS algorithms on image classification datasets were compared and analyzed.Finally,it pointed out the application potential of DNAS in complex scenarios such as edge device deployment,medical signal analysis,and cross-modal matching,and proposed future research directions toward multi-objective optimization,task-driven search space design,and cross-task transfer and reuse.