结合贝叶斯优化与强化学习的序贯配方优化设计OA

Bayesian Optimization(BO)is a powerful tool for sequential optimal design in formulation development.However,conventional BOis sensitive to initial samples,easily falls into a local optimum,and cannot dynamically adapt to changes in the process easily.By combining the global search capability of Reinforcement Learning(RL)with the efficient exploration strategy of BO,this study proposes a sequential formulation optimization design method based on a Deep Deterministic Policy Gradient(DDPG)to achieve adaptive formulation optimization.By leveraging real-time feedback from the data,this approach dynamically adjusts the optimization strategy,enabling efficient global optimization starting from random initial points.It effectively addresses challenges such as limited data,inaccuracies in surrogate models,and measurement noise.During the optimization process,the method employs a Gaussian Process(GP)to construct a surrogate model of the objective function and adopts an inner-outer dual-loop framework to enhance optimization efficiency.In the inner loop,the actor network generates candidate actions,and the Upper Confidence Bound(UCB)strategy is used to select optimal experimental points.In the outer loop,the experimental strategy is dynamically adjusted based on historical experience,and the surrogate model is updated using new observations.The results from simulation cases and wine blending experiments demonstrate that the proposed method outperforms traditional optimization methods in terms of optimization efficiency,convergence speed,and quality of the final solution.Moreover,it exhibits a strong generalization ability in new environments,providing a novel solution for complex formulation optimization problems.