基于离线强化学习的自动协商方法OA

Automated negotiation is a key approach to achieving cooperation and collaboration in multi-agent systems.While Rein-forcement Learning(RL)-based negotiating agents have attained remarkable success across various scenarios,they still face constraints imposed by real-world implementation environments.In particular,these agents require extensive online interactions with opponents for training,which is often infeasible and unrealistic in practical applications.Therefore,a novel method is needed to enable the learning of effective negotiation strategies directly from offline datasets.Additionally,during subsequent online negotiations,opponents may al-ter their strategies due to various factors—such as changes in risk attitudes or market conditions—posing significant challenges to auto-mated negotiation.In this work,we propose a new negotiating agent that enhances performance via offline-to-online RL.The proposed agent is able ① to interact with opponents using an RL-based strategy to improve its adaptability to dynamic negotiation environments;② to learn negotiation strategies from historical offline data without the need for extensive online active interactions;and ③ to optimize the online fine-tuning process to facilitate rapid and stable performance improvements of the pre-learned offline strategies.Extensive experi-mental results are presented,based on multiple negotiation scenarios and winning agents from recent Automated Negotiating Agents Com-petitions(ANAC).The results demonstrate that the proposed agent outperforms state-of-the-art alternatives and remains effective even when opponents switch to different strategies.