基于大模型的世界模型研究综述(特邀)OA

World models are generally believed to understand and represent the external world and predict future states based on current world states and actions.Large models leverage massive training data and vast parameter scales to exhibit outstanding capabilities in learning,understanding,representing,and generating textual knowledge,as exemplified by language large models such as GPT-4 and LLaMA.In recent years,research on world models has attracted significant attention from both industry and academia,leading to significant research and commercial achievements in domains such as autonomous driving,social simulation,embodied intelligence,and video generation.Moreover,researchers have applied the remarkable results of various large models to world models,further enhancing their performance.This paper comprehensively reviews world models built using large models across different domains,covering both language large model-and Vision Large Model(VLM)-based approaches.Several important application areas,including embodied intelligence,smart cities,social simulation,and physical environment simulation,are selected to introduce relevant models.This paper classifies world models based on the modality of the large models used,highlighting the functional differences between world models based on different modalities.Subsequently,important open-source resources and benchmarks for world models are presented to help researchers in related fields understand and utilize world models quickly.Finally,this paper is summarized and future research directions are presented.