无监督数据驱动的垂直领域大模型微调数据集生成框架OA

Large language models have achieved remarkable progress in natural language processing,but general-purpose models face challenges in vertical domains due to the lack of domain-specific knowledge and limited data availability.This paper proposes an unsupervised data-driven fine-tuning dataset generation framework(GEO).By integrating genera-tion,evaluation,and optimization modules,the framework automatically constructs and refines question-answer pairs to produce high-quality,domain-adaptive training data.The method is applied to the low-carbon energy domain to build a specialized dataset based on the GEO framework.LoRA-based fine-tuning experiments are conducted on Chinese models including Qwen2.5-7B,Baichuan2-7B,and ChatGLM3-6B.Results demonstrate that the GEO-generated dataset signifi-cantly enhances text generation quality,semantic consistency,and domain adaptability,while preserving the general capa-bilities of the base models,validating the effectiveness of the approach.