Clinical evaluation of a large language model based intelligent preconsultation system in traditional Chinese medicine:A tertiary hospital studyOA
Objective:To evaluate the utilization,completion quality,and physicians’perceptions of a large language model(LLM)-based intelligent pre-consultation system in traditional Chinese medicine(TCM)clinical settings and to examine its role in the emerging physician-artificialintelligence(AI)-patient triadic model,thereby providing evidence for future system optimization.Methods:A total of 21731 pre-consultation records from a tertiary TCM hospital between July 2025 and February 2026 were analyzed for patient demographics and completion status.In addition,42 clinical physicians completed a clinical effectiveness evaluation scale for the pre-consultation system.Descriptive statistics,reliability and validity analyses,and subgroup comparisons were performed using the Mann-Whitney U and Kruskal-Wallis H tests.Qualitative data from physician interviews were thematically analyzed to complement the quantitative findings.Results:The patient completion rate was 77.10%(16754/21731).The physician survey scale demonstrated excellent reliability(Cronbach’sα=0.954)and validity(Kaiser-Meyer-Olkin=0.825).The overall mean physician rating was 3.62±0.78,with the structural dimension(information collection)scoring the highest(3.77±0.72)and the outcome dimension(medical output and perception)scoring the lowest(3.49±0.86).Among the pre-consultation content sections,allergy history received the highest rating(80.83/100),whereas the chief complaints received the lowest rating(70.29/100).Subgroup comparisons revealed significantage-related differences in satisfaction(H=10.006,P=.019),with physicians aged 31-40 years reporting lower satisfaction(median=3.00)and those aged 51-60 years reporting higher satisfaction(median=4.00).No significantdifferences were observed across specialties,sex,professional titles,or digital tool preferences.Conclusion:The LLM-based pre-consultation system demonstrated preliminary physician acceptance,particularly for information collection within the TCM hospital setting.However,substantial improvements are still required in chief complaint acquisition,decision support,workflowintegration,and documentation burden reduction.Future optimization should prioritize clinically actionable information extraction,stronger integration of TCM diagnostic logic,and improved accessibility for elderly users.
Yangyang Han;Guowang Yang;Wujun Tang;Wei Han;Xu Yuan;Yuejiao Chang;Yige Yang;Jingzhi Li;Ye Jin;Meng Yuan
Beijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,ChinaBeijing Hospital of Traditional Chinese Medicine,Capital Medical University,Beijing 100010,China
信息技术与安全科学
Large language modelArtificial intelligenceTraditional Chinese medicineMedical informaticsPre-consultation system
《Journal of Traditional Chinese Medical Sciences》 2026 (3)
P.319-325,7
supported by the National Natural Science Foundation of China(72574152).
评论