ChatGPT-4.0与DeepSeek-V3两种人工智能语言模型在回答近视问题的基准分析比较OA

Objective:To compare the difference of performance of ChatGPT-4.0 and DeepSeek-V3 of two kinds of artificial intelligence(AI)Chatbots in response to questions about myopia,so as to provide references for application of AI chatbot.Method:From October 2024 to March 2025,a comparative test about two kinds of AI chatbots,namely ChatGPT-4.0 and DeepSeek-V3 of large language model(LLM),was conducted on the responses to questions about myopia at the National University Hospital of Singapore(NUHS)and Beijing Jingmei Group General Hospital of China.The accuracy and comprehensiveness were detected and evaluated by specialists.The content of the myopia question and answer(Q&A)consisted of 30 myopia-related questions in ophthalmic clinical practice,covering six themes about myopia:the pathogenesis,clinical manifestations,diagnosis,treatment,prevention,and prognosis.The evaluation was conducted by storing two kinds of AI chatbots from two aspects including accuracy and comprehensiveness.Results:In terms of accuracy evaluation,11 results(36.7%)of the answers of ChatGPT-4.0 chatbot were detected and evaluated as"good",and 23 results(76.7%)of the answers of DeepSeek-V3 chatbot were detected and evaluated as"good",and the difference of the proportion between two groups was significant(x2=9.791,P<0.05).In terms of the evaluation for comprehensiveness,the comprehensive score of the ChatGPT-4.0 chatbot was(2.44±0.33)points in answering questions,and that of the DeepSeek chatbot was(2.63±0.17)points,and there was not statistically significant difference between them(P>0.05).Conclusion:AI chatbot can provide effective helps about consulting myopia for users.The accuracy of the DeepSeek-V3 chatbot in responding to questions about myopia is superior to that of the ChatGPT-4.0 chatbot.