Assessing Large Language Models for Early Article Identification in Otolaryngology—Head and Neck Surgery Systematic ReviewsOA
Background:Assess ChatGPT and Bard''s effectiveness in the initial identification of articles for Otolaryngology—Head and Neck Surgery systematic literature reviews.Methods:Three PRISMA-based systematic reviews(Jabbour et al.2017,Wong et al.2018,and Wu et al.2021)were replicated using ChatGPTv3.5 and Bard.Outputs(author,title,publication year,and journal)were compared to the original references and cross-referenced with medical databases for authenticity and recall.Results:Several themes emerged when comparing Bard and ChatGPT across the three reviews.Bard generated more outputs and had greater recall in Wong et al.''s review,with a broader date range in Jabbour et al.''s review.In Wu et al.''s review,ChatGPT-2 had higher recall and identified more authentic outputs than Bard-2.Conclusion:Large language models(LLMs)failed to fully replicate peer-reviewed methodologies,producing outputs with inaccuracies but identifying relevant,especially recent,articles missed by the references.While human-led PRISMA-based reviews remain the gold standard,refining LLMs for literature reviews shows potential.
Ajibola B.Bakare;Young Lee;Jhuree Hong;Claus-Peter Richter;Jonathan P.Kuriakose
Tulane University School of Medicine,New Orleans,Louisiana,USACentral Michigan University College of Medicine,Saginaw,Michigan,USASoftware Development Engineer,Independent Researcher,Seattle,Washington,USADepartment of Biomedical Engineering,Northwestern University,Evanston,Illinois,USA Department of Communication Sciences and Disorders,Northwestern University,Evanston,Illinois,USA The Hugh Knowles Center,Northwestern University,Evanston,Illinois,USA Department of Otolaryngology—Head&Neck Surgery,Northwestern Feinberg School of Medicine,Chicago,Illinois,USADepartment of Otolaryngology—Head&Neck Surgery,Northwestern Feinberg School of Medicine,Chicago,Illinois,USA
医药卫生
artificial intelligenceBardChatGPTlarge language modelssystematic review
《Health Care Science》 2026 (1)
P.19-28,10
评论