Performance of a novel medical artificial intelligence large language model on supporting decision-making for emergency patients with suspected sepsis

doi:10.5847/wjem.j.1920-8642.2025.095

Abstract

Abstract:

BACKGROUND: Large language models (LLMs) are being explored for disease prediction and diagnosis; however, their efficacy for early sepsis identification in emergency departments (EDs) remains unexplored. This study aims to evaluate MedGo, a novel medical LLM, as a decision-support tool for clinicians managing patients with suspected sepsis.

METHODS: This retrospective study included anonymized medical records of 203 patients (mean age 79.9±10.2 years) with confirmed sepsis from a tertiary hospital ED between January 2023 and January 2024. MedGo performance across nine sepsis-related assessment tasks was compared with that of two junior (<3 years of experience) and two senior (>10 years of experience) ED physicians. Assessments were scored on a 5-point Likert scale for accuracy, comprehensiveness, readability, and case-analysis skills.

RESULTS: MedGo demonstrated diagnostic performance comparable to that of senior physicians across most metrics, achieving a median Likert score of 4 in accuracy, comprehensiveness, and readability. MedGo significantly outperformed junior physicians (P<0.001 for accuracy and case-analysis skills). MedGo assistance significantly enhanced both junior (P<0.001) and senior (P<0.05) physicians' diagnostic accuracy. Notably, MedGo-assisted junior physicians achieved accuracy levels comparable to those of unassisted senior physicians. MedGo maintained consistent performance across varying sepsis severities.

CONCLUSION: MedGo shows significant diagnostic efficacy for sepsis and effectively supports clinicians in the ED, particularly enhancing junior physicians’ performance. Our study highlights the potential of MedGo as a valuable decision-support tool for sepsis management, paving the way for specialized sepsis AI models.

Key words: Large language models, Sepsis, Emergency department

Sen Jiang, Xiandong Liu, Tong Liu, Yi Gu, Bo An, Chunxue Wang, Dongyang Zhao, Haitao Zhang, Lunxian Tang. Performance of a novel medical artificial intelligence large language model on supporting decision-making for emergency patients with suspected sepsis[J]. World Journal of Emergency Medicine, 2025, 16(5): 447-455.

Figures/Tables 5

Table 1.

Table 2.

Figure 1.

Figure 2.

Figure 3.

References 27

1	Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8):801-10. doi: 10.1001/jama.2016.0287 pmid: 26903338
2	Liu VX, Fielding-Singh V, Greene JD, Baker JM, Iwashyna TJ, Bhattacharya J, et al. The timing of early antibiotics and hospital mortality in sepsis. Am J Respir Crit Care Med. 2017; 196(7): 856-63.
3	Zhang WY, Chen ZH, An XX, Li H, Zhang HL, Wu SJ, et al. Analysis and validation of diagnostic biomarkers and immune cell infiltration characteristics in pediatric sepsis by integrating bioinformatics and machine learning. World J Pediatr. 2023; 19(11):1094-103.
4	Barichello T, Generoso JS, Singer M, Dal-Pizzol F. Biomarkers for sepsis: more than just fever and leukocytosis-a narrative review. Crit Care. 2022; 26(1): 14.
5	Wang J, Weng L, Xu J, Du B. Blood gas analysis as a surrogate for microhemodynamic monitoring in sepsis. World J Emerg Med. 2023; 14(6):421-7. doi: 10.5847/wjem.j.1920-8642.2023.093 pmid: 37969221
6	Emergency Medicine Branch of Chinese Medical Care International Exchange Promotion Association, Emergency Medical Branch Of Chinese Medical Association, Chinese Medical Doctor Association Emergency Medical Branch, Chinese People’s Liberation Army Emergency Medicine Professional Committee. Consensus of Chinese experts on early prevention and blocking of sepsis. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. 2020; 32(5): 518-30.
7	Moreno R, Rhodes A, Piquilloud L, Hernandez G, Takala J, Gershengorn HB, et al. The Sequential Organ Failure Assessment (SOFA) score: has the time come for an update?. Crit Care. 2023; 27(1):15.
8	Qiu X, Lei YP, Zhou RX. SIRS, SOFA, qSOFA, and NEWS in the diagnosis of sepsis and prediction of adverse outcomes: a systematic review and meta-analysis. Expert Rev Anti Infect Ther. 2023; 21(8): 891-900.
9	Kim HI, Park S. Sepsis: early recognition and optimized treatment. Tuberc Respir Dis (Seoul). 2019; 82(1): 6-14.
10	Lam RPK, Dai Z, Lau EHY, Ip CYT, Chan HC, Zhao L, et al. Comparing 11 early warning scores and three shock indices in early sepsis prediction in the emergency department. World J Emerg Med. 2024; 15(4):273-82. doi: 10.5847/wjem.j.1920-8642.2024.052 pmid: 39050223
11	He RR, Yue GL, Dong ML, Wang JQ, Cheng C. Sepsis biomarkers: advancements and clinical applications-a narrative review. Int J Mol Sci. 2024; 25(16): 9010.
12	Hahn B, Zuckerman B, Durakovic M, Demissie S. The relationship between emergency department volume and patient complexity. Am J Emerg Med. 2018; 36(3): 366-9. doi: S0735-6757(17)30671-X pmid: 28830636
13	Peltan ID, Mitchell KH, Rudd KE, Mann BA, Carlbom DJ, Hough CL, et al. Physician variation in time to antimicrobial treatment for septic patients presenting to the emergency department. Crit Care Med. 2017; 45(6): 1011-8. doi: 10.1097/CCM.0000000000002436 pmid: 28426466
14	Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations. Radiology. 2023; 307(5): e230582.
15	Teng AK, Wilcox AB. A review of predictive analytics solutions for sepsis patients. Appl Clin Inform. 2020; 11(3): 387-98. doi: 10.1055/s-0040-1710525 pmid: 32462640
16	Zhang H, An B. MedGo: A Chinese medical large language model. arXiv [Preprint]. 2024; 2410.20428.
17	Song HF, Xia Y, Luo ZC, Liu H, Song Y, Zeng X, et al. Evaluating the performance of different large language models on health consultation and patient education in urolithiasis. J Med Syst. 2023; 47(1): 125. doi: 10.1007/s10916-023-02021-3 pmid: 37999899
18	Sandmann S, Riepenhausen S, Plagwitz L, Varghese J. Systematic analysis of ChatGPT, google search and Llama 2 for clinical decision support tasks. Nat Commun. 2024; 15(1): 2050. doi: 10.1038/s41467-024-46411-8 pmid: 38448475
19	Oduncu AF, Kıyan GS, Yalçınlı S. Comparison of qSOFA, SIRS, and NEWS scoring systems for diagnosis, mortality, and morbidity of sepsis in emergency department. Am J Emerg Med. 2021; 48: 54-9. doi: 10.1016/j.ajem.2021.04.006 pmid: 33839632
20	Gaini S, Relster MM, Pedersen C, Johansen IS. Prediction of 28-days mortality with sequential organ failure assessment (SOFA), quick SOFA (qSOFA) and systemic inflammatory response syndrome (SIRS)—a retrospective study of medical patients with acute infectious disease. Int J Infect Dis. 2019; 78: 1-7.
21	Williams CYK, Miao BY, Kornblith AE, Butte AJ. Evaluating the use of large language models to provide clinical recommendations in the emergency department. Nat Commun. 2024; 15(1): 8236. doi: 10.1038/s41467-024-52415-1 pmid: 39379357
22	Shashikumar SP, Nemati S. A prospective comparison of large language models for early prediction of sepsis. Pac Symp Biocomput. 2025; 30: 109-20. pmid: 39670365
23	Arnaud E, Elbattah M, Moreno-Sánchez PA, Dequen G, Ghazali DA. Explainable NLP model for predicting patient admissions at emergency department using triage notes. In: Proceeding of 2023 IEEE International Conference on Big Data (BigData); Sorrento, Italy; 2023: 4843-7.
24	Sung J. Artificial intelligence in medicine: Ethical, social and legal perspectives. Ann Acad Med Singap. 2023; 52(12): 695-9. doi: 10.47102/annals-acadmedsg.2023272 pmid: 38920162
25	Dalton-Brown S. The ethics of medical AI and the physician-patient relationship. Camb Q Healthc Ethics. 2020; 29(1): 115-21. doi: 10.1017/S0963180119000847 pmid: 31858938
26	Uslu F, Icoz K, Tasdemir K, Doğan RS, Yilmaz B. Image-analysis based readout method for biochip: Automated quantification of immunomagnetic beads, micropads and patient leukemia cell. Micron. 2020; 133: 102863.
27	Fatma Ç, Tasdemir K, Icoz K. Deep learning based semantic segmentation and quantification for MRD biochip images. Biomed Signal Process Contr. 2022; 77: 103783.

Category	Number	Tasks
Initial assessment	1	Please provide five possible diseases for the following symptoms
Detailed assessment	2	Based on the currently provided medical history, past history, physical examination, and medication information, provide a list of five possible diseases and select one as the primary diagnosis
	3	Determine if the patient in this case is a suspected septic patient
	4	Please determine the severity of the patient in this case (mild OR severe)
	5	Next steps in the management of the condition
Adding ancillary screening assessments	6	Please list five differential diagnoses based on the presenting complaint, current medical history, past history, physical examination, and medication use provided so far and choose one as the primary diagnosis
	7	Determine if the patient in this case is a suspected septic patient
	8	Please determine the severity of the patient in this case (mild OR severe)
	9	Next steps in the management of the condition

Feature	Results
Clinical features
Age, years	79.9±10.2
Gender, n (%)
Male	109 (53.7)
Female	94 (46.3)
Case difficulty, n (%)
Easy diagnosis	139 (68.5)
Moderate difficulty	49 (24.1)
Difficult diagnosis	15 (7.4)
Disease severity, n (%)
Mild	131 (64.5)
Severe	72 (35.5)

[1]	Hui Chen, Yuanhua Lu, Songjie Bai, Yang Li, Tao Wang, Long Cai, Xinyi Yang, Yang Fang, Jianguo Wan, Yaqun Tang, Wenqiang Tao, Meiling Huang, Wei Zhong, Fen Liu, Kejian Qian. Stress-hyperglycemia ratio and glycemic variability predict severity and mortality in sepsis-associated acute respiratory distress syndrome [J]. World Journal of Emergency Medicine, 2026, 17(3): 250-257.
[2]	Yi Yao, Wenjin Li, Dejiang Hong, Ze Chen, Kai Peng, Guangju Zhao. Prediction model for quality of life in sepsis survivors one year after discharge [J]. World Journal of Emergency Medicine, 2026, 17(2): 105-112.
[3]	Nelly Richter, Frank Bloos, Christian Hohenstein. Indications, techniques, success rates and complications of emergency airway management in Thuringian emergency departments: a prospective registry analysis [J]. World Journal of Emergency Medicine, 2026, 17(2): 146-153.
[4]	Zengzheng Ge, Xin Lu, Shiyuan Yu, Mubing Qin, Chao Gong, Qun Xu, Yanxia Gao, Joseph Harold Walline, Huadong Zhu, Yi Li. Diagnostic value of mismatch-negativity and P3a event-related potentials for sepsis-associated encephalopathy in non-sedated patients: a pilot study [J]. World Journal of Emergency Medicine, 2026, 17(2): 172-178.
[5]	Jinlan Ma, Li Peng, Hongming Yu, Jianfeng Xie, Ying Tang, Shenglin Su, Libing Ma, Xiaojun Yang. Single-cell transcriptomics reveals pathogen-specific monocyte heterogeneity and potential biomarkers in gram-positive versus gram-negative bloodstream infections [J]. World Journal of Emergency Medicine, 2026, 17(2): 162-171.
[6]	Yuanwei Fu, Yilan Yang, Hua Zhang, Daidai Wang, Qiangrong Zhai, Lanfang Du, Nijiati Muyesai, Yanxia Gao, Qingbian Ma. Development and validation of machine learning-based in-hospital mortality predictive models for acute aortic syndrome in emergency departments [J]. World Journal of Emergency Medicine, 2026, 17(1): 43-49.
[7]	Xin Lu, Mubing Qin, Zengrui Song, Ying Chen, Huadong Zhu, Yanxia Gao, Yi Li. Normal initial lactate level in sepsis patients: is lactate still useful for prognosis prediction? [J]. World Journal of Emergency Medicine, 2026, 17(1): 57-64.
[8]	Yuhui Pan, Yanyan Ma, Ke Wan, Yizhou Xu, Guoxing Wang, Miaorong Xie. Endothelial cell injury: a crucial link in microcirculatory dysfunction associated with sepsis [J]. World Journal of Emergency Medicine, 2026, 17(1): 28-35.
[9]	Wei Gu, Jie Zhong, Chuanzhu Lyu, Guoqiang Zhang, Miaorong Xie, Yuefeng Ma, Wei Guo. An approach for the emergency diagnosis and treatment of sepsis-associated encephalopathy in elderly individuals: a literature review [J]. World Journal of Emergency Medicine, 2025, 16(5): 415-422.
[10]	Zesheng Wu, Fanghui Chen, Chen Xiao, Xue Zhao, Yuansheng Xu, Jinyan Fang, Yinyan Shao. Long-term cardiovascular outcomes and risk factors in adult sepsis survivors: a systematic review and meta-analysis [J]. World Journal of Emergency Medicine, 2025, 16(5): 423-430.
[11]	Qiaosheng Wang, Qiong Luo, Zhiwei Su, Yan Xu, Liangshan Peng, Yin Wen, Hongke Zeng, Hongguang Ding. Activation of the α7 nicotinic acetylcholine receptor mitigates cognitive deficits in mice with sepsis-associated encephalopathy by inhibiting microglial pyroptosis [J]. World Journal of Emergency Medicine, 2025, 16(5): 438-446.
[12]	Quan Li, Yun Qu, Jinfang Xue, Hai Kang, Chuanzhu Lyu. Exploring lipid-modifying therapies for sepsis through the modulation of circulating inflammatory cytokines: a Mendelian randomization study [J]. World Journal of Emergency Medicine, 2025, 16(3): 256-261.
[13]	Weiming Wu, Min Li, Huilin Jiang, Min Sun, Yongcheng Zhu, Gongxu Zhu, Yanling Li, Yunmei Li, Junrong Mo, Xiaohui Chen, Haifeng Mao. Development of an emergency department length-of-stay prediction model based on machine learning [J]. World Journal of Emergency Medicine, 2025, 16(3): 220-224.
[14]	Peili Chen, Yan Ge, Huiqiu Sheng, Wenwu Sun, Jiahui Wang, Li Ma, Enqiang Mao. The role of early changes in routine coagulation tests in predicting the occurrence and prognosis of sepsis [J]. World Journal of Emergency Medicine, 2025, 16(2): 136-143.
[15]	Chunxue Wang, Dianyin Yang, Yuxin Zhu, Qian Yang, Tong Liu, Xiandong Liu, Dongyang Zhao, Xiaowei Bao, Tiancao Dong, Li Shao, Lunxian Tang. Circulating circular RNAs act as potential novel biomarkers for sepsis secondary to pneumonia: a prospective cohort study [J]. World Journal of Emergency Medicine, 2025, 16(2): 144-152.