Comparing 11 early warning scores and three shock indices in early sepsis prediction in the emergency department

doi:10.5847/wjem.j.1920-8642.2024.052

Abstract

Abstract:

BACKGROUND: This study aimed to evaluate the discriminatory performance of 11 vital sign-based early warning scores (EWSs) and three shock indices in early sepsis prediction in the emergency department (ED).

METHODS: We performed a retrospective study on consecutive adult patients with an infection over 3 months in a public ED in Hong Kong. The primary outcome was sepsis (Sepsis-3 definition) within 48 h of ED presentation. Using c-statistics and the DeLong test, we compared 11 EWSs, including the National Early Warning Score 2 (NEWS2), Modified Early Warning Score, and Worthing Physiological Scoring System (WPS), etc., and three shock indices (the shock index [SI], modified shock index [MSI], and diastolic shock index [DSI]), with Systemic Inflammatory Response Syndrome (SIRS) and quick Sequential Organ Failure Assessment (qSOFA) in predicting the primary outcome, intensive care unit admission, and mortality at different time points.

RESULTS: We analyzed 601 patients, of whom 166 (27.6%) developed sepsis. NEWS2 had the highest point estimate (area under the receiver operating characteristic curve [AUROC] 0.75, 95%CI 0.70-0.79) and was significantly better than SIRS, qSOFA, other EWSs and shock indices, except WPS, at predicting the primary outcome. However, the pooled sensitivity and specificity of NEWS2 ≥ 5 for the prediction of sepsis were 0.45 (95%CI 0.37-0.52) and 0.88 (95%CI 0.85-0.91), respectively. The discriminatory performance of all EWSs and shock indices declined when used to predict mortality at a more remote time point.

CONCLUSION: NEWS2 compared favorably with other EWSs and shock indices in early sepsis prediction but its low sensitivity at the usual cut-off point requires further modification for sepsis screening.

Key words: Sepsis, Emergency department, Clinical prediction rule, Early warning score, Shock index

Rex Pui Kin Lam, Zonglin Dai, Eric Ho Yin Lau, Carrie Yuen Ting Ip, Ho Ching Chan, Lingyun Zhao, Tat Chi Tsang, Matthew Sik Hon Tsui, Timothy Hudson Rainer. Comparing 11 early warning scores and three shock indices in early sepsis prediction in the emergency department[J]. World Journal of Emergency Medicine, 2024, 15(4): 273-282.

Figures/Tables 5

Figure 1.

Table 1.

Clinical characteristics of patients with and without sepsis

Variables	All included patients (n=601)	Patients with sepsis (n=166)	Patients without sepsis (n=435)
Age, year, median (IQR)	79.0 (66.5-89.0)	85.5 (74.8-92.0)	75.0 (63.0-87.0)
Sex, n (%)
Female	308 (51.2)	81 (48.8)	227 (52.2)
Male	293 (48.8)	85 (51.2)	208 (47.8)
Elderly home residence, n (%)	221 (36.8)	83 (50.0)	138 (31.7)
Charlson Comorbidity Index, median (IQR)	2 (0-4)	2 (1-5)	2 (0-3)
Organ system involved, n (%)
Respiratory	192 (31.9)	74 (44.6)	118 (27.1)
Urogenital	90 (15.0)	18 (10.8)	72 (16.6)
Gastrointestinal	92 (15.3)	27 (16.3)	65 (14.9)
Soft tissue	43 (7.2)	2 (1.2)	41 (9.4)
Pathogens, n (%)
Gram-positive bacteria	45 (7.5)	9 (5.4)	36 (8.3)
Gram-negative bacteria	139 (23.1)	45 (27.1)	94 (21.6)
Fungus	15 (2.5)	9 (5.4)	6 (1.4)
Mixed organisms	66 (11.0)	17 (10.2)	49 (11.3)
COVID-19 infection, n (%)	19 (3.2)	3 (1.8)	16 (3.7)
Triage vital signs, median (IQR)
Temperature, °C	37.9 (36.9-38.6)	38.1 (37.0-38.7)	37.8 (36.9-38.5)
Systolic blood pressure, mmHg	145 (125-163)	143 (121-163)	146 (127-163)
Diastolic blood pressure, mmHg	75 (64-86)	71 (60-82)	76 (65-87)
Mean arterial pressure, mmHg	98 (86-110)	95 (83-106)	99 (87-111)
Pulse rate, beats per minute	97 (84-109)	101 (88-114)	96 (82-108)
SpO₂, %	96 (95-97)	96 (94-97)	96 (95-98)
Supplemental O₂, n (%)	124 (20.6)	84 (50.6)	40 (9.2)
Respiratory rate, breath per minute	16 (16-18)	18 (16-22)	16 (16-18)
Conscious level, n (%)
Alert	583 (97.0)	156 (94.0)	427 (98.2)
Verbal	9 (1.5)	5 (3.0)	4 (0.9)
Pain	8 (1.3)	4 (2.4)	4 (0.9)
Unconscious	1 (0.2)	1 (0.6)	0 (0)
Worst SOFA within 48 h, median (IQR)	1 (0-2)	2 (2-3)	0 (0-1)
ICU admission, n (%)	8 (1.3)	8 (4.8)	0 (0)
2-day mortality, n (%)	8 (1.3)	8 (4.8)	0 (0)
7-day mortality, n (%)	19 (3.2)	15 (9.0)	4 (0.9)
30-day mortality, n (%)	74 (12.3)	41 (24.7)	33 (7.6)
90-day mortality, n (%)	141 (23.5)	69 (41.6)	72 (16.6)

Table 1.

Figure 2.

Table 2.

Diagnostic metrics of early warning scores and shock indices in predicting sepsis within 48 h emergency department presentation (n=601)

Early warning score/ shock index	Cut-off point	AUROC (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)	Positive predictive value (95%CI)	Negative predictive value (95%CI)	Positive likelihood ratio (95%CI)	Negative likelihood ratio (95%CI)
NEWS2	≥ 5	0.75 (0.70-0.79)	0.45 (0.37-0.52)	0.88 (0.85-0.91)	0.59 (0.50-0.68)	0.81 (0.77-0.84)	3.83 (2.81-5.22)	0.63 (0.54-0.72)
MEWS	> 4	0.61 (0.55-0.66)	0.21 (0.15-0.28)	0.87 (0.83-0.90)	0.38 (0.28-0.48)	0.74 (0.70-0.78)	1.58 (1.08-2.31)	0.91 (0.83-0.99)
RAPS	> 2	0.53 (0.47-0.58)	0.22 (0.16-0.29)	0.86 (0.83-0.89)	0.38 (0.29-0.49)	0.74 (0.70-0.78)	1.63 (1.12-2.35)	0.90 (0.82-0.98)
REMS	≥ 6	0.63 (0.58-0.68)	0.85 (0.79-0.90)	0.35 (0.31-0.40)	0.33 (0.29-0.38)	0.86 (0.80-0.91)	1.31 (1.20-1.45)	0.43 (0.29-0.62)
WPS	≥ 3	0.69 (0.64-0.75)	0.39 (0.31-0.47)	0.89 (0.86-0.92)	0.58 (0.48-0.67)	0.79 (0.75-0.83)	3.63 (2.61-5.06)	0.68 (0.60-0.78)
HEWS	≥ 3	0.68 (0.63-0.72)	0.57 (0.49-0.65)	0.68 (0.63-0.72)	0.40 (0.34-0.47)	0.81 (0.76-0.84)	1.77 (1.46-2.13)	0.63 (0.52-0.76)
PRESEP	≥ 4	0.60 (0.55-0.65)	0.57 (0.49-0.64)	0.57 (0.53-0.62)	0.34 (0.28-0.40)	0.78 (0.73-0.82)	1.33 (1.12-1.58)	0.76 (0.62-0.91)
CIP	≥ 4	0.64 (0.58-0.69)	0.03 (0.01-0.07)	1.00 (0.98-1.00)	0.71 (0.29-0.96)	0.73 (0.69-0.76)	6.55 (1.28-33.44)	0.97 (0.95-1.00)
PSP-S	≥ 3	0.60 (0.55-0.65)	0.34 (0.27-0.42)	0.75 (0.70-0.79)	0.34 (0.27-0.42)	0.75 (0.71-0.79)	1.37 (1.05-1.78)	0.88 (0.78-0.99)
REWS	≥ 4	0.62 (0.57-0.67)	0.25 (0.19-0.33)	0.89 (0.86-0.92)	0.47 (0.36-0.58)	0.76 (0.72-0.79)	2.33 (1.60-3.39)	0.84 (0.76-0.92)
mSOS	≥ 4	0.62 (0.57-0.67)	0.19 (0.14-0.26)	0.91 (0.88-0.93)	0.44 (0.33-0.56)	0.75 (0.71-0.78)	2.09 (1.36-3.20)	0.89 (0.82-0.96)
qSOFA	≥ 2	0.62 (0.57-0.67)	0.05 (0.02-0.09)	0.99 (0.98-1.00)	0.73 (0.39-0.94)	0.73 (0.69-0.77)	6.99 (1.88-26.02)	0.96 (0.93-0.99)
SIRS	≥ 2	0.65 (0.60-0.70)	0.72 (0.64-0.78)	0.50 (0.45-0.55)	0.35 (0.30-0.41)	0.82 (0.77-0.87)	1.44 (1.26-1.65)	0.56 (0.43-0.73)
SI	≥ 0.7	0.57 (0.52-0.62)	0.51 (0.43-0.59)	0.56 (0.52-0.61)	0.31 (0.26-0.37)	0.75 (0.70-0.80)	1.18 (0.98-1.41)	0.86 (0.72-1.03)
MSI	≥ 1	0.60 (0.55-0.64)	0.60 (0.52-0.67)	0.55 (0.50-0.59)	0.33 (0.28-0.39)	0.78 (0.73-0.83)	1.31 (1.12-1.55)	0.74 (0.60-0.91)
DSI	≥ 2	0.62 (0.57-0.67)	0.06 (0.03-0.11)	0.98 (0.96-0.99)	0.56 (0.31-0.78)	0.73 (0.69-0.77)	3.28 (1.32-8.16)	0.96 (0.92-1.00)

Table 2.

Table 3.

References 50

[1]	Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. Lancet. 2020 ; 395(10219): 200-11.
[2]	Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. 2021; 47(11): 1181-247. doi: 10.1007/s00134-021-06506-y pmid: 34599691
[3]	Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996; 22(7): 707-10.
[4]	Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8): 801-10. doi: 10.1001/jama.2016.0287 pmid: 26903338
[5]	Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic Inflammatory Response Syndrome criteria in defining severe sepsis. N Engl J Med. 2015; 372(17): 1629-38.
[6]	Maitra S, Som A, Bhattacharjee S. Accuracy of quick Sequential Organ Failure Assessment (qSOFA) score and Systemic Inflammatory Response Syndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta-analysis of observational studies. Clin Microbiol Infect. 2018; 24(11): 1123-9.
[7]	Adegbite BR, Edoa JR, Ndzebe Ndoumba WF, Dimessa Mbadinga LB, Mombo-Ngoma G, Jacob ST, et al. A comparison of different scores for diagnosis and mortality prediction of adults with sepsis in low-and-middle-income countries: a systematic review and meta-analysis. EClinicalMedicine. 2021;42: 101184.
[8]	Song JU, Sin CK, Park HK, Shim SR, Lee J. Performance of the quick Sequential (sepsis-related) Organ Failure Assessment score as a prognostic tool in infected patients outside the intensive care unit: a systematic review and meta-analysis. Crit Care Lond Engl. 2018; 22(1): 28.
[9]	Fernando SM, Tran A, Taljaard M, Cheng W, Rochwerg B, Seely AJE, et al. Prognostic accuracy of the quick Sequential Organ Failure Assessment for mortality in patients with suspected infection: a systematic review and meta-analysis. Ann Intern Med. 2018; 168(4): 266-75. doi: 10.7326/M17-2820 pmid: 29404582
[10]	Nannan Panday RS, Minderhoud TC, Alam N, Nanayakkara PWB. Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): a narrative review. Eur J Intern Med. 2017;45: 20-31.
[11]	Royal College of Physicians. ational Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party. London: RCP. 2017. Available at: https://lof.se/filer/NEWS2-final-report.pdf
[12]	Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified early warning score in medical admissions. QJM. 2001; 94(10): 521-6.
[13]	Rhee KJ, Fisher CJ Jr, Willitis NH. The rapid acute physiology score. Am J Emerg Med. 1987; 5(4): 278-82. pmid: 3593492
[14]	Olsson T, Terent A, Lind L. Rapid Emergency Medicine Score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. J Intern Med. 2004; 255(5): 579-87. doi: 10.1111/j.1365-2796.2004.01321.x pmid: 15078500
[15]	Duckitt RW, Buxton-Thomas R, Walker J, Cheek E, Bewick V, Venn R, et al. Worthing Physiological Scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. Br J Anaesth. 2007; 98(6): 769-74.
[16]	Skitch S, Tam B, Xu M, McInnis L, Vu A, Fox-Robichaud A. Examining the utility of the Hamilton Early Warning Scores (HEWS) at triage: retrospective pilot study in a Canadian emergency department. CJEM. 2018; 20(2): 266-74.
[17]	Mellhammar L, Linder A, Tverring J, Christensson B, Boyd JH, Sendi P, et al. NEWS2 is superior to qSOFA in detecting sepsis with organ dysfunction in the emergency department. J Clin Med. 2019; 8(8): 1128.
[18]	Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017; 195(7): 906-11.
[19]	Mahala P, Kaeley N, Kabi A, Choudhary S, Hazra A, Vempalli S. Utility of early warning scores to predict mortality in COVID-19 patients: a retrospective observational study. Int J Crit Illn Inj Sci. 2021; 11(3): 161.
[20]	Suttapanit K, Dangprasert K, Sanguanwit P, Supatanakij P. The Ramathibodi Early Warning Score as a sepsis screening tool does not reduce the timing of antibiotic administration. Int J Emerg Med. 2022; 15(1): 18.
[21]	Thodphetch M, Chenthanakij B, Wittayachamnankul B, Sruamsiri K, Tangsuwanaruk T. A comparison of scoring systems for predicting mortality and sepsis in the emergency department patients with a suspected infection. Clin Exp Emerg Med. 2021; 8(4): 289-95. doi: 10.15441/ceem.20.145 pmid: 35000356
[22]	Seymour CW, Kahn JM, Cooke CR, Watkins TR, Heckbert SR, Rea TD. Prediction of critical illness during out-of-hospital emergency care. JAMA. 2010; 304(7): 747-54. doi: 10.1001/jama.2010.1140 pmid: 20716737
[23]	Lane DJ, Wunsch H, Saskin R, Cheskes S, Lin S, Morrison LJ, et al. Screening strategies to identify sepsis in the prehospital setting: a validation study. CMAJ. 2020; 192(10): E230-E239.
[24]	Bayer O, Schwarzkopf D, Stumme C, Stacke A, Hartog CS, Hohenstein C, et al. An early warning scoring system to identify septic patients in the prehospital setting: the PRESEP score. Acad Emerg Med. 2015; 22(7): 868-71. doi: 10.1111/acem.12707 pmid: 26113162
[25]	Baez AA, Cochon L. Acute Care Diagnostics Collaboration: assessment of a Bayesian clinical decision model integrating the Prehospital Sepsis Score and point-of-care lactate. Am J Emerg Med. 2016; 34(2): 193-6. doi: 10.1016/j.ajem.2015.10.007 pmid: 26585200
[26]	Berger T, Green J, Horeczko T, Hagar Y, Garg N, Suarez A, et al. Shock Index and early recognition of sepsis in the emergency department: pilot study. West J Emerg Med. 2013; 14(2): 168-74.
[27]	Devendra Prasad KJ, Abhinov T, Himabindu KC, Rajesh K, Krishna Moorthy D. Modified Shock Index as an indicator for prognosis among sepsis patients with and without comorbidities presenting to the emergency department. Cureus. 2021; 13(12): e20283.
[28]	Ospina-Tascón GA, Teboul JL, Hernandez G, Alvarez I, Sánchez-Ortiz AI, Calderón-Tapia LE, et al. Diastolic shock index and clinical outcomes in patients with septic shock. Ann Intensive Care. 2020; 10(1): 41.
[29]	Keep JW, Messmer AS, Sladden R, Burrell N, Pinate R, Tunnicliff M, et al. National Early Warning Score at emergency department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study. Emerg Med J. 2016; 33(1): 37-41. doi: 10.1136/emermed-2014-204465 pmid: 25971890
[30]	Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the emergency department. Am J Emerg Med. 2019; 37(8): 1490-7. doi: S0735-6757(18)30889-1 pmid: 30470600
[31]	Wattanasit P, Khwannimit B. Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department. Am J Emerg Med. 2021;46: 284-8.
[32]	Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351: h5527.
[33]	Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD). Ann Intern Med. 2015; 162(10): 735-6.
[34]	Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987; 40(5): 373-83. doi: 10.1016/0021-9681(87)90171-8 pmid: 3558716
[35]	Rice TW, Wheeler AP, Bernard GR, Hayden DL, Schoenfeld DA, Ware LB, et al. Comparison of the SpO₂/FIO₂ ratio and the PaO₂/FIO₂ ratio in patients with acute lung injury or ARDS. Chest. 2007; 132(2): 410-7.
[36]	McNarry AF, Goldhill DR. Simple bedside assessment of level of consciousness: comparison of two simple assessment scales with the Glasgow Coma Scale. Anaesthesia. 2004; 59(1): 34-7. doi: 10.1111/j.1365-2044.2004.03526.x pmid: 14687096
[37]	Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform. 2014;48: 193-204.
[38]	Fischer JE, Bachmann LM, Jaeschke R. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med. 2003; 29(7): 1043-51.
[39]	Rubin DB. Multiple imputation for nonresponse in surveys. New York: John Wiley and Sons; 1987.
[40]	DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44(3): 837-45. pmid: 3203132
[41]	National Health Services England.. National Early Warning Score (NEWS). Available at: https://www.england.nhs.uk/ourwork/clinical-policy/sepsis/nationalearlywarningscore/
[42]	Durr D, Niemi T, Despraz J, Tusgul S, Dami F, Akrour R, et al. National Early Warning Score (NEWS) outperforms quick Sepsis-related Organ Failure (qSOFA) score for early detection of sepsis in the emergency department. Antibiotics. 2022; 11(11): 1518.
[43]	Osawa I, Sonoo T, Soeno S, Hara K, Nakamura K, Goto T. Clinical performance of early warning scoring systems for identifying sepsis among anti-hypertensive agent users. Am J Emerg Med. 2021;48: 120-7.
[44]	Faisal M, Richardson D, Scally AJ, Howes R, Beatson K, Speed K, et al. Computer-aided National Early Warning Score to predict the risk of sepsis following emergency medical admission to hospital: a model development and external validation study. CMAJ. 2019; 191(14): E382-E389.
[45]	Zonneveld LEEC, van Wijk RJ, Olgers TJ, Bouma HR, Ter Maaten JC. Prognostic value of serial score measurements of the National Early Warning Score, the quick Sequential Organ Failure Assessment and the Systemic Inflammatory Response Syndrome to predict clinical outcome in early sepsis. Eur J Emerg Med. 2022; 29(5): 348-56. doi: 10.1097/MEJ.0000000000000924 pmid: 36062434
[46]	McGrath SP, Perreard I, MacKenzie T, Calderwood M. Improvement of sepsis identification through multi-year comparison of sepsis and early warning scores. Am J Emerg Med. 2022;51: 239-47.
[47]	Prasad PA, Fang MC, Martinez SP, Liu KD, Kangelaris KN. Identifying the sickest during triage: using point-of-care severity scores to predict prognosis in emergency department patients with suspected sepsis. J Hosp Med. 2021; 16(8): 453-61. doi: 10.12788/jhm.3642 pmid: 34328843
[48]	Baez AA, Hanudel P, Wilcox SR. The Prehospital Sepsis Project: out-of-hospital physiologic predictors of sepsis outcomes. Prehosp Disaster Med. 2013; 28(6): 632-5. doi: 10.1017/S1049023X1300890X pmid: 24229512
[49]	Sabir L, Ramlakhan S, Goodacre S. Comparison of qSOFA and Hospital Early Warning Scores for prognosis in suspected sepsis in emergency department patients: a systematic review. Emerg Med J. 2022; 39(4): 284-94.
[50]	Spencer W, Smith J, Date P, de Tonnerre E, Taylor DM. Determination of the best early warning scores to predict clinical outcomes of patients in the emergency department. Emerg Med J. 2019; 36(12): 716-21. doi: 10.1136/emermed-2019-208622 pmid: 31366627

Early warning score/ shock index	ICU admission AUROC (95%CI)	2-day mortality AUROC (95%CI)	7-day mortality AUROC (95%CI)	30-day mortality AUROC (95%CI)	90-day mortality AUROC (95%CI)
NEWS2	0.83 (0.69-0.91)	0.85 (0.68-0.94)	0.73 (0.60-0.83)	0.65 (0.58-0.72)	0.63 (0.58-0.68)
MEWS	0.85 (0.71-0.92)	0.66 (0.50-0.80)	0.60 (0.48-0.71)	0.57 (0.51-0.64)	0.55 (0.50-0.60)
RAPS	0.70 (0.46-0.87)	0.70 (0.51-0.84)	0.58 (0.44-0.71)	0.53 (0.46-0.60)	0.53 (0.48-0.58)
REMS	0.68 (0.42-0.86)	0.80 (0.65-0.89)	0.68 (0.56-0.77)	0.64 (0.58-0.70)	0.65 (0.59-0.69)
WPS	0.77 (0.58-0.89)	0.78 (0.53-0.91)	0.70 (0.54-0.82)	0.65 (0.58-0.72)	0.65 (0.59-0.70)
HEWS	0.80 (0.59-0.92)	0.79 (0.62-0.89)	0.63 (0.48-0.76)	0.61 (0.54-0.68)	0.56 (0.50-0.61)
PRESEP	0.73 (0.53-0.87)	0.45 (0.29-0.62)	0.45 (0.34-0.56)	0.51 (0.44-0.58)	0.49 (0.44-0.54)
CIP	0.76 (0.47-0.92)	0.73 (0.52-0.88)	0.70 (0.55-0.81)	0.63 (0.56-0.70)	0.62 (0.56-0.67)
PSP-S	0.86 (0.75-0.93)	0.56 (0.40-0.72)	0.57 (0.45-0.69)	0.57 (0.50-0.64)	0.54 (0.49-0.60)
REWS	0.83 (0.65-0.93)	0.65 (0.45-0.80)	0.55 (0.42-0.67)	0.56 (0.50-0.63)	0.54 (0.49-0.60)
mSOS	0.80 (0.65-0.90)	0.67 (0.49-0.81)	0.56 (0.43-0.68)	0.56 (0.49-0.63)	0.54 (0.49-0.59)
qSOFA	0.74 (0.50-0.89)	0.74 (0.50-0.89)	0.72 (0.57-0.83)	0.62 (0.54-0.69)	0.61 (0.55-0.67)
SIRS	0.74 (0.57-0.87)	0.63 (0.42-0.80)	0.60 (0.47-0.72)	0.57 (0.50-0.64)	0.55 (0.50-0.61)
SI	0.87 (0.75-0.94)	0.57 (0.37-0.75)	0.60 (0.47-0.71)	0.57 (0.50-0.64)	0.57 (0.51-0.62)
MSI	0.86 (0.73-0.93)	0.60 (0.38-0.78)	0.61 (0.48-0.73)	0.59 (0.52-0.65)	0.59 (0.54-0.64)
DSI	0.82 (0.65-0.91)	0.62 (0.40-0.80)	0.61 (0.47-0.73)	0.59 (0.52-0.65)	0.60 (0.55-0.65)