Emergency medicine involves a density of decision-making that exceeds that of any other medical specialty. Emergency physicians face high-stakes decisions related to diagnosis, treatment, and disposition with limited information and under intense time pressure during every shift. However, the first critical decision in the emergency department (ED) is often not made by physicians, but by emergency nurses. Within minutes of a patient’s arrival, these frontline clinicians are tasked with assigning triage acuity levels that dictate the course of care for individuals and shape the operational efficiency of the entire department.^1,2 Information and time constraints are most intense at triage and variability in decision making is high.³ Such challenging circumstances are where data-driven clinical decision support (CDS) is most beneficial.

The most commonly used triage tool in the US is the Emergency Severity Index (ESI).^4,5 ESI is a five-level triage scale that relies heavily on operator intuition with an associated risk for bias and untoward variability.^3,6–9 Vital signs are the only objective data considered, with severe derangements signaling that assignment to high acuity (Level 1 or 2) should be contemplated. Differentiation between Levels 3 through 5 is determined based on anticipated ED resource utilization, with limited consideration of risk for adverse clinical outcome.⁴ Patients assigned to high or low ESI acuity have definitive care trajectories; high-acuity patients are seen within minutes of arrival⁴ and low-acuity patients are often diverted to separate workstreams for rapid treatment and disposition.² In contrast, those triaged to the mid-point (ESI Level 3) have an uncertain clinical course and experience extended wait times. Unfortunately, 50 to 70 percent of all ED patients are assigned to ESI Level 3.^10–12

Very recently, Sax et al., performed the largest study of ED triage in history. “Evaluation of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage,” published in JAMA Network Open, included 5.3 million ED encounters from 21 hospitals, all of which used ESI.¹³ Its authors utilized a granular electronic health record (EHR)-derived database and developed rigorous objective criteria to determine the accuracy of triage. They reported that one in three patients was mistriaged using ESI. Alarmingly, just 66 percent of ED patients who required life-stabilizing interventions were properly identified as high-risk (Level 1 or 2). They also found that several vulnerable populations—including those with complex medical histories, those living in poorer neighborhoods, and those who self-identified as Black—were at particularly high risk for mistriage.

The findings of Sax et al., are concerning, but not surprising. Two recent systematic reviews revealed similar deficiencies.^3,14 They demonstrated that ESI, along with all other legacy triage scales in use across the globe, has poor sensitivity for critical illness and is subject to high variability. Sax et al.’s findings also add to a multitude of studies reporting inequity in triage under ESI, including those demonstrating lower triage acuity assignment for Black and Hispanic patient and under-estimation of illness severity in elderly populations.^9,15–19

In their supplement, Sax et. al. provide evidence that ESI, even if optimally applied, has limited capacity for patient differentiation in our current practice environment. Less than half of patients in their health system met objective ESI criteria for high (3.1 percent) or low (37.2 percent) acuity under ESI; the majority (59.7 percent) were left to a single category: Level 3.¹³ This is similar to the proportion of patients triaged to ESI Level 3 in a report that included 25 EDs from 11 different US healthcare organizations in 10 states.¹⁰ Majority allocation to a single ambiguous midpoint within a 5-Level triage system runs counter to the fundamental objectives of triage: to differentiate and prioritize.³

We can do better. It is possible to make triage more accurate and equitable. It is possible to distribute patients more effectively across triage levels, and to match those levels to operational capacity and needs of individual departments. All these things can be achieved, but they require charting a new course in our approach to ED triage.

First, we must abandon the notion that resource utilization is a sufficient proxy for illness severity, patient complexity or even ED care intensity. Instead, as with every other decision in emergency medicine, risk of adverse outcome should guide decision-making for ED triage. Under our current approach to triage, a 20-year-old with no medical problems and a 70-year-old with hypertension, hyperlipidemia, and diabetes who both present with chest pain and have similar vital signs would each be triaged to Level 3. In a crowded ED, they would wait with identical prioritization. This should not be the case; their clinical risk profiles are dramatically different. Protocolized front-end care pathways can be used to expedite diagnostic evaluation and without abnormal results, our 20-year-old with chest pain could be quickly treated and dispositioned in a ‘vertical care’ or ‘fast-track’ area. Often, less time in treatment space is required for such a patient than for an abscess, laceration, or strep throat. None of these patients should compete for care with an elderly patient with chest pain.

Second, as highlighted by Sax et al., in their Conclusion, we must embrace a more data-driven and objective approach to ED triage.¹³ Widespread adoption of the EHR has generated continuously growing pools of clinical data with potential to inform and improve ED care delivery. Artificial intelligence (AI) applications that leverage these data to provide easily accessible (i.e., embedded within EHR workflow) decision support are a promising means to achieve more accurate triage.^11,20,21 AI algorithms can use historical data to rapidly estimate clinical risk for individual patients in real-time and can provide decision rationale. These algorithms can be adapted to each ED site to account for differences in patient populations, resource availability, and operational objectives.

AI-driven approaches also generate opportunities for increasing triage equity. AI in medicine has been met with justified concern for perpetuation of bias and exacerbation of social inequities.^22–23 However, most risk for algorithmic bias is conferred from datasets used for algorithm development; datasets that were created by human-based structures and systems. The same data science methods that empower AI can provide a means to interrogate, expose, and understand existing bias. Once uncovered, AI-based methods can be used to mitigate bias.^24,25 This includes the power to intervene on potential biases directly at the point-of-care. The need for such an approach to ED triage is clear.

Nearly a decade ago, through a federally funded collaboration between data scientists, emergency nurses and physicians, our institution developed a CDS tool that leverages AI to generate risk-driven triage acuity recommendations embedded into the EHR workflow.^11,12 In 2017, we implemented this tool in place of ESI. Using it, we have been able to more reliably identify patients with critical illness and reduce the time these patients wait for care. We have decreased the proportion of patients allocated to mid-point Level 3 by increasing our usage of Levels 4 and 5—without increasing risk or length of stay for this low-acuity group.^26,27 Our data-driven approach has also generated outcome-rich data streams that inform quality and nursing leadership and facilitate practice-based learning. In 2018, the tool became the cornerstone for our department’s Nursing Magnet Designation. With support from the National Science Foundation, it has since been commercialized and is now being introduced to other EDs worldwide.

Dr. Hinson is an associate professor of emergency medicine and co-director of the Center for Data Science in emergency medicine at The Johns Hopkins University.

Dr. Levin is the senior director for Research and Innovation for the Clinical Decision Support Solutions Unit at Beckman Coulter Diagnostics.

References

1. Calder LA, Forster AJ, Stiell IG, et al. Mapping out the emergency department disposition decision for high-acuity patients. Ann Emerg Med. 2012;60(5):567-576. e4.
2. Wiler JL, Gentle C, Halfpenny JM, et al. Optimizing emergency department front-end operations. Annals of emergency medicine. 2010;55(2):142-160.e1.
3. Hinson JS, Martinez DA, Cabral S, et al. Triage Performance in Emergency Medicine: A Systematic Review. Annals of Emergency Medicine. Published online November 22, 2018.
4. Ahrq Q. Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care. 4th ed. (Gilboy N, Tanabe P, Travers D, Rosenau AM, eds.).; 2012.
5. McHugh M, Tanabe P, McClelland M, Khare RK. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Academic Emergency Medicine. 2012;19(1):106-109.
6. Mistry B, Stewart De Ramirez S, Kelen G, et al. Accuracy and Reliability of Emergency Department Triage Using the Emergency Severity Index: An International Multicenter Assessment. Annals of Emergency Medicine. 2018;71(5):581-587.e3.
7. Zhang X, Carabello M, Hill T, Bell SA, Stephenson R, Mahajan P. Trends of Racial/Ethnic Differences in Emergency Department Care Outcomes Among Adults in the United States From 2005 to 2016. Front Med (Lausanne). 2020;7:300.
8. Mistry B, Balhara KS, Hinson JS, et al. Nursing Perceptions of the Emergency Severity Index as a Triage Tool in the United Arab Emirates: A Qualitative Analysis. J Emerg Nurs. 2018;44(4):360-367.
9. Joseph JW, Landry AM, Kennedy M, et al. Association of Race and Ethnicity With Triage Emergency Severity Index Scores and Total Visit Work Relative Value Units for Emergency Department Patients. JAMA Network Open. 2022;5(9):e2231769.
10. Chmielewski N, Moretz J. ESI Triage Distribution in U.S. Emergency Departments. Adv Emerg Nurs J. 2022;44(1):46-53.
11. Levin S, Toerper M, Hamrock E, et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Annals of Emergency Medicine. 2018;71(5):565-574.
12. Dugas AF, Kirsch TD, Toerper M, et al. An Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes. Journal of Emergency Medicine. 2016;50(6):910-918.
13. Sax DR, Warton EM, Mark DG, et al. Evaluation of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage. JAMA Network Open. 2023;6(3):e233404.
14. Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open. 2019;9(5):e026471.
15. Vigil JM, Alcock J, Coulombe P, et al. Ethnic Disparities in Emergency Severity Index Scores among U.S. Veteran’s Affairs Emergency Department Patients. PLoS One. 2015;10(5):e0126792.
16. Platts-Mills TF, Travers D, Biese K, et al. Accuracy of the Emergency Severity Index triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Acad Emerg Med. 2010;17(3):238-243.
17. Grossmann FF, Zumbrunn T, Frauchiger A, Delport K, Bingisser R, Nickel CH. At risk of undertriage? Testing the performance and accuracy of the emergency severity index in older emergency department patients. Ann Emerg Med. 2012;60(3):317-325.e3.
18. Ginsburg AD, Oliveira J E Silva L, Mullan A, et al. Should age be incorporated into the adult triage algorithm in the emergency department? Am J Emerg Med. 2021;46:508-514.
19. Malinovska A, Pitasch L, Geigy N, Nickel CH, Bingisser R. Modification of the Emergency Severity Index Improves Mortality Prediction in Older Patients. West J Emerg Med. 2019;20(4):633-640.
20. Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JMC. Clinical Decision Support Systems for Triage in the Emergency Department using Intelligent Systems: a Review. Artificial Intelligence in Medicine. 2020;102:101762.
21. Miles J, Turner J, Jacques R, Williams J, Mason S. Using machine-learning risk prediction models to triage the acuity of undifferentiated patients entering the emergency care system: a systematic review. Diagnostic and Prognostic Research. 2020;4(1):16.
22. Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern Med. 2018;178(11):1544-1547.
23. Parikh RB, Teeple S, Navathe AS. Addressing Bias in Artificial Intelligence in Health Care. JAMA. 2019;322(24):2377-2378.
24. Yang J, Soltan AAS, Eyre DW, Yang Y, Clifton DA. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digit Med. 2023;6:55.
25. Agarwal R, Bjarnadottir M, Rhue L, et al. Addressing algorithmic bias and the perpetuation of health inequities: An AI bias aware framework. Health Policy and Technology. 2023;12(1):100702.
26. Success Stories From the AHRQ Digital Healthcare Research Program – HopScore: An Electronic Outcomes-Based Emergency Triage System. Published online January 23, 2020. Accessed January 23, 2020.
27. Levin S, Toerper M, Hinson J, et al. Machine-learning-based electronic triage: a prospective evaluation. Annals of Emergency Medicine. 2018;72(4).

Disclosure

Dr. Hinson, Dr. Levin and Johns Hopkins University are entitled to royalty distributions related to CDS technology that was evaluated. This arrangement has been reviewed and approved by Johns Hopkins University in accordance with conflict-of-interest policies.