AI Triage in the Emergency Department, What It Does Well and Where It Fails

AI triage in the emergency department does one thing well and one thing badly, and the two are easy to confuse. It can rank incoming patients by their risk of deterioration or admission more sharply than the acuity scales in use today. It has also, in its most widely deployed form, failed in practice, firing unreliable alerts that clinicians learned to ignore. The evidence points to a narrow and defensible role, decision support that a clinician validates and acts on, and away from the autonomous alerting that vendors tend to sell.

Triage is a sorting problem, and the current tools are already imperfect

Triage is the first clinical decision in emergency care, sorting undifferentiated patients by how urgently they need attention. Most departments use a structured scale, the Emergency Severity Index (ESI) in the United States, the Canadian Triage and Acuity Scale (CTAS), the Manchester Triage System (MTS) across much of Europe, or the South African Triage Scale (SATS) in many lower-resource settings. These are rule-based scores a nurse applies in a minute or two, and they are imperfect by design. A 2025 systematic review and meta-analysis of the ESI across 27 studies and 510,777 patients found it identifies critically ill adults with useful but incomplete accuracy (strong evidence, meta-analysis) (Diagnostic test accuracy of the ESI, 2025). In validation work on the South African scale, under-triage and over-triage rates across some clinical vignettes ranged widely, a reminder that the human baseline AI is measured against already leaks in both directions (moderate evidence, validation studies) (Reliability and validity of SATS in low-resource settings). This is the gap AI is meant to close.

Where machine learning genuinely beats the legacy scales

The strongest case for AI triage is discrimination, separating who will deteriorate from who will not. A machine-learning electronic triage system developed at Johns Hopkins differentiated patients by clinical outcome more accurately than the ESI, and a later multi-site implementation reported gains without widening disparities across patient groups (moderate evidence, prospective development plus multi-site implementation) (Annals of Emergency Medicine, 2024). Outside the United States, an XGBoost model trained on 163,452 visits at a Thai hospital predicted ICU admission and resource use better than CTAS, with the area under the ROC curve rising to 0.917 from 0.882 and, more tellingly for a rare outcome, precision-recall rising to 0.629 from 0.333 (limited evidence, single-center retrospective, internal validation only) (Scientific Reports, 2025).

Two caveats keep this honest. First, better discrimination is a prediction result, not proof that patients do better. A 2025 systematic review of AI triage found only six studies that met inclusion criteria and concluded that real-world clinical impact remains inadequately characterized (limited evidence, systematic review) (Cureus, 2025). Second, model quality varies. A 2024 systematic review of machine learning and natural language processing for ED triage found promising accuracy but a high risk of bias across studies when assessed with a formal tool (moderate evidence, systematic review) (BMC Emergency Medicine, 2024). Sharper ranking is real. Proven benefit at the bedside is not yet established.

The cautionary tale, a deployed model that clinicians learned to ignore

The most instructive failure in this field is the Epic Sepsis Model, a proprietary alert built into a widely used electronic health record to warn clinicians of impending sepsis. An external validation at Michigan Medicine covering 38,455 hospitalizations found an area under the curve of 0.63, well below the 0.76 to 0.83 the vendor reported, with 33 percent sensitivity and a 12 percent positive predictive value (strong evidence, external validation) (Wong et al., JAMA Internal Medicine, 2021). In the emergency setting the picture was worse. A retrospective validation across two county EDs and 145,885 encounters found 14.7 percent sensitivity, a 7.6 percent positive predictive value, and a median alert lead time of zero minutes, meaning the alert often arrived no earlier than the clinician's own recognition (moderate evidence, external validation) (External validation in 2 county EDs, 2024). The practical result was alert fatigue, a flood of low-value pop-ups that staff learn to dismiss. A later multicenter validation of an updated version, spanning 227,091 encounters across four health systems, reported improved discrimination but persistent institutional variability, low positive predictive value, and a heavy alert burden (moderate evidence, multicenter validation) (External validation of a localizable sepsis model, 2025). The lesson is not that prediction is impossible. It is that a model validated in one system can behave very differently in another, and deploying it without local validation is a clinical risk, not a shortcut.

Large language models at the front desk overtriage and drift

The newest entrant is the general-purpose large language model, prompted to assign an acuity level from a patient's description. The evidence says not yet. In a comparative study of 124 emergency vignettes, GPT-4 agreed with expert triage about as well as untrained doctors and less well than trained raters, and the models tended to overtriage while untrained doctors undertriaged (moderate evidence, comparative study) (Masanneck et al., JMIR, 2024). A disaster-triage study using the START protocol found ChatGPT reached only 63.9 percent accuracy with a 32.9 percent overtriage rate, and its answers shifted with the wording of the prompt, ranging from 46.7 to 71.8 percent (limited evidence, simulation study) (Franc et al., JMIR, 2024). Overtriage is the safer error, but at scale it clogs the very queue triage exists to protect, and prompt sensitivity means the same patient can be sorted differently depending on phrasing. These tools are not ready to sit at the front desk unsupervised.

The evidence gap that matters most sits in the crowded, low-resource ED

Here is the finding most relevant to the health systems Curely builds for, and the one most often skipped. Almost all of this evidence comes from well-resourced hospitals in high-income countries. The setting where triage carries the highest stakes, an overcrowded department with one clinician for hundreds of waiting patients, is barely represented in the AI triage literature. The tools purpose-built for these settings are still mostly rule-based, such as the South African Triage Scale, developed for lower-resource African emergency centers and used by Médecins Sans Frontières, which shows reasonable validity for adults and trauma and weaker performance in paediatric use (moderate evidence, validation studies) (SATS for mortality prediction in resource-constrained settings). A model that reliably flags the sickest patient in a crowded queue would matter most exactly there, and that is also where local data is thinnest, connectivity is least reliable, and an unvalidated model is most likely to fail quietly. Closing that gap is a research problem before it is a product, which is why it belongs on our research agenda rather than in a launch announcement.

What a defensible deployment looks like

The verdict is specific. AI can rank emergency patients by risk more sharply than the scales in use today, and that ranking is genuinely useful as a second read for an experienced clinician. It is not ready to alert autonomously, and the one model deployed at scale to try shows why. So the test before deploying any triage model is concrete. Has it been validated on our own population, not just the vendor's. Does it support a clinician's decision rather than replace it. Is its alert rate low enough that staff will still trust it in month six. And can we measure whether patients are actually better off, not just whether the model's numbers look good. Where those four hold, an AI triage tool earns its place at the door, and its role is to help a person decide faster, not to decide for them. That is the version worth building, and it is the version AI clinical decision support should aim for.