In 1999, the Institute of Medicine published a report entitled, “To Err Is Human,” that estimated 44,000 to 98,000 patients die annually in hospitals due to medical error.¹ In 2016, a sensational publication claimed medical error as the third-leading cause of death in the United States.² Now, a new systematic review published by the Agency for Healthcare Research and Quality (AHRQ) has put the emergency department (ED) in its crosshairs.³

This AHRQ review claims diagnostic error occurs in nearly one in 18 ED patients, resulting in 2.6 million adverse events with 370,000 serious harms, including 250,000 deaths. The response to this article has been swift, with all the major emergency medicine professional societies signing on to a response conveying their dismay.⁴ This dismay is not grounded in the harsh focus on the state of emergency medicine practice, but on the flawed analysis itself.

The core issues repeatedly raised involve the studies used to estimate the frequency of diagnostic error. The authors of the AHRQ review generate their estimate from three small studies examining the outcomes of a mere 1,758 patients in Spain, Switzerland, and Canada.

Two of these studies form the basis for their estimate of the rate of diagnostic error occurring in the ED. Only one of these studies specifically measures error in those discharged from the ED. This study was conducted over 15 years ago in Tenerife, in the Canary Islands, an archipelago off the coast of Morocco and collectively administered as an autonomous community of Spain.⁵ It reviewed outcomes of 500 patients, specifically selecting half from those having an unscheduled 72-hour return to the ED. The practice environment differed substantially from the U.S., with nearly 90 percent of patients evaluated by residents and non-emergency physician staff, with these physicians averaging three patients per hour in a seasonally overcrowded department. Blood tests were performed on only approximately half of patients and CTs on a mere two percent.

Within the cohort of unscheduled 72-hour returns, 20 percent displayed discordant diagnoses between initial and subsequent ED visits, while those without unscheduled returns displayed a four percent rate of discordant diagnoses at primary health center follow-up. Even if the decades-old performance of this remote archipelago were a reasonable proxy for modern U.S. medical care, it is immediately obvious a discrepantly coded diagnosis is not a reliable surrogate for diagnostic error. A handful of core definitions of diagnostic error exist, including one from the National Academy of Medicine, and each requires full case review to determine missed or delayed opportunities to make a correct or timely diagnosis.⁶ Absent any sort of structured review, no accurate estimate for the rate of diagnostic error can be ascertained.

A second study of patients admitted to the internal medicine service in Switzerland uses the same approach of looking for diagnostic discrepancies.⁷ Inselspital University Hospital in Berne, Switzerland, is a university hospital of approximately 40,000 annual visits staffed by an interdisciplinary unit within the ED, intensive care, and anesthesiology. In this study, clinicians reviewed ED admitting and IM discharge diagnoses to classify them as discrepant, clearly stating, “this study investigated discrepancies in diagnoses, not error, which would require a thorough review of the diagnostic process.” These authors ultimately identified discrepancies between admitting and discharge diagnoses in 12.3 percent of cases, three-quarters of which did not have the ED admitting diagnosis among the discharge diagnoses. The authors of the AHRQ review subsequently blend the diagnostic discrepancies from these two studies, from 15 years ago in the Canary Islands and seven years ago in Berne, Switzerland, to produce an overall 5.7 percent rate for diagnostic error in present-day U.S. EDs.

Attempting to Ascertain the Rate of Harm

The next portion of the AHRQ review uses these studies, and a third one from Canada, in an attempt to ascertain the rate of harm from diagnostic error.⁸ In the Canadian study conducted in 2004, follow-up was performed on 503 adult patients evaluated in “high acuity” areas of the ED. Of these patients, a single death was observed. A patient with chest pain was referred to cardiology with an elevated troponin level, but was subsequently diagnosed with, and died as a result of, an aortic dissection. As the sole “high-quality” study directly addressing the question of death from diagnostic error, this 0.2 percent estimate forms the foundation of all subsequent estimates.

A true 95 percent confidence interval from this study would generate an absurdly uninformative estimate of annual deaths attributed to diagnostic error ranging from 6,500 to 1.4 million. However, rather than admit this insurmountable limitation, the authors of the AHRQ review invent a new biostatistic, the “plausible range.” This made-up descriptor has no rigorous precedent, but rather represents an arbitrary ±2-fold range ultimately made up around the single death from aortic dissection observed in Canada.

To support this supposed “plausible range,” the authors construct increasingly specious chains of extrapolation from the small numbers of events in these studies from nearly 20 years ago, along with other mathematical calisthenics based on population-based death statistics. These other data are inappropriately elevated above observational studies conducted in ED populations, with estimates of death due to diagnostic error ranging from merely zero percent to 0.0074 percent.^9,10,11 With up to 1000-fold differences in estimates between data sources, the clear answer is the lack thereof: the true incidence and harms from diagnostic error across the heterogenous ED landscape in the U.S. remains unknown. It is irresponsible to publish an estimate and to assume such precision.

Diagnosis-specific estimates also suffer from similar issues, particularly with respect to the incidence of stroke in patients presenting to the ED with dizziness. The authors of the AHRQ review, in repeated and prominent calls for future research in their personal area of academic work, cite an “estimated 45,000 to 75,000” missed strokes in dizziness annually. The source for this statistic in the AHRQ review is, in fact, an editorial by these same authors, which subsequently cites their own practice seminar article regarding the HINTS (Head Impulse, Nystagmus, Test of Skew) exam, which ultimately extrapolates data from a study of patients with dizziness.¹²

This retrospective study from Neuces County, Texas, was conducted in 2000-2003, and focuses on 53 patients who were adjudicated to have had a cerebrovascular diagnosis following chart review. The AHRQ authors’ analyses of this study neglect to mention a third of these patients were not diagnosed with stroke, but with transient ischemic attack, and it is the mere 46 ED cases from this cohort forming the foundation for the proposed rate of missed strokes in modern clinical evaluation. It is absolutely the case patients with dizziness can manifest underappreciated etiologies, but studies of over 40,000 patients provide estimates of subsequent stroke diagnoses of 0.18 percent within 30 days, a far cry from tens of thousands of missed strokes.¹³

Finally, the accounting of the frequency of various clinical conditions affected by diagnostic error is derived substantially from a U.S. database of closed malpractice claims. While there is certainly alluring face validity to serious harms percolating to the level of a tort claim, these data cannot realistically inform any sort of reliable estimate of relative disease-specific errors. Likewise, using these tort data to approximate estimates of the frequency of types of diagnostic error is likewise invalid. The authors of the AHRQ admit as much in the text, but do not refrain from heavily utilizing this citation.

Unsupported and Misleading

The field of diagnostic error, patient safety, and cognitive biases in medicine is of profound importance to the specialty of emergency medicine. These issues of diagnostic accuracy must also be considered within the challenges of resource stewardship, overdiagnosis, and unintended consequences. The findings promulgated by this AHRQ review are, bluntly, unsupported by the evidence cited and misleading as to the gaps requiring further study. In light of the comprehensive issues marring this publication, I personally believe it should be retracted for further revision.

Dr. Radecki is an emergency physician and informatician with Christchurch Hospital in Christchurch, New Zealand. He is the Annals of Emergency Medicine podcast co-host and Journal Club editor and can be found on Twitter @emlitofnote.

References

1. Institute of Medicine (US) Committee on Quality of Health Care in America, Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington (DC): National Academies Press (US); 2000.
2. Makary M A, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016;353:i2139.
3. Newman-Toker DE, Peterson SM, Badihian S, et al. Diagnostic errors in the Emergency Department: A systematic review. Effective Health Care (EHC) Program. https://effectivehealthcare.ahrq.gov/products/diagnostic-errors-emergency/research. Accessed January 25, 2023.
4. Multi-Organizational Letter Regarding AHRQ Report on Diagnostic Errors in the Emergency Department.
5. Nuñez S, Hexdall A, Aguirre-Jaime A. Unscheduled returns to the emergency department: an outcome of medical errors? Qual Saf Health Care. 2006;15(2):102-108.
6. Giardina TD, Hunte H, Hill MA, et al. Defining diagnostic error: a scoping review to assess the impact of the national academies’ report improving diagnosis in health care. J Patient Saf. 2022;18(8):770-778.
7. Hautz WE, Kämmer JE, Hautz SC, et al. Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room. Scand J Trauma Resusc Emerg Med. 2019;27(1):54.
8. Calder LA, Forster A, Nelson M, et al. Adverse events among patients registered in high-acuity areas of the emergency department: a prospective cohort study. CJEM. 2010;12(5):421-430.
9. Aaronson E, Borczuk P, Benzer T, et al. 72h returns: A trigger tool for diagnostic error. The American journal of emergency medicine. 2018;36(3):359-61.
10. Vanbrabant P, Knockaert D. Short-term return visits of ‘general internal medicine’ patients to the emergency department: extent and risk factors. Acta clinica Belgica. 2009;64(5):423-8.
11. Calder L, Pozgay A, Riff S, et al. Adverse events in patients with return emergency department visits. BMJ quality & safety. 2015;24(2):142-8.
12. Kerber KA, Brown DL, Lisabeth LD, et al. Stroke among patients with dizziness, vertigo, and imbalance in the emergency department: a population-based study. Stroke. 2006;37(10):2484-2487.
13. Atzema CL, Grewal K, Lu H, et al. Outcomes among patients discharged from the emergency department with a diagnosis of peripheral vertigo. Ann Neurol. 2016;79(1):32-41.