Editor’s Note: Read Dr. Cedric Dark’s commentary on this EMRA + PolicyRx Health Policy Journal Club article.

Researchers recently leveraged Yelp ratings to provide a general idea of how satisfied patients are with the care they receive in emergency departments and urgent care centers (UCCs).¹ There were three key takeaways. First, people are indeed using Yelp as a way to comment on their experiences in these settings; there were more than 100,000 ratings across both settings between 2005 and 2017. Second, similar to all Yelp business reviews, ratings for both emergency departments and UCCs follow a bimodal pattern, ie, spikes in both 1-star (worst) and 5-star (best) ratings. UCCs had overall higher ratings than emergency departments (47 percent of users gave emergency departments 1-star ratings versus 30 percent for UCCs), and only 27 percent of users gave 5-star ratings to emergency departments (versus 51 percent for UCCs). Third, themes that emerged from the comments can inform what aspects of care correlate with high versus low ratings (eg, good bedside manner was associated with 5-star ratings). 

To make sense of all the comments, researchers employed natural language processing tools to focus on the comments in the extreme ratings (1-star and 5-star), then used differential language analysis to match up the topics that were correlated with low and high ratings. For the emergency departments, topics related to quality of care were more likely to be correlated with high ratings whereas comments on service were more likely to be correlated with low ratings. The opposite was true for UCCs. 

The authors rightfully acknowledge that Yelp can be easily dismissed as a data source since it is unverifiable and unstructured and cannot be considered to be truly representative of all patients. However, Yelp offers rich, narrative data and serves as a platform that allows people to share reflections in real time, unlike publicly inaccessible proprietary data sources such as Press Ganey.

Despite of these limitations, tens of thousands of people feel compelled to share their thoughts through online platforms, and as researchers and clinicians, we should figure out how to best listen and learn from these raw reflections. Further, in an era where patient satisfaction is increasingly considered an important quality measure (despite its perhaps surprising inverse relationship with actual quality of care), it will become important to more effectively analyze all data points—regardless of the source—to inform how we can better care for patients.

Dr. Scott is a medical student at Harvard Medical School in Boston.

Reference

1. Agarwal AK, Mahoney K, Lanza AL, et al. Online ratings of the patient experience: emergency departments versus urgent care centers. Ann Emerg Med. 2019;73(6):631-638.