You Might Also Like

• 2015 Emergency Department Survey Shows Spike in Volume, Structural Changes, Patient Boarding Concerns
• ACEP Leadership Team Meets with the Centers for Medicare & Medicaid Services, Joint Commission, American Hospital Association, and More
• National Hospital Ambulatory Medical Care Survey Data Show Increase in Emergency Department Visits

Explore This Issue

ACEP Now: Vol 37 – No 06 – June 2018

ILLUSTRATION: Chris Whissen & shutterstock.com

It is no secret that emergency physicians have been unhappy about patient satisfaction surveys. While there is much evidence in the medical literature that the patient experience and patient clinical outcomes are interdependent, the surveys that have measured “patient satisfaction” have suffered from significant methodological flaws including:

• Too many questions leading to a poor response rate (anywhere from 3 to 17 percent, averaging around 10 to 11 percent).
• Inappropriate questions such as, “During your ER visit, did the doctors and nurses do everything they could to help you with your pain?” while in the midst of an opioid epidemic.
• An inappropriately small sample size leading to wide variation in scores and questions regarding statistical validity.
• A delay of six to eight weeks in the survey results, making the response to patient perception difficult in terms of performance improvement.
• The inability to give individual physicians actionable feedback unless the sample is gathered over six to 12 months, with too long a lapse by the time the physician receives feedback.

On top of those issues, use of the survey results by hospitals and physician employers has been problematic due to their decisions to implement:

• Physician credentialing based on patient experience as one of the criteria, when the sample size is far too small.
• Payment incentives based on ED patient satisfaction when physician communication and behavior is one of a multitude of factors, over most of which the emergency physician has little or no control.

History of Hospital Satisfaction Surveys

Patient satisfaction surveys have been in existence since the mid 1980s, and many physicians who were practicing at that time remember that even back then there was intense pressure from senior hospital leaders to improve patient satisfaction as a way to grow market share and build the financial bottom line.

More than 10 years ago, the Centers for Medicare and Medicaid Services (CMS) implemented the Hospital Consumer Assessment of Health Care Providers and Systems (HCAHPS) survey. This survey, on inpatients age 18 and older who are discharged home, has been the foundation on which other surveys, such as the Clinician and Group CAHPS survey (which measures patients seen in the outpatient clinic/physician office setting), have been built. For the past four to five years, CMS, utilizing the RAND Corporation, has been working on the government-sponsored ED survey, which now has a new name, the ED Patient Experience of Care (EDPEC) survey. While initial development included versions to be used for admitted as well as discharged patients, the survey in current development is only for patients “discharged to community” (DTC).

Physician leaders of ACEP have been active in the ongoing transformation of that survey. In October 2014, we wrote to CMS: “On behalf of more than 33,000 members and the 136 million patients seen annually in the nation’s emergency departments, the American College of Emergency Physicians (ACEP) appreciates the opportunity to provide comments on the latest revision to the draft survey instrument, now titled ‘Emergency Department Patient Experience of Care’ (EDPEC). Overall, we urge CMS to rethink the objectives of gathering this information (aside from fulfilling the ACA mandate) and articulate how the information will advance and improve patient care.”

The original draft survey had 53 questions, of which 32 were about care and 21 were demographics. The questions on physicians included the following:

• “During this emergency room visit, how often did doctors treat you with courtesy and respect?”
• “During this emergency room visit, how often did doctors listen carefully to you?”
• “During this emergency room visit, how often did doctors explain things in a way you could understand?”
• “During this emergency room visit, how often did doctors spend enough time with you?”

There were four possible answers: Never, Sometimes, Usually, and Always. However, the only answer that counted as positive was “Always.”

The questions on pain were:

• “During this emergency room visit, did you have any pain?”
• “During this emergency room visit, did you get medicine for pain?”
• “During this emergency room visit, did the doctors and nurses do everything they could to help you with your pain?”

Patients could choose one of three responses: Yes, definitely; Yes, somewhat; and No. The only response that counted as positive was “Yes, definitely.”

ACEP’s Response

CMS asked for comments from the medical community. William Sullivan, DO, JD, FACEP, a member of ACEP’s Medical Legal Committee, clinical assistant professor emergency medicine at the University of Illinois at Chicago, attending physician at St. Margaret’s Hospital in Spring Valley, Illinois, and owner of Sullivan Law Office in Frankfort, Illinois, and I, in conjunction with Barbara Tomar, ACEP’s former director of regulatory affairs who is now retired, and ACEP’s Washington, D.C., office, did not just make comments but rewrote the survey for CMS. We suggested that 53 questions were too many, some were duplicative, and some were poorly worded.

We suggested a maximum of 15 questions. We recommended that the pain questions be removed entirely, as we were seeing the effects of the opioid epidemic and the pressure we had received for many years to treat “pain as the fifth vital sign.” We advised that “emergency room” should be changed to “emergency department” and that all of the questions should have the three-response option of “Yes, definitely; Yes, somewhat and No,” in view of the ED visit being a one-time event unlike the multiday inpatient experience.

CMS accepted some of our suggestions. It reduced the number of questions from 53 to 43, and it removed the question, “How often did doctors spend enough time with you?” It refused to change “emergency room” to “emergency department,” but it did change the pain questions to the following:

• “During this emergency room visit, did you have any pain?”
• “During this emergency room visit, did the doctors and nurses try to help reduce your pain?”
• “During this emergency room visit, did you get medicine for pain?”
• “Before giving you pain medicine, did the doctors and nurses describe possible side effects in a way you could understand?”

Physician leaders of ACEP have been active in the ongoing transformation of [the EDPEC] survey.

Testing and Revision

Then CMS field-tested its draft survey on a selected number of emergency departments.

In 2017, CMS convened a technical expert panel on the EDPEC survey. Thom Mayer, MD, FACEP, executive vice president of EmCare, founder and CEO of Best Practices, Inc., and clinical professor of emergency medicine at George Washington University in Washington, D.C., and I served on that panel. We had a several-hour conference call and follow-up communication. The newest draft published in August 2017 has a total of 39 questions (29 questions about the care, 10 demographic), and these are the pain questions:

• “During this emergency room visit, did you have any pain?”
• “During this emergency room visit, did the doctors or nurses talk with you about how much pain you had?”

In a later section under “Leaving the Emergency Room,” there are additional questions regarding pain:

• “Before you left the emergency room, did the doctors or nurses give you as much information as you wanted about how to treat your pain at home?”
• “Before you left the emergency room, did the doctors or nurses talk with you about things you could do at home to reduce your pain other than take medicine?”

Response options include the three “Yes, definitely; Yes, somewhat; and No” and also a fourth option, “I did not need to reduce pain after I got home from the emergency room.”

We hope that emergency physicians will view this as a major win for us. While the pain questions are not completely gone, they are much improved, and there is a question on the nonpharmaceutical treatment of pain.

CMS and the RAND EDPEC team have been working with us to make the survey appropriate and meaningful. The second round of feasibility testing for the EDPEC DTC survey is now being completed. The testing has a push-to-web focus. This round of feasibility testing is examining issues such as:

• Survey response rates and differences by survey administration arm
• Email and mobile phone coverage rates
• Feasibility of email and text invitations to a web survey
• Testing of the use of a QR code in a mailed invitation
• Paradata, such as how long the surveys (and individual items) take, what type of device individuals use to respond to web surveys, whether people are changing their answers, etc.

CMS and the RAND EDPEC team have asked us to not share the current draft of the survey, as the survey will continue to evolve in response to results from feasibility testing. They will be holding another round of technical expert panels this summer once preliminary data from the field testing are available, and we will again serve on that panel to represent the views of emergency physicians and the patients we serve.

There is currently no defined timeline for implementation of the CMS EDPEC survey, but it will likely happen within the next year. If we can move in the direction of more rapid information feedback and a more representative sample of our patient population, both emergency physicians and our patients will benefit. The entire purpose of the survey should be to give emergency physicians and nurses information they can act upon to improve the ED patient experience and, by doing so, increase patient compliance with recommended therapies and improve clinical outcomes. Stay tuned; we will keep you informed.

Dr. Kaplan is medical director of care transformation at LCMC Health in New Orleans, clinical associate professor of medicine in the section of emergency medicine at Louisiana State University Health Sciences Center, University Medical Center New Orleans, and a Past President of ACEP.