Nurses Speaking Out for Patient Safety: Ruse or Real Warning
Key Findings: There is a surprisingly strong correlation between the CMS Hospital Quality Stars scores and the CareLuminate Nurse Quality Score. While both measurements attempt to measure hospital care quality, their approaches are extremely different, with the CMS Stars score measuring an aggregate of quantifiable documented outcomes and patient experience, while the CareLuminate Nurse Quality Score measures RN observations of quality.
These findings strongly suggest that the combined nurse voice should be considered a valid quality indicator for hospitals, and that when nurses speak out together about quality concerns, their voice should be taken seriously as it likely points to real quality failings.
What should policy makers and the public think when nurses strike for safety reasons? Are nurses good judges of safety concerns in hospitals?
CareLuminate is a new research company that is interviewing hundreds, and soon to be thousands, of nurses across the country, regionally reporting on hospital care quality through the eyes of RNs (see our recent Stockton CA report: https://www.careluminate.com/san-joaquin-county-results). CareLuminate is not a nurse advocacy organization—instead CareLuminate reports through the eyes of RNs to educate the public and improve care quality.
Do nurses really know safety issues?
The Case For Nurses Knowing Quality
Nurses have the front-row seat to observe all aspects of care delivery in a hospital. They are highly trained, they spend more time with patients than any other group, and they are involved in every aspect of care delivery from transitions of care to surgery. They are also known for being strong and trusted patient care advocates.
The Case For Nurses Not Knowing Quality
While highly educated, nurses may misinterpret care quality according to existing methods of quality evaluation like CMS Stars and HCAPHS. Their role is often not focused across the organization, but rather one of giving care at the bedside. Some have claimed the nurses can become so focused on their own needs, wages or friction with administration that they lose their impartiality to judge care quality.
Correlating to the CMS Five Star Quality Rating System
To test whether nurses are a voice for patient care quality, we compare their feedback by hospital to the CMS Hospital Five-Star Quality Ratings.
Multiple organizations publish quality ratings for hospitals, including a host of measurements from the Centers for Medicare and Medicaid services, as well as some independent quality measurements. Healthcare delivery is complicated, and no measurement is perfect, but two of the most comprehensive, impactful and well-known measurements for hospitals are the CMS Five-Star Quality Rating System and the CMS Hospital HCAHPS measurements (also on a 5-point scale). The CMS Stars ratings incorporate aspects of the HCAHPS ratings into the overall measurement.
A comparison of the CMS Five Star Quality Rating System to the CarLumiante Nurse Voice Quality Rating shows differences between how these two measurements are collected.
CMS Five Star Quality Rating System
-
What They Are: Summary of a variety of CMS-mandated measurements across 5 areas of quality into a single star ratings: mortality, safety, readmission, patient experience, and timely & effective care.
-
Reporting: Measures are grouped, standardized, weighted, assigned thresholds based on ‘peer groups’ and clustered through an algorithm to assign the final ratings.
-
Collection Timeline: No data is newer than 6 months old and in some cases is 4+ years old.
CareLuminate Nurse Voice Quality Rating
-
What They Are: CareLuminate works to consistently interview RNs at different hospitals, asking them questions about their likelihood to recommend, and confidence their hospital’s outcomes, safety, teamwork, compassion of care, staffing, updated equipment/protocols, administration, transitions of care, and communication.
-
Reporting: All questions are equally weighted into an overall score which represents the percent of questions that respondents answered with a positive or ‘agree’ answer.
-
Collection Timeline: All data collected within the last 4 months.
Source: https://data.cms.gov/provider-data/topics/hospitals/overall-hospital-quality-star-rating
Both the CMS Star Ratings and the CareLuminate Quality Score attempt to measure the same thing: the quality of care given to patients. However, their approaches could not be more different. Given the different timelines for collection, and the vastly different measurement approaches, we would expect significant differences between these measurements.
Instead what we found was a dramatically strong correlation. In an ordinary weighted least squares regression of the CMS hospital stars against the CareLuminate Quality Score, the correlation was both highly statistically significant the coefficient was very large.
Ordinary Weighted Least Squares Regression Results of CMS Stars Score (Independent Variable) Against the CareLuminate Quality Score (Dependent Variable), Weighted by the Number of CareLuminate RN Interviews Per Hospital
.png)
The magnitude of this correlation was surprising even for the CareLuminate team, who has a strong belief in the power of RN feedback. While the correlation did not explain all variation in CMS Stars scores (r-squared = .33), the coefficient scale was large.
That two measures correlate so tightly across different time periods of measurement felt suspicious for the CareLuminate researchers, so we compared the CMS Stars Scores of past years to the recent CareLuminate Quality Score. The results showed that the correlation sharply drops as the stars measurements are regressed in time.
Because the CareLuminate Nurse Quality Score is a composite score, we compared each of the CareLuminate interview questions against the most recent CMS Stars Ratings in a series of independent regressions, with the dependent variable being the CMS Stars Hospital Quality Score and the dependent variable being the percent of nurses that answer the question with a positive or ‘agree’ answer (for some questions a ‘disagree’ answer is a positive answer).
The findings from this analysis are noteworthy. All questions except for the question about RN compensation were found to have a statistically significant correlation to the CMS Stars (p<.05). Overall, the care quality questions were found to have a stronger relationship than the RN work experience questions. Interestingly, two of the highest rated questions about compassion and teamwork were found to have the strongest explanatory power, indicating that when a hospital fails in these two areas it is an indicator that overall care is failing.
Conclusion
CareLuminate will continue to update this research as more data is collected on additional hospitals, but these early findings strongly suggest that the collective nurse voice should be considered an important care quality indicator.
These findings are great news for hospitals since the nurse voice does not just offer statistics about care quality strengths or concerns, but the nurses offer powerful insights about specific opportunities for care quality improvement.
