Hospital Ratings Affected by Statistical Noise
Of the hospitals originally classified in the “best” performance quartile, 51% dropped to a lower quartile after adjusting for statistical noise. Of the hospitals that traditional classification placed in the “worst” quartile, 26% rose to a higher category after adjusting for noise… Overall, the adjustment moved 43% of hospitals into a different quartile of performance…
Full article on statistical “noise” in hospital quality-of-care scorecards.
One more reason not to trust quality ratings.
The problem with trying to establish quality metrics that reflect hospital quality is that nobody can seemingly agree on which metrics reflect quality. Moreover, some of the metrics that are associated with quality might not lead to better quality when hospitals put effort into boosting those specific metrics. For instance, all high-quality hospitals may emphasize hand washing. But merely emphasizing hand washing might not improve quality. Rather it may be a byproduct.
So there is a good reason for patients to ignore quality information.
Everyone games the metrics anyway. Assuming that we could come to some agreement about the proper metrics, we would then have to deal with hospital staff doing things to artificially inflate scores.