Teaching Hospital Comparisons Benefit from Context

The Dartmouth Atlas Project released a study on October 30, 2012, entitled, “What Kind of Physician Will You Be? Variation in Health Care and Its Importance for Residency Training.” The data, and the conclusions of the report’s authors, bear closer examination.

The data set focuses on the differences in the intensity of services that Medicare patients receive, and highlights the academic medical centers rated by U.S. News and World Report as the best hospitals for clinical excellence in 2012-13. The 24-page report, authored by two Geisel Medical School students, was issued to inform medical students about the practices at teaching hospitals where they may receive further training.

All data is helpful, but it only begins the conversation. The authors hold underlying assumptions and perpetrate inconsistencies in their analysis that should be addressed. I don’t suggest dismissing their findings, but rather proceeding with caution.

First, the authors include publicly available information from Hospital Compare , but don’t highlight patient mortality as a quality indicator. Of the seven hospitals in the United States that have attained four successive “Better than U.S. National Rate” ratings for all three 30-day death measures on heart attack, heart failure, and pneumonia patients, several are prominently displayed at the top of the “intensity of care” list.

These hospitals, and their rank on the Dartmouth list, are:

What does it mean to be on that list? For the 2012 update, to achieve even a single “Better than U.S. National Rate” rating, a hospital must be in the top 2 to 4 percent. There are 22 facilities that achieved all three “Better” ratings — about 0.45 percent of nearly 5,000 hospitals reporting. These seven hospitals are better than 99.85 percent of the reporting hospitals — an extremely difficult achievement! (The data cited here is courtesy of Alein Chun, PhD, IQCP Manager, Data Quality Management Unit (DQMU), Cedars-Sinai.)

This begs the question, does a greater intensity of care lead to decrease in mortality in complex patients?

Context for the data is extremely important — and enlightening. New York’s Mount Sinai scores very low on the metric reflecting the percentage of patients referred to hospice in the last six months of life. However, Mount Sinai has one of the premier palliative care programs in the country. The inpatient surgical rate for knee replacement was highest in Salt Lake City, Utah. Does that data reflect a population that self-selects, because many 65-year-old ski bums have winter homes in the Wasatch Mountains?

Data is critical for generating the information we need to identify problems and create solutions, to better manage the health of populations and best deploy our resources. But data is not, in itself, knowledge. It must be analyzed, put into context, and thoughtfully debated to become information, which is then deployed to knowledge. The release of the Dartmouth Atlas Project report creates a great opportunity for all teaching hospitals to reflect on how they are delivering care; and to begin the important conversations about the differences in regional and population utilization.