Poll Position – How You Ask the Question Matters

A recent article I was reading began like this: “Americans don’t think health care reform is working, and they are especially dubious of government priorities, according to a survey released on Monday by the Deloitte Center for Health Solutions.”

A recent article I was reading began like this: “Americans don’t think health care reform is working, and they are especially dubious of government priorities, according to a survey released on Monday by the Deloitte Center for Health Solutions.”

A lot of folks stop reading right there–or at least stop thinking right there. In fact, many probably don’t get beyond the phrase “American’s don’t think health care reform is working.” Period. End of sentence. Such tendencies are especially strong in a world besieged by endless streams of nearly instantaneous information. We don’t have (or don’t take) the time to think critically about the information presented to us. It is unfortunate to say the least.

The phrase “Americans don’t think health care reform is working” seems to suggest that all Americans espouse that view. In fact, however, you have to keep reading to find that 49% consider the ACA a good start, while 30% consider it “a step in the wrong direction.” That’s a lot more mixed than the opening sentence would suggest. Indeed, the same survey finds that 76% of consumers do not have a strong understanding of how the health care system works. I could, if I chose, use that result to amend the original headline: “More than 75% of Americans don’t understand how the health care system works, but know ‘Obamacare’ won’t fix it.” Framing matters, and that starts with the polling firm. Ask yourself: Who actually conducted the poll? Do they have a known bias? Were they paid on behalf of an organization with a political agenda?

Ultimately, you need to go directly to the poll itself. Here you’ll find not only the data (hopefully), but the instruments used to collect that data. The data is important for the simple reason that figures can be misleading. For example, percentages may look impressive, but may be trivial in nominal terms, or vice versa. Just as important, however, is question wording. The public rarely scrutinizes polls at this level, but the way in which questions are worded can have a profound effect on how people answer the question. Ask people if government should define marriage as an act between a man and a woman, and you’ll get a much more favorable response than you will if you ask whether they believe government should make it illegal for two persons of the same sex to marry each other. The first instance presents a generally favorable idea being supported by government. The second instance asks the same thing in a negative light by focusing on government limiting individuals’ rights. These framing effects are well known, and they often lead to polling results being oversold.

David Grande, Sarah Gollust, and David Asch, all of the University of Pennsylvania, have a wonderful article out in the latest issue of Health Affairs that looks at the various ways in which polls about health reform worded questions pertaining to the individual mandate and the public option. They found what polling experts have long known: How you ask the question matters. Their conclusion–looking across all the different polls–was that “Americans support an expanded role for government in health care that provides more choices and makes insurance more affordable. However, Americans do not want to see the government assume a more prominent role as a dictator of individual decisions.” I think their synthesis of a wide variety of polling data effectively hits the nail on the head, and tells us far more about the American public’s views than any single poll taken as a snapshot of the gospel.