September 15, 2009

Misinformation 101: Spin Influences Debates

Posted in health care, media coverage tagged , , , , , , , , , at 6:41 pm by realitytax

Polling can reliably reveal whatever the person who constructs/conducts the poll was investigating – if we’re given the raw data and a good description of the sampling procedure. But in practice, the data is usually glossed over in favor of a sound-bite summary tending to support the interests of the person and/or network doing the reporting on it, and the description of the sample is only included by the most rigorous of editors.

Without knowing how the sample of people was selected (and randomized) you simply can’t know anything more than what’s reported about a poll. You can’t know, for instance, if its findings are useful in any logical sense, because you don’t know who the sample represents.

It’s all in the design

I can ask 13, 17, or 21 people a question, and come back with convincing-looking numbers that don’t look “too round.” But if I select who most of those people are it will darn sure tell you what I want you to think I learned.

An example of shaping a poll

Imagine I go to a GOP Town Hall meeting, and survey 15 people wearing shirts or carrying signs that say either “Nobama,” or, “Joe Wilson was right!” I’ll ask them one simple question:

Are you a) “for” Obama’s government takeover of our health care system that he’s pushing through the congress under the name of “reform” or b) “against reform” that will make changes that undermine the free market system that has given us the best health care in the world and cost the tax payers even more money?

OK, I’ve plausibly got 15 “b) against reform” responses now in my hypothetical example.  I’ll ask 6 additional people, more or less randomly selected, that same question. Let’s say for the sake of argument that most of them magically favor reform (not a given the way the question’s phrased, is it?) But for the example say I got 4 out of 6 favorable replies.

Now I’ll summarize the poll for you based on that (fake) survey:

“In a [hypothetical] survey conducted Wednesday, only 19% of those responding favor the proposed reforms to health care, while  nearly 81% said they were ‘against change.’ That’s more than 4 out of 5 in our survey who are hoping their representatives in Congress will stop the President’s take-over of business.”

If you believe what anybody in the media tells you without understanding both the sample and the data, all you know is what the reporter’s boss wants you to believe. If you choose to believe on that basis – which you just might if it agrees with your political leanings – rather than examining the poll itself, then you’re gullible indeed.  The good news is: the politicians on your side and the ratings-hungry networks (who are on the side of earning a living from ad revenues) both love you. They’ll go out of their way to validate your “wisdom and insight” into the issue.

If the poll isn’t conducted on a random sample, but merely open to those who respond…? Well, my friends, that will tell you a bit about the people who responded, of course, but one must be wary of extrapolating to draw any useful conclusions about a larger population. We call it spin. But knowing that they’re gaming us doesn’t stop the echoes.

Media complicity

In fact, it won’t surprise me to find this utterly fake survey example quoted elsewhere within days, if not hours.  Can’t you see it, at DIGG maybe, or on another blog, or even on Fox? Something like:

A post on Wednesday at a liberal-leaning blog about politics and economics described a survey which concluded that, quote, “only 19% of those responding favor the proposed reforms to health care, while nearly 81% said they were ‘against change.’” In other words, that’s more than 4 out of 5 who want their representatives in Congress to stop the President’s assault on insurance providers and let capitalism work.

There you go:  lifted out of context, but the quote is nearly character-for-character what I reported in the fake “summary” above. Now we’re set up for the media echoes to persist even though the numbers are clearly unreal.  Now they’re not reporting on the survey, they’re reporting on the reporting, which is just an irresponsible excuse to keep repeating the misleading numbers. Next thing you know, nobody knows how many people were at that rally on the mall in DC, but everybody believes the numbers support their hopes.

Misinformation distorts any debate. I could easily have made the example go the opposite way, of course, but I don’t want somebody to echo a story that falsely represents support for reform.  In fact, some well-constructed surveys do reveal that over 90% favor “at least some reform.”  But then, who wouldn’t favor “at least some” unless they were making money from the insurance industry? It’s like asking who wants lower taxes without considering how you’d pay for those government services you realize you benefit from.

Are you leading an “unexamined” life?

You know that commercial media outlets rely on advertising revenues. So, do you follow the money? Better still, my favorite (somewhat cynical) question: Why do you trust who you always have to report on things you care about? And yet, those are the sources most people trust to describe the town hall “meetings” as well as the “expert” arguments for and against reforming the health care insurance system.

Digg this post

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: