Misreporting Polls

We’ve often criticized pollsters on various grounds. At least as big a problem as bad polls, however, is bad reporting on polls. Very often, when one reads a news story about a poll and then reads the actual poll results, the poll doesn’t support the story.
It looks like a case in point is currently going around the web. On the Drudge Report is a link that says “POLL: Most Britons believe religion does more harm than good…” That caught my eye, so I followed the link to an AFP story headlined: “Most Britons believe religion does more harm than good: poll.” The story reported on a poll done by the Guardian and ICM, a British polling firm. So I went to the Guardian’s article on the poll; there was the headline again: “Religion does more harm than good – poll.”
Well, it’s not very hard to figure out what message the headline writers want us to take away from the story. The funny thing, though, is that if you actually read the Guardian’s article, there is no indication that respondents were even asked whether religion does more harm than good. The Guardian says:

More people in Britain think religion causes harm than believe it does good, according to a Guardian/ICM poll published today. It shows that an overwhelming majority see religion as a cause of division and tension – greatly outnumbering the smaller majority who also believe that it can be a force for good.

Got that? What they purport to be doing is comparing majorities responding to different questions. Further explanation:

It paints a picture of a sceptical nation with massive doubts about the effect religion has on society: 82% of those questioned say they see religion as a cause of division and tension between people. Only 16% disagree.

It seems pretty clear that the claim that most think that religion does more harm than good is based on the 82% who see religion as a cause of division and tension. But that’s absurd. Of course religion is a cause of division and tension. I’d answer that question Yes, too; even more so, if I lived in London and had been reading about the police searching for teams of suicide bombers. But what does that have to do with religion doing more harm than good? It isn’t even inconsistent with Christian (not to mention Muslim!) doctrine. (See, e.g., Matthew 10, 34-36.)
As far as one can tell from the Guardian article, no questions were asked about the good that religion does, let alone questions asking respondents to compare the good with the harm. The reference, quoted above, to a “smaller majority who also believe that it can be a force for good” is unexplained.
The actual poll results do not appear to be available on the Guardian site, nor are they posted on ICM’s site. So it’s possible that the relevant question was asked, but inexplicably–given the headline–omitted from the Guardian’s article. The much more likely explanation is that the headlines that have universally been given to the story misstate the poll results.
It is possible, given that 63% of respondents say they are “not religious,” that most Britons do think religion is mostly a bad thing. But you certainly can’t determine that from this poll, as it is reported by the Guardian. It appears that what is happening here is that the headline that every news source has attached to the article about the poll has been chosen to fit a journalistic agenda, not to accurately reflect the contents of the poll as described in the article itself.

Responses

Books to read from Power Line