Why the Polls Are So Often Wrong

The big story today, as over the weekend, is the resurgence of Republicans in a number of recent polls. That resurgence has been picked up broadly enough that it is fair to assume it is real. Still, though, we are seeing it through the medium of the polls. There is a more basic issue that clouds the polls’ ability to predict what will happen tomorrow, and the ABC/Washington Post poll, one of those showing a Republican comeback, exemplifies the reason why.
The most difficult challenge for every pollster is figuring out which poll respondents will actually vote, i.e., are “likely voters.” Pollsters try to get at this by first asking respondents whether they are registered voters. If the respondent claims to be registered, the pollster asks whether he intends to vote in the coming election, and whether he has voted in the past.
In this particular ABC/Washington Post poll, 80% claimed to be registered voters. (Question number 905.) But this undoubtedly exaggerated the number of registered voters in the pool, since as of 2002, only 66% of eligible voters were registered.
Next, take a look at question number three in the ABC/Post poll: no fewer than 70% of those who answered the telephone, and claimed to be registered voters, said they are “absolutely certain” to vote, while another 11% said they “probably” will vote and 5% said they already had voted. Those numbers add up to 86% of registered voters, and, if 80% of respondents really were registered, 69% of eligible voters.
That’s not all: the 70% who said they were absolutely certain to vote were asked whether they always, usually or sometimes vote in off-year elections. The result? 95% said they either “always” (71%) or “nearly always” (17%) or “usually” vote in midterm elections.
Pity the poor pollster. An overwhelming majority of his respondents tell him they surely will vote tomorrow, and, indeed, always do. But the pollster knows that over the past twenty years, the percentage of eligible voters who actually voted in a midterm election has never topped forty percent, while the turnout among registered voters has never exceeded 72%.
Which means that in this particular poll, probably around a quarter to a third of the respondents who assured the pollster that they will vote, won’t. (Here’s where I need a mathematician like Dafydd ab Hugh to check my calculation.)
So how can the pollster know which of his respondents will in fact show up at the polls tomorrow? He can’t know, actually. But one thing we do know from experience is that Republicans tend to turn out and vote more reliably than Democrats, no matter how much enthusiasm for the process Democrats express to pollsters. Based on experience, what a pollster really should do is weight his poll results in favor of Republicans. But I doubt whether any of them do that.
In short, we really don’t know what will happen tomorrow. The election will be decided by those who show up, and somewhere up to a third of those who claim they are going to vote–and always do!–won’t.

Notice: All comments are subject to moderation. Our comments are intended to be a forum for civil discourse bearing on the subject under discussion. Commenters who stray beyond the bounds of civility or employ what we deem gratuitous vulgarity in a comment — including, but not limited to, “s***,” “f***,” “a*******,” or one of their many variants — will be banned without further notice in the sole discretion of the site moderator.

Responses