The methodology used in the

The methodology used in the Star Tribune poll report summarized by Rocket Man below is described in Saturday’s Star Tribune as follows:
“This report is based on the most recent Star Tribune Minnesota Poll, a random-digit-dial (RDD) telephone survey of 1,048 adults statewide Oct. 11-16.
“Market Solutions Group, Inc., of Minneapolis conducted the interviewing from its central interviewing facility where interviewers were trained prior to calling, and supervised and monitored during the interviewing. MSG used a stratified, area-probability sample that the newspaper provided. It was probability-proportionate-to-size, and stratified by county. The Star Tribune’s polling unit provided the random-digit-dial sample of telephone numbers from the unit’s server-based telephone sampling database, which contains all working residential prefix-area code combinations in the state. (Consequently, all adults in the state who live in households with telephones were potential respondents; the sample was not limited to those with listed phone numbers, or newspaper subscribers, or other inappropriate populations.) Interviewers used the ‘most-recent-birthday’ technique to choose the adult from each household to be interviewed.
“The sample first was weighted to take into account unequal probability of selection from sampling: Weighting accounts for the number of telephone lines in a sampled household and the number of adults in the household, because many households have more than one adult or one phone line, and the poll only called one phone line in the household and interviewed only one adult (18 or older). It also is weighted on certain demographic characteristics, including gender, age and education, based on the 2000 census of the adult population.
“Weighting in such a manner allows one to assume the sample is representative of adults in all English-speaking Minnesota households, within the margin of sampling error. Finally, results were weighted to account for likelihood to vote.
“Researchers modeled the likely electorate for the general election using four questions: past voting history, current registration status, interest in the election, and self-professed probability of voting. Summing the responses to those questions produced an 8-point scale. Respondents were weighted according to their scale scores. Those most likely to vote (registered, voted in ’98, definitely will vote, high interest) were assigned larger weights; those least likely to vote (not registered, didn’t vote in ’98, won’t vote and low interest) received smaller weights. Assignments are based on formulas verified in past elections. This model suggests a turnout of 57.2 percent of the voting age population, about the same as the June and September estimates. In the last comparable election (U.S. Senate and governor in a non-presidential year, 1994) the turnout was 53.4 percent. However, in 1998, a gubernatorial-only election, 60.5 percent of the eligible adults turned out.
“The maximum margin of sampling error for percentages based on 1,048 is 3 percentage points, plus or minus, at a 95 percent confidence level, if one ignores the effect of sample design. Those tolerances for smaller groups, such as Democrats or Republicans, will be larger. Other things such as question wording, question order and some practical difficulties of interviewing may affect the results. These difficulties include a limited interviewing period, and the effect of news events and campaign activities on public opinion . Generally accepted social science research procedures were employed at every step of the research to reduce such problems.
“Another factor that could influence results is the number of people excluded from the originally drawn sample. The extent to which those persons who did not respond to the survey are different from those in the larger population may affect the results. The cooperation rate (COOP4, as defined by the American Association for Public Opinion Research) for this poll is 67 percent.”
End quote. This is me speaking again. In the 2000 presidential election, the final Star Tribune poll overpredicted Al Gore’s Minnesota vote by something like 5 points (and, I believe, Mark Dayton’s as well, though I may be mistaken on that score). I called the Strib’s pollster to ask about the polling methodology and say I thought it produced inaccurate results, overpredicting the Democratic vote in a way that was calculated to demoralize Republican voters. The Star Tribune’s pollster attributed the discrepancy to last minute shifts in sentiment that occurred after the polling was completed. But anyone who followed the presidential polls regardng the 2000 election knows that voter sentiment shifted throughout the country in the last few days before the election to Gore, not Bush. According to the Star Tribune pollster, Minnesota was an exception.
I don’t believe it. I think the polling methodology is a Rube Goldberg contraption that seems conveniently to favor Democratic candidates on a fairly regular basis. The Star Tribune’s lengthy explanation above avoids measuring poll results against actual election results. Why?
Forgive me for repeating the following observation, but I think it bears repetition. At the City Center shopping mall in downtown Minneapolis, I have heard the fire alarm go off while City Center security staff scramble to determine the cause. When the alarm is determined to be false and is turned off, the staff announces over the public address system that “the alarm has been verified as false.” I think that is the sense in which the Star Tribune’s polling methodology has been “verified.”

Responses

Books to read from Power Line