Fatally flawed poll on voters

Independent should never be confused with accurate. A poll conducted by North Dakota’s Forum of Fargo-Moorhead is likely independent, as its purveyors claim, but fatally, indeed laughably, flawed. 

First rule of polling — sample the right people. If you are trying to gauge sentiment in California and exclude everyone north of Los Angeles, the poll won’t be an accurate guide to the state. Survey Kansas and you won’t know much about Iowa. Poll primary voters only and you can’t say anything meaningful about the general election. 

{mosads}Yet the last is just what the Forum did. The paper and its pollster, Essman/Research, were honest enough to disclose this fact but apparently ignorant of its implications. After polling voters judged “Likely to vote in the June 12 primary election,” the Forum not only tried to assess support for two measures on the primary ballot but went far beyond the data, drawing conclusions about “the candidates that voters will likely support for … House and … Senate” in November’s general election.

The Forum’s ballot measure findings might be informative, but because it systematically excluded those who will vote in the general election, but not in the primary, its results for the Senate and House races are, simply, worthless. You can’t measure “sentiment” in 

the general election from a poll of primary voters. It’s an error any undergraduate should pick up, but seemingly neither the Forum nor its pollster understood the extraordinary bias in the methodology. 

(I acknowledge an interest here — Heidi Heitkamp, the Democratic candidate for Senate, is our client, but as I hope readers know, I owe an even higher loyalty to accuracy.)

Forum editor Matt Von Pinnon defended his paper, asserting primary turnout will likely be “similar … to a general election … So is it not reasonable to think their answers might mirror those of the general electorate?”

No. It is not only unreasonable, it’s foolish. Start with the premise — that turnout in the primary will be similar to the general election. Just how likely is this? About as likely as it is for Von Pinnon to praise this column in his.

Historical records are readily available. In 2008, 102,886 North Dakotans cast primary ballots, but more than three times as many voted in the general election. In 2004, almost three and a half times as many voted in the general as in the primary. In fact, not once in the last four decades has primary turnout approached that in the general. 

Is it reasonable to suspect that the roughly 200,000 people who vote in generals, but not in primaries, are different from the 100,000 (plus or minus) who vote in primaries? Of course. The very fact that they bother to vote in primaries makes them different, as does their willingness to participate in party contests.

This poll might be burdened with yet another flaw. I say “might” because there is uncertainty. Not a single voter was undecided on either of the two ballot measures. Having done initiative polling for decades, I’ve never found one on which every voter had made up his or her mind. The paper says it “know[s] of nobody who said they were undecided.” However, a conservative blog reported hearing from one interviewee who was terminated after saying they were undecided. 

 It seems crazy, but it’s possible, despite its claim, that the Forum poll excluded every respondent who was undecided on the initiatives, as if indecision on a June ballot measure in May renders you ineligible to vote in November.

Could this poll be accurate? Only in the way a broken watch is right twice a day.

The Forum and Essman/Research have done a grave disservice to survey research, to journalism and to North Dakota. An ethically responsible newspaper would admit error and retract the story. An ethically responsible research company would not allow its work to be misused in this fashion without publicly disavowing the conclusions.

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leader of the Senate and the Democratic whip in the House.