A More Perfect Poll

In the 2012 presidential election, we all thought we were smarter than the pollsters. Conservatives flocked to a site called UnskewedPolls.com, whose proprietor reconstituted the polls of major media organizations in proportions better suited to his vision of the American electorate—that is, one with more Republicans in it. Liberals, for their part, elevated to demigod status the statistician and New York Times blogger Nate Silver, who poured those same polls into a meat grinder and produced a neatly encased pronouncement that Barack Obama was overwhelmingly likely to win.

We didn’t just follow the polls, as we’ve always done; we questioned them, dissected them, tore them apart. Underlying all the partisan paranoia was a kernel of truth: political polling has been in crisis for years. But the real problem has nothing to do with a conspiracy to favor Democrats or Republicans. It’s that pollsters can’t get you on the phone. Telephone polling took over from door-to-door and mail-based surveys in the 1960s because it was reliable and cost-effective. Or rather, it was until Americans began ditching landlines: more than 30 percent of us now rely solely on cellphones, which are harder for pollsters to contact. An even greater difficulty stems from the fact that cellphone and landline users alike don’t answer the phone anymore. Pew’s response rate for its U.S. opinion research in 2012—the percentage of households in which someone agreed to be interviewed—was only 9 percent, down from 36 percent in 1997.

But a funny thing happened last fall, even as polling paranoia was raging: the polls got smarter, thanks in part to Internet-based polling, a method that had previously been seen as the industry’s redheaded stepchild. After the election, when Silver ranked 23 pollsters by how closely they approximated the presidential-election result, firms that had conducted their polls online took four of the top seven spots; in a separate ranking by a Fordham University professor, they took three of the top seven. Meanwhile, traditional, telephone-based survey groups like Gallup and the Associated Press scored near the bottom of both lists. That’s right: in 2012, polls that relied on people clicking on the equivalent of those “Your Opinion Counts!” pop-up ads proved a more effective gauge of the American electorate than the venerable Gallup Poll.

There’s reason to believe the Internet-based survey may be the future of political polling. If people are increasingly inaccessible by phone, they’re increasingly accessible online. Market research by big corporations, which have economic incentives to pursue fast, cheap, accurate data, has largely migrated to the Internet already. Darrell Bricker, the CEO of Ipsos Public Affairs, whose online polls for Reuters placed a respectable sixth out of 28 in the Fordham professor’s rankings, told me that about 70 percent of corporate opinion studies are now Internet-based. Political polling is another matter, however. Bricker says he’s heard online researchers mocked on more Washington, D.C., political panels than he can count. Political consultants and the media have been cautious, even hidebound, about changing their polling standards in recent years. “I’m all for experimentation,” Jon Cohen, the polling director for The Washington Post, told me. “But until it’s been justified methodologically the way random polling with telephones has been, I’m skeptical.”

Some of this skepticism is understandable. Whereas most phone surveys follow a well-established protocol, online polling’s sheer newness means practitioners are still refining techniques and working out methodological kinks. The chief issue is how to get people to participate. Most online-survey participants have opted in, sometimes in exchange for cash or other rewards. Some companies, like Ipsos, bring in additional respondents through social media and ads on Web sites (for example, a poll that needs more 18-to-34-year-old men might advertise on a gaming site). Nonetheless, the traditionalists charge that online polls don’t achieve a truly “probabilistic” sample, in which each member of the electorate has a theoretically equal chance of being contacted. Just about everyone who votes has a phone, but only 80 percent of American adults have Internet access, and those who don’t are predominantly older and lower-income.

For Ipsos, a French opinion-research conglomerate that takes in more than $1.5 billion annually, early success in online market research for clients like Coca-Cola opened minds to the Internet’s possibilities. The company began experimenting with online political polling in 2004, in Canada. The country proved to be an ideal laboratory, as a series of unstable minority governments led to four national elections in seven years. Conducting surveys online and by telephone simultaneously, Ipsos refined its techniques until, by the fourth election, its Internet surveys were consistently more reliable than its phone polling. The company later persuaded Reuters to go online for the 2012 election, a decision that was vindicated almost immediately, in Florida’s January 2012 Republican primary. Primaries are notoriously difficult to forecast, because they involve relatively small—and fickle—blocs of voters. But while other pollsters were off by an average of nearly 6 percentage points, Ipsos came within half a point of the Florida result.

When it came to the general election, online-poll results were quite a bit more accurate, on average, than their offline competitors. When Silver compared polls in the final weeks of the presidential campaign with the outcome, Internet polls had an average error of 2.1 points, while telephone polls by live interviewers had an average error of 3.5 points. Might the polling traditionalists finally come around? You might think so, and yet The New York Times—the very paper that hosts FiveThirtyEight, Silver’s blog—still refuses to cite Internet-based polls in its news reporting.

“In order to be worthy of publication in The Times, a survey must be representative,” the paper’s policy declares. Online polls are “therefore not reliable.” That policy was issued in 2006 and hasn’t been revised since. Maybe it’s time for The Times to get with the times.