Skip to main contentSkip to navigationSkip to navigation
Barack Obama and Mitt Romney
Barack Obama's current lead in the polls leaves Romney only a slender hope of winning in November. Photograph: Mary Altaffer, Evan Vucci/AP
Barack Obama's current lead in the polls leaves Romney only a slender hope of winning in November. Photograph: Mary Altaffer, Evan Vucci/AP

'Unskewing' polls with party ID is, statistically speaking, bunk

This article is more than 11 years old
The idea that current polling is 'too Democratic' and needs re-weighting for party identification simply doesn't pass muster

"The mainstream media is skewing its polls with too many Democrats because they want Obama to win."

Statements like this one are zipping around the internet a mile a minute these days. The general idea is that there are too many respondents identifying as Democrats in public surveys compared to past elections' exit polls. There's even a website titled unskewedpolls.com that purports to adjust media polls to correct for these errors, based on Rasmussen Reports data.

I have spent plenty of words already debunking the use of party identification to dismiss a poll's results. Party ID is a fluid variable that can change for certain voters depending on their voting choice. Very few pollsters weight for party ID. A rare example might be if everything about a poll stayed steady from the previous ones, except for a sudden and drastic change in party identification for no apparent reason. Only under this scenario will some pollsters adjust their data.

Recently, the "too many Democrats" chorus has reached a fever pitch. Many other polling pundits have laughed off these complaints. One went so far as to compare the proponents to 9/11 conspiracy theorists.

Indeed, it is a ridiculous charge that the media purposely skews polls. If so, they have persuaded many Republicans to get in on the act. Polls from Fox News – an organization few would accuse of a pro-Democratic bias – have shown poor results for Romney. (Its polling is also partially run by the openly Republican Daron Shaw.) The NBC/Wall Street Journal national poll is part run by Bill McInturff: a Republican, John McCain's own pollster, and professional partner to Mitt Romney's pollster. The GWU/Battleground poll has Republican pollster Ed Goeas on board. The list goes on and on.

However, just because the polls aren't purposely being manipulated doesn't mean the polls are infallibly right. One estimate has the chance of Romney winning, if the election were held today, at about 2%. That's very small, but it's not 9/11 conspiracy territory, not even close. Put another way, the chance of the Mitt Romney winning, if the election were right now, is slightly better than Vegas odds of the Buffalo Bills winning the Super Bowl, and the Bills have a pretty good team.

It's even less crazy to think that the current polling does not reflect the reality of what may happen in November. Estimates of Romney winning in November are higher, at somewhere between about 10% and 20%. Again, these are low numbers, but they are not negligible.

There are also reasons to believe that Rasmussen Reports will accurately forecast the party identification in the exit polls. Yes, Rasmussen remains a maverick figure in much of the mainstream media, but take a look at how its partisan make-up in its three-month average before the election matched up against the exit polling since 2004.

Exit polls v Rasmussen

Rasmussen actually comes fairly close to the result in every year. It has actually tended to overestimate the edge Democrats hold over Republicans in the party identification measured by the exit polls. It's rather amazing that even in a year where Rasmussen's final poll for the national House vote was 5 percentage points "too Democratic", its party identification was actually 2 percentage points "too Democratic".

How is it possible that Rasmussen produced good data on party identification compared to the exit polls, but wasn't all that accurate? As I have pointed out, party identification just isn't very predictive of what a poll will find. It's entirely possible that Rasmussen gets the party spread correct, but incorrectly predicts outcome. Indeed, there is pretty much no relationship between the spread across the candidates and party ID.

Obama lead v party ID

That's why Scott Rasmussen himself has said that you can't apply one firm's part identification to another firm's results and expect anything accurate to come of it. But let's say that, despite all these warnings, we still wanted to weight polls by party identification.

It should noted that Pew Research also has a solid record with regard to party identification and the exit polls.

Exit polls v Pew

Pew also tends to have very consistent party identification throughout the year. What Pew is showing now – a high single-digit lead for Democrats on party identification – is likely what they will find in their final poll.

We therefore have two pollsters with good track records showing very different party identification make-ups for the 2012 electorate. Should we choose Pew or Rasmussen?

Pew does everything a pollster should do. They call cellphones and have very long survey periods to ensure an accurate representation of all parts of the American electorate, even those who might be difficult to reach. Pew was also correct in the 2010 midterm elections, while Rasmussen was off.

So, even if I were to weight by party ID, why would anyone choose Rasmussen over Pew? A statistical answer that doesn't approach conspiracy theory would be appreciated.

What I'm trying to say is that there are reasons to believe Mitt Romney can still win this election. They don't start with using party ID to "unskew" the polling.

Most viewed

Most viewed