Much hangs on how the nine US supreme court justices rule later this year on the two same-sex marriage cases they've heard this week, but if given the chance would Americans vote in favor of marriage equality?
Would they have done so last year, or, more importantly, today? I think the answer to all those questions is likely yes – but that differs from Nate Silver's more equivocal judgment and what Chris Stirewalt believes.
Let's start with the fact that the aggregate of all the same-sex marriage polling had support from about 50% of the population and opposition from 43% in 2012. Since then, we've seen about 1-2pt increase in the percentage favoring marriage. So, in order to believe that gay marriage wouldn't pass a nationwide referendum, you'd have to think that this polling data is unrepresentative of an election's results.
One reason you might believe that is because actual voters tend to be more conservative than adults at large. That's why Obama was winning handily among all adults, but was in a tight race with Romney among likely voters. Is this gap apparent with regard to same-sex marriage? As it turns out, not really.
On the eve of the 2012 election, Pew saw a minimal difference between how adults as a whole and likely voters viewed same-sex marriage. All adults were in favor of same-sex marriage, 49% to 40%, while likely voters were in favor by 49% to 42%.
The other argument against these national polls posits some sort of "gay Bradley effect". That is, polls for same-sex marriage state ballot measures have had a tendency to underestimate voters who wanted to ban same-sex marriage. Some, like Stirewalt, argue that a social desirability bias makes people not want to admit voting against perceived civil rights.
Either way, Patrick Egan found that the anti-gay marriage side (pdf) did 7pt better than polls suggested, while the pro-gay marriage side did about as well. California's Prop 8, for instance, trailed in the polls before passing.
When I ran my own data back in 2009, however, it was fairly clear that the "gay Bradley effect" was lessening over time. Polls taken closer to 2009 were more accurate than ones taken further back.
Since 2009, there have been five same-sex marriage ballots with polling conducted within 10 days of the election. The 10 days are key because ballot wording can be very confusing, and voters only really tune into campaigns in the final weeks. Gregory Lewis and Charles Gossett showed that confusing ballot wording is a likely part of the reason why polling on Prop 8 was inaccurate.
When we look at the ballot measures polls taken within 10 days of the elections, from 2009 onward, the pro same-sex marriage side didn't suffer anywhere close to the 7pt penalty that Egan had discovered. Maine's 2009 Question 1 had one poll taken within 10 days of the election. Public Policy Polling (pdf) (PPP) had voters "vetoing" same-sex marriage by a 4pt margin, and it was vetoed by a 6pt margin. That's only a difference of 2pt, not 7pt.
North Carolina's May 2012 Amendment 1 had two polls taken within 10 days of the election. An average of PPP's and SurveyUSA's polls, with undecideds allocated proportionally to decided voters, had North Carolinians banning gay marriage by a 19pt margin. They ended up banning it by a 22pt margin, a difference of 3pt.
Maine's November 2012 Question 1 had three polls taken within 10 days of the election. The median result of the polls from Critical Insights, Maine's People Resource Center and PPP had voters approving of same-sex marriage by 7.2pt. It passed by 5.4pt. A difference of a little less than 2pt.
Minnesota's Amendment 1 had three polls taken within 10 days of the election. The median result of two SurveyUSA polls and PPP's survey, which allocated undecideds, had the amendment to ban same-sex marriage failing by 1pt. The actual voters were against the amendment by 3.8pt. In this case, the polls overstated those favoring the ban by about 3pt.
Finally, Washington's Referendum 74 had two relevant polls. The average result of the PPP and SurveyUSA, with undecideds allocated, had same-sex marriage passing by 10pt. It passed by 7.4pt, for a difference of 2.6pt.
In four of the five instances, the opposition to the same-sex marriage side did better than polling predicted – but that difference was not particularly significant; certainly not of the order of 7pt, as previously found. The largest difference was just 3pt, and the median error in margin was only 2pt.
Applying this 2pt penalty to the 7pt edge that the pro-same-sex marriage crowd had over the opposition in 2012 national polling still gives the pro side a 5pt edge. If we apply an additional 2pt penalty (see Pew above), because most national polls question adults, and not likely voters, the pro same-sex marriage side still had a 3pt edge going into the 2012 election. Already, now, that lead would be closer to 4-5pt.
We can check my work by looking at the 2012 national exit polls. These polls are weighted to the actual results of the November elections. Voters cast their ballots anonymously, away from any interviewers, and drop their ballots in a box. There's no reason to think voters would fear giving an "undesirable" answer, as nobody's around to judge.
The exit poll asked: "Should your state legally recognize same-sex marriage?" "Yes" beat "no" by the exact 3pt that our adjustment of national surveys would suggest. The fact that the exit polls and national polls matched up makes me even more confident in the finding.
So, yes, I think Americans would have voted in favor of same-sex marriage, had they been given the opportunity in 2012-13. The national polls may be overstating support right now, but any error is not enough to erase the majority in favor of gay marriage.
In the end, though, I don't think there's much disagreement about the road going forward. Same-sex marriage support is increasing every day. A national same-sex marriage ballot measure would likely win by a huge margin in 2016. The tide has turned. It's tough to imagine it receding.