Sam Wang looks at the Senate races from the last two election cycles that wound up being decided by ten percentage points or less. The red numbers (in the Final column) indicate that the Republican won or that they had a polling lead in the aggregate of the polling over the final three weeks of the campaign (in the Polls column). The blue numbers indicate the same for the Democrats. The Bonus column shows how much better the candidate did than the polls had indicated that they would do.
In 2010, the polls in seven out of eight close Senate races underestimated the actual performance of the Democratic candidate, causing the pollsters to incorrectly predict the winner in Nevada (Harry Reid) and Colorado (Michael Bennet). In 2012, the pollsters made all the correct predictions, but they still underestimated the Democrats’ performance in nine out of eleven races.
Prof. Wang also demonstrates that the polling bias for Senate elections in midterm elections has been five times as large as in presidential elections going back to 1990. Despite the numbers above, the polling bias in that period has not favored one party over another, but the uncertainty level in midterms has been much higher.
It’s easy to see that the pollsters would have had a very bad night in 2012 if their polling bias had been reversed and favored the Democrats. If their polling bias had been 3.0 in favor of the Democrats, they might have whiffed on the elections in North Dakota, Montana, Wisconsin, Indiana, and Virginia.
With the polls so close in so many elections in this cycle, a similar pro-Republican bias could result in exactly that kind of colossal failure on the pollsters’ part.
I don’t want to go down as the Dick Morris of the 2014 elections, so I want to be clear that I am not confidently predicting that the polls have a strong pro-Republican bias and that the Democrats are going to win all the close elections. I am only pointing out that the historical record shows that it is normal for the pollsters to have a large bias in midterm elections, so it would be surprising if that were not the case this year. The challenge is in predicting which direction they are biased this time around. I have already asserted that I think it is much more likely that the polls are either accurate or that they are too optimistic for the Republicans than it is that they are too optimistic for the Democrats. If they’re accurate, it will still be a bad night for the Democrats and they will lose control of the Senate. But there is another, better, possibility out there that doesn’t involve “skewed polling” theories based on nothing more than wishful thinking.
The reason that midterm polls are less accurate is because it is harder in midterms to identify who will and who will not vote. So, go get someone who wasn’t going to vote to go to the polls and you’ll be doing your part to prove the pollsters wrong.
Okay, so the pollers got it wrong in 2010. Did the aggregator/predictors get it wrong, too? Was Sam Wang even around in 2010?
Yes, and Wang had Harry Reid losing.
Reid used his union thugs to beat out Angle. Hard to capture that in a poll.
Maybe the poll aggregators should work on the “secret sauce.”
Well, the more we can demonstrate that polls are not accurate, perhaps the more people will actually vote by not believing in the inevitability of polls. Could be good.
The results from 2010 and 2012 indicate that Democrats are showing up and voting just fine in Senate contests.
Drilling down. There were only two races where the D had a large lead in the polls: WA by 10% and NM by 7.5%. Were the polls wrong or did it give Democrats that found it too difficult or inconvenient to vote permission not to vote?
Was something similar in operation for Republicans in WI and PA?
Why no instance of a GOP leading in the polls performing as well or better on election day? Why would Democrats bother showing up for several contests that were lost causes?
The NV 2012 Senate race in interesting. No Obama coattails. And Berkley didn’t need all that much to win. (Heller underperformed Romney but only by a much smaller amount.) Why no prominent Latino politicians in NV?
If history is any guide, the 2014 polls in the very tight races are likely too optimistic for Republicans. Possibly enough so that a few prognosticators are going to wake up with egg on their faces when the actual results get posted.
The reason midterm polls are less accurate is because it is harder to identify (especially for the House) who will and will not vote.
What is the variance over the last twenty years’ midterm elections on missing the spread? Does it correlate with rules of thumb about midterm elections? That is, are pollsters inadvertent building in special sauce in their screens of samples that are based on rules of thumb that bias the results? In what ways do the errors appear not to be random runs?