2008.11.05
The pre-election polls: Who got it right?
Ralph Berrier, my friend and colleague, recently wrote a small item for the paper about his obsession with polls, his insensible desire to find any evidence that his candidate was gaining ground or holding steady.
This year, of all years, it was hard to know what to trust. The number of polling companies seems limitless -- Gallup, Marist, Zogby, Mason-Dixon, Survey USA, and so on -- as did the variety in the results.
Even for polls on the same day.
In the same spot on the map.
The results were all over the place. You could seemingly find evidence for any trend you hoped for -- a widening nationally, a tightening in battleground states, or whatever.
Pollsters start with certain assumptions about voter turnout among certain groups, and certain ideas about who is registered to vote and who will actually vote. There are registered voter models, likely voter models, and so on.
The trouble this year was, could anybody really know what the electorate look like? With 500,000 new registered voters in Virginia, for example, would a pollsters old assumptions still stand up?
Inevitably, you wonder, now that the only poll that matters has been conducted -- the election -- did anybody get it right? At least I wondered.
So, without putting a lot of work into it, here's a snapshot of polls versus results.
Let's start at the national level. Here are the last round of national polls of the presidential race from Real Clear Politics:
| Poll | Date | Sample | MoE | Obama (D) | McCain (R) | Spread |
|---|---|---|---|---|---|---|
| RCP Average | 10/29 - 11/03 | -- | -- | 52.1 | 44.5 | Obama +7.6 |
| Marist | 11/03 - 11/03 | 804 LV | 4.0 | 52 | 43 | Obama +9 |
| Battleground (Lake)* | 11/02 - 11/03 | 800 LV | 3.5 | 52 | 47 | Obama +5 |
| Battleground (Tarrance)* | 11/02 - 11/03 | 800 LV | 3.5 | 50 | 48 | Obama +2 |
| Rasmussen Reports | 11/01 - 11/03 | 3000 LV | 2.0 | 52 | 46 | Obama +6 |
| Reuters/C-SPAN/Zogby | 11/01 - 11/03 | 1201 LV | 2.9 | 54 | 43 | Obama +11 |
| IBD/TIPP | 11/01 - 11/03 | 981 LV | 3.2 | 52 | 44 | Obama +8 |
| FOX News | 11/01 - 11/02 | 971 LV | 3.0 | 50 | 43 | Obama +7 |
| NBC News/Wall St. Jrnl | 11/01 - 11/02 | 1011 LV | 3.1 | 51 | 43 | Obama +8 |
| Gallup | 10/31 - 11/02 | 2472 LV | 2.0 | 55 | 44 | Obama +11 |
| Diageo/Hotline | 10/31 - 11/02 | 887 LV | 3.3 | 50 | 45 | Obama +5 |
| CBS News | 10/31 - 11/02 | 714 LV | -- | 51 | 42 | Obama +9 |
| ABC News/Wash Post | 10/30 - 11/02 | 2470 LV | 2.5 | 53 | 44 | Obama +9 |
| Ipsos/McClatchy | 10/30 - 11/02 | 760 LV | 3.6 | 53 | 46 | Obama +7 |
| CNN/Opinion Research | 10/30 - 11/01 | 714 LV | 3.5 | 53 | 46 | Obama +7 |
| Pew Research | 10/29 - 11/01 | 2587 LV | 2.0 | 52 | 46 | Obama +6 |
Real Clear likes to average polls to give you a quick look. You can see the average of these polls was a 7.6 percentage point lead for Barack Obama. The actual was 6 percent (52 to 46) in the national popular vote.
Did any single poll get it right? Well, look at those last few. They're all over the place, but Rasmussen did in fact predict the exact final results. A few days earlier, a Pew Research poll did the same thing.
Good for them. Others were close -- Battleground was only off by a point. But what of the others? The Zogby poll that showed an 11 point gap. The other Battleground poll at 2 points. It seems to me that if you've got polls covering every possible numerical spread from a squeaker to a blow-out, somebody's bound to get it right, or at least get it close.
It's like playing roulette and putting a bet on every number. They all picked the right winner, at least. Beyond that, I'm not impressed.
Now, what about our state, Virginia.
| Poll | Date | Sample | MoE | Obama (D) | McCain (R) | Spread |
|---|---|---|---|---|---|---|
| Final Results | -- | -- | -- | 51.8 | 47.3 | Obama +4.5 |
| RCP Average | 10/29 - 11/03 | -- | -- | 50.2 | 45.8 | Obama +4.4 |
| FOX News/Rasmussen | 11/02 - 11/02 | 1000 LV | 3.0 | 51 | 47 | Obama +4 |
| Reuters/Zogby | 10/31 - 11/03 | 690 LV | 4.1 | 52 | 45 | Obama +7 |
| ARG | 10/31 - 11/03 | 600 LV | 4.0 | 51 | 47 | Obama +4 |
| SurveyUSA | 10/30 - 11/01 | 672 LV | 3.9 | 50 | 46 | Obama +4 |
| Mason-Dixon | 10/29 - 10/30 | 625 LV | 4.0 | 47 | 44 | Obama +3 |
This is more impressive to me. The average of these polls was 4.4 percent. The actual spread wound up at 4.5.
More impressive is the consistency across the board. Three polls were within half a point, and Mason-Dixon, which was a little low. In it's defense, it was also completed almost a week before election day. Things can change. Once again, Zogby is the outlier with the really high spread.
I'm not sure what to make of all this. I'm tempted to think that a more tightly focused poll that covers a single state is more likely to be accurate than a national poll in a country as big as ours, but this only looks at Virginia. In other words, too small a sample to be statistically valid.
I'd really like to see a more academic study of the polling this election, one that looks at not only who got it right, but which pollster started with the right assumptions about the electorate.
Did anybody really get it right in a year that seemed bust every mold for what we thought a presidential election could be.
Or did they just guess right?







Interesting recap. Thanks!
Comment by Beth Wellington — November 5, 2008 @ 6:20 pm
These polls have margins of error of 3.0 to 4.1 and the largest miss was only 2.5. So, all of them were accurate well within their margins of error. They all easily 'beat the spread'.
Let's remember that in four years when people start moaning and groaning about how far off the polls were back in '08
Comment by MHammond — November 5, 2008 @ 9:46 pm