Nate Silver continues his series extolling the virtues of polling averages as predictors of outcomes with some really impressive data:
I have a database containing almost all polls conducted in all U.S. Senate and gubernatorial races since 1998. I say almost all because it excludes internal polls released by campaigns and other explicitly partisan groups, and it excludes Internet polls conducted by Zogby Interactive, which in my view are not scientific. My database also does not include polls for irregularly-scheduled special elections, like the one in Massachusetts earlier this year — only contests in November.
[…]
Let’s construct about the simplest possible study around this:
Step 1. Take all polls conducted between 30 and 60 days from the election.
Step 2. Average them together.
That’s it. We’re not doing any of the fancy stuff that we do in our actual Senate model, like weighting the polls based on sample size or the quality of the pollster. We’re just taking a simple average.
There is one “trick”, though: we’re only looking at races in which at least two different polling firms published a survey in the 30-to-60 day window. If you have just one company polling a race, you don’t really have much of an average, properly speaking.
Doing this simple test, he derives the following table:
This is truly stark: ”[N]o Senate candidate with a lead of more than 5.5 points in the polling average, with 30 days to go in the race, has lost his race since 1998: these candidates are 68-0.” The accuracy is slightly less in gubernatorial contents, as two candidate with 6-9 point margins and one with a 9-12 point margin has lost. Still, the overall record for gubernatorial candidates with 6 or point leads in the polls has been an amazing 124 for 127, or 97.3%.
Even granting that these numbers are grossly inflated by the fact that many contests are laughers, with polling margins of 15 points or more, Silver’s point is strong: Simply averaging reputable pre-election polls gives a tremendous snapshot of who will go on to win.
This is especially impressive when one factors in two things. First, a lot of these polls are simply of “registered voters” or even “adults.” Polls that apply a “likely voter” screen are much more accurate. Second, a lot can theoretically happen in the last 30 days of a campaign. Scandals, gaffes, debates, advertising, and so forth are magnified late in the cycle. And, yet, candidates in statewide races win almost every race when they lead by 6 points or more a month out.
