David Leonhardt notes that, while the polling aggregates got 48 of 50 states right in the Presidential race, they were sufficiently off in other contests that we should rightly question their utility.
Indeed, even with the Presidential contest, the polls were wildly off by degree if not direction. Here’s his compilation based on the NYT aggregates:
While many of these are nearly spot-on, some—notably Wisconsin, Ohio, and Iowa—were massively wrong. And all in one direction. That’s a systematic error.
And, yes, it matters. The press are misleading the public with false expectations:
In response, polling firms are asking whether they need to accelerate their shift to new research methods, such as surveying people by text message. And media organizations including The New York Times, which financially support and promote polls, are re-evaluating how they portray polls in future coverage. Some editors believe the best approach may be to give them less prominent coverage, despite intense interest from readers and despite the dominant role polls play in shaping campaign strategies.
And, because the same errors impacted internal polls conducted by the campaigns and parties, it impacted strategy:
This year’s misleading polls had real-world effects, for both political parties. The Trump campaign pulled back from campaigning in Michigan and Wisconsin, reducing visits and advertising, and lost both only narrowly. In Arizona, a Republican strategist who worked on Senator Martha McSally’s re-election campaign said that public polling showing her far behind “probably cost us $4 or $5 million” in donations. Ms. McSally lost to Mark Kelly by less than three percentage points.
Mr. Biden spent valuable time visiting Iowa and Ohio in the campaign’s final days, only to lose both soundly. Democrats also poured money into races that may never have been winnable, like the South Carolina Senate race, while paying less attention to some of their House incumbents who party leaders wrongly thought were safe. The party ended up losing seats.
“District-level polling has rarely led us — or the parties and groups investing in House races — so astray,” David Wasserman of the Cook Political Report, a nonpartisan publication that analyzes races, wrote last week.
Of course, at least in terms of the Presidential race, this is exacerbated by the anachronism of the Electoral College. The polls led us to believe for months that Biden would win and he did. But the fact that, once again, the state-by-state allocation could have handed a major popular vote loser the election by shifting a handful of votes in three states, magnified the polling errors.
So, what went wrong? Well, lots of things.
- People’s decreasing willingness to respond to polls — thanks partly to caller ID — has reduced average polling response to only 6 percent in recent years, according to the Pew Research Center, from above 50 percent in many polls during the 1980s. At today’s level, pollsters cannot easily construct a sample of respondents who resemble the population.
- Some types of voters seem less willing to respond to polls than others, perhaps because they are less trusting of institutions, and these voters seem to lean Republican.
- The polling industry tried to fix this problem after 2016, by ensuring that polling samples included enough white working-class voters in 2020. But that is not enough if response rates also vary within groups — for instance, if the white or Hispanic working-class voters who respond to polls have a different political profile than those who do not respond.
- This year’s polls may have suffered from pandemic-related problems that will not repeat in the future, including a potential turnout decline among Democratic voters who feared contracting the coronavirus at a polling place.
- A much-hyped theory that Trump supporters lie to pollsters appears to be wrong or insignificant. Polls did not underestimate his support more in liberal areas, where supporting Mr. Trump can be less socially acceptable, than in conservative areas.
- In what may be the most complex pattern, polls underestimated the support of multiple Senate Republican candidates even more than Mr. Trump. This means the polls missed a disproportionate number of Americans who voted for both Mr. Biden and a Republican Senate candidate — and that the problems do not simply involve Mr. Trump’s base.
In the 2016 cycle, I still had a landline phone because it was included in my Internet bundle and I had small children and liked the assurance of 9-1-1 service. Because Virginia was still considered a swing state then, I got so many calls from pollsters a day that I finally just turned off the ringer.
Now that I’m mobile-only, I answer zero calls from people not in my contacts list. Indeed, once iPhone allowed auto-rejection of such calls, they all go straight to voicemail. I would expect that this is the norm for people younger than me.
This means we now have a no-no: the self-selected sample. Rather than a randomized group of people weighted to assured demographic representativeness, we have people who either can’t figure out how to avoid unwanted calls or just really, really want to share their opinions with pollsters. They’re unlikely to match the attitudes of the population as a whole.
Interestingly, I would have expected this to lean in the other direction: older, less technologically savvy people who would be more pro-Trump. But, of course, this is offset by the “shy Trump voter” phenomenon.
While I had long understood that to be some sort of conspiracy by Trump supporters to fool the pollsters and “prank the libs,” it’s actually something more basic: in certain swaths of the country, people fear being socially ostracized if they admit to voting for Trump.
