NYT political analyst David Leonhardt asks, “Are the Polls Wrong Again?“
The final polls in the 2020 presidential election overstated Joe Biden’s strength, especially in a handful of states.
The polls reported that Biden had a small lead in North Carolina, but he lost the state to Donald Trump. The polls also showed Biden running comfortably ahead in Wisconsin, yet he won it by less than a percentage point. In Ohio, the polls pointed to a tight race; instead, Trump won it easily.
In each of these states — and some others — pollsters failed to reach a representative sample of voters. One factor seems to be that Republican voters are more skeptical of mainstream institutions and are less willing to respond to a survey. If that’s true, polls will often understate Republican support, until pollsters figure out how to fix the problem. (I explained the problem in more depth in a 2020 article.)
This possibility offers reason to wonder whether Democrats are really doing as well in the midterm elections as the conventional wisdom holds. Recent polls suggest that Democrats are favored to keep control of the Senate narrowly, while losing control of the House, also narrowly.
But the Democrats’ strength in the Senate campaign depends partly on their strength in some of the same states where polls exaggerated Democratic support two years ago, including the three that I mentioned above: North Carolina, Ohio and Wisconsin.
Emphasis mine.
He provides this handy dandy chart showing the deltas:
He points us to a column by his colleague Nate Cohn, “Yes, the Polling Warning Signs Are Flashing Again.”
Ahead of the last presidential election, we created a website tracking the latest polls — internally, we called it a “polling diary.” Despite a tough polling cycle, one feature proved to be particularly helpful: a table showing what would happen if the 2020 polls were as “wrong” as they were in 2016, when pollsters systematically underestimated Donald J. Trump’s strength against Hillary Clinton.
Very much like the table Leonhardt used above.
We created this poll error table for a reason: Early in the 2020 cycle, we noticed that Joe Biden seemed to be outperforming Mrs. Clinton in the same places where the polls overestimated her four years earlier. That pattern didn’t necessarily mean the polls would be wrong — it could have just reflected Mr. Biden’s promised strength among white working-class voters, for instance — but it was a warning sign.
That warning sign is flashing again: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.
After diving into one particular instance, he observes,
If the polls are wrong yet again, it will not be hard to explain. Most pollsters haven’t made significant methodological changes since the last election. The major polling community post-mortem declared that it was “impossible” to definitively ascertain what went wrong in the 2020 election.
The pattern of Democratic strength isn’t the only sign that the polls might still be off in similar ways. Since the Supreme Court’s Dobbs decision on abortion, some pollsters have said they’re seeing the familiar signs of nonresponse bias — when people who don’t respond to a poll are meaningfully different from those who participate — creeping back into their surveys.
Brian Stryker, a partner at Impact Research (Mr. Biden is a client), told me that his polling firm was getting “a ton of Democratic responses” in recent surveys, especially in “the familiar places” where the polls have erred in recent cycles.
None of this means the polls are destined to be as wrong as they were in 2020. Some of the polling challenges in 2020 might have since subsided, such as the greater likelihood that liberals were at home (and thus more likely to take polls) during the pandemic. And historically, it has been hard to anticipate polling error simply by looking at the error from the previous cycle. For example, the polls in 2018 weren’t so bad.
Some pollsters are making efforts to deal with the challenge. Mr. Stryker said his firm was “restricting the number of Democratic primary voters, early voters and other super-engaged Democrats” in their surveys. The New York Times/Siena College polls take similar steps.
But the pattern is worth taking seriously after what happened two years ago.
So, look, this has long been the challenge in polling. Back when telephone polling was king, the difference between the numbers shown by various polling outfits was their “likely voter screen”—the model they applied to weight the responses. If your screen is off, then your poll is off.
It’s gotten harder in recent years because it’s much easier to screen calls. And because online polling is simply a different animal. In both cases, we now have the additional problem that we’ve violated the first rule of polling: our samples are now self-selected, not random.
Anywho, Cohn then presents the same chart Leonhardt used above and observes,
The apparent Democratic edge in Senate races in Wisconsin, North Carolina and Ohio would evaporate. To take the chamber, Republicans would need any two of Georgia, Arizona, Nevada or Pennsylvania. With Democrats today well ahead in Pennsylvania and Arizona, the fight for control of the chamber would come down to very close races in Nevada and Georgia.
Regardless of who was favored, the race for Senate control would be extremely competitive. Republican control of the House would seem to be a foregone conclusion.
Leonhardt offers some hope for Democrats:
Nate is also careful to acknowledge what he doesn’t know, and he emphasizes that the polls may not be wrong this year in the same way that they were wrong in 2020. It’s even possible that pollsters are understating Democratic support this year by searching too hard for Republican voters in an effort to avoid repeating recent mistakes.
The unavoidable reality is that polling is both an art and a science, requiring hard judgments about which kinds of people are more or less likely to respond to a survey and more or less likely to vote in the fall. There are still some big mysteries about the polls’ recent tendency to underestimate Republican support.
The pattern has not been uniform across the country, for instance. In some states — such as Georgia, Nevada and Pennsylvania — the final polls have been pretty accurate lately. This inconsistency makes the problem harder to fix because pollsters can’t simply boost the Republican share everywhere.
There is also some uncertainty about whether the problem is as big when Trump is not on the ballot — and he is obviously not running for office this year. Douglas Rivers, the chief scientist of the polling firm YouGov, told me that he thought this was the case and that there is something particular about Trump that complicates polling. Similarly, Nate noted that the polls in the 2018 midterms were fairly accurate.
Finally, as Nate points out, the 2022 campaign does have two dynamics that may make it different from a normal midterm and that may help Democrats. The Supreme Court, dominated by Republican appointees, issued an unpopular decision on abortion, and Trump, unlike most defeated presidents, continues to receive a large amount of attention.
As a result, this year’s election may feel less like a referendum on the current president and more like a choice between two parties. Biden, for his part, is making this point explicitly. “Every election’s a choice,” he said recently. “My dad used to say, ‘Don’t compare me to the Almighty, Joey. Compare me to the alternative.’”
The bottom line here is that we really don’t know a hell of a lot about how the vote less than two months hence is going to go.
The obvious solution, then, is to quit spending a lot of money on polling for the purpose of generating “news” that is then analyzed and caveated to death. But, of course, that’s not going to happen because political junkies enjoy consuming these stories.
It’s much like the argument that sportswriters should quit generating, for example, preseason rankings of college football teams given how much teams change from year to year. Why, just play the games and we’ll know who’s good and who isn’t! But games are (mostly) played on Saturdays and only from September to January. What else are they going to use to generate content if not speculation?
