Outside the Beltway

The Elusive ‘Likely Voter’

Where the science and art of polling intersect.

James Joyner · Tuesday, September 24, 2024 · 10 comments

Adding to our long-running conversation of how public opinion researchers have adapted to changing realities (see “Is Political Polling Still Useful?” and “Political Polling in the Trump Era” for recent examples), the NYT weighs in with “Accurate Polls Hinge on a Tricky Question: Who’s Actually Going to Vote?“

In the flood of election polls you’ll see over the next few weeks, most polling groups will include responses from “likely voters.” And often from nobody else.

In theory, these poll numbers should yield more accurate results, since the people who actually vote are the ones who dictate the outcome on Election Day. But creating a precise picture of who will vote in November is a complicated endeavor.

After all, how exactly can a pollster know who is “likely” to vote, and who therefore will be the focus of their results? There’s no one right answer, and every polling firm has its own strategy.

After a long discursion into the obvious, they turn back to said strategies.

There are a few different ways to identify likely voters, and one is simply to ask poll respondents whether they plan to vote. But that question by itself has limitations; people are not very good at predicting their future behavior when asked about it in surveys. And because many people feel as though they ought to vote, they will often say they plan on doing so, even if they don’t.

Brian Schaffner, a political science professor at Tufts University, is a co-director of the Cooperative Election Study, an annual survey from Harvard and YouGov. When Professor Schaffner and his team compared poll responses from 2020 to voter records for that year, they found that 27 percent of those who had said they would “definitely” vote had not been recorded voting in those elections.

Polling groups have developed a few different ways of dealing with this problem. Some, like Ipsos, which often conducts polls with media partners like ABC News, The Washington Post and Reuters, use a formula that analyzes responses to a series of questions that go beyond a simple “Will you vote?”

Employing demographic information, asking respondents if they have voted before and inquiring as to the location of their polling place can help create a profile of a likely voter. (Younger voters tend to be less likely to vote, for example; people who know the location of their polling place are more likely.)

Amusingly, I’m not sure I know my polling place’s location because I’ve voted at three different places in the five years I’ve lived here. But I know how to Google.

Another approach is to examine voter file data derived from government records, which can tell pollsters if a respondent has actually voted in the past (though not whom they voted for). Since people who have voted before are likelier to vote again, this can help improve pollsters’ accuracy in identifying those who will cast ballots.

But this approach relies heavily on the accuracy of voter files. “The voter file itself essentially starts to immediately decay because people move, people die, new people age into the electorate who weren’t in it,” said Chris Jackson, a senior vice president at Ipsos Public Affairs. Some firms blend the two methods, both asking voters questions and examining their voter files.

Granting that computerization has made this sort of thing easier than ever, it’s certainly labor-intensive. And a bit concerning. Obviously, voter registration and turnout records are public information. But pollsters tying responses to particular individuals undermines the secret ballot. Who knows what they’re doing with that information. And, even if they’re not sharing that information, they’re subject to hacking.

Once a subset of likely voters has been identified, what is to be done with it? Some pollsters use a “cutoff” model, simply removing voters they judge least likely to vote from the total sample.

Other polls, including The Times/Siena Poll, use what’s known as a probabilistic model. Rather than eliminating low-likelihood voters from the sample entirely, pollsters combine the available data to estimate how likely each respondent is to vote, and their responses are weighted accordingly.

“We feel that, because we have the information from the voter file, we should be using it in our data to contribute what it can,” said Jennifer Agiesta, the polling director at CNN, which this year switched from a cutoff model to a probabilistic model for the first time.

This, ultimately, is guesswork. It’s all we have, of course, but largely useless in tight elections. And, in elections that aren’t tight, we don’t really need polling.

President Biden, while he was running for re-election, tended to perform slightly better in national polls among likely voters than among registered voters. But in the tight race between Vice President Kamala Harris and former President Donald J. Trump, pollsters are somewhat split on which candidate benefits among likely voters.

In a Times/Siena national poll published Thursday, Ms. Harris and Mr. Trump were even among likely voters; among registered voters, Mr. Trump led by one percentage point.

In recent state-level Times/Siena polls, it was a mixed bag. In some states, such as North Carolina, Mr. Trump did better among likely voters than among registered voters, while in others, such as Michigan, it was Ms. Harris who had the advantage among this key group.

This reverses longstanding trends but isn’t surprising, as there’s been a modest partisan re-alignment over the past decade or so, with college-educated whites shifting to Democrats and blue-collar whites shifting toward the MAGA Republicans.

What do these mixed results tell us about November? In part, they tell the same story that all polling has been telling lately: This is an exceptionally close race. When considering a specific state, some of the differences can be attributed to a candidate’s relative strength among segments of the state’s population that are more or less likely to turn out on Election Day.

For example, in the Michigan poll from August, registered voters under the age of 45 favored Mr. Trump overall. But the people in that age group who are most likely to cast a ballot, according to historical turnout records, are overwhelmingly college educated, politically engaged liberals. So the likely voter model will, by definition, discount many of the younger Trump-leaning voters.

Again, that college-educated folks are more likely to vote is not new—but a greater percentage of them are now voting Democrat.

In the weeks to come, not all polls will report both registered voter and likely voter results, but when they do, these numbers can offer context in terms of how pollsters are viewing the electorate.

Just how accurate they will be will depend on who shows up to vote. That’s still one of the biggest unknowns for pollsters, Professor Schaffner said.

“The reason you should care is because it does reflect that turnout matters,” he said.

This is all a long way of saying—or not saying—“Who the fuck knows?”