Given the fact that President Obama won North Carolina by less than half a percentage point in 2008, it’s not entirely surprising that the latest Public Policy Polling poll is showing Mitt Romney pulling ahead of the President in a state that has been traditionally Republican in Presidential elections going back all the way to 1980. What is surprising is something that Business Insider found while delving into the cross tabs:
20% of the African-American vote for Romney? That’s actually quite a shock considering that Obama won 95% of the African-American vote in 2008. The number is consistent with Obama’s approval rating in the state, where the polls shows that some 21% of African-American respondents had a negative opinion of the job the President was doing. That’s quite different from nationwide polls where the President’s job approval rating among African-Americans has been consistently near 90%. Moreover, the most recent non-partisan poll of the state had Obama winning African-Americans 84% to 8%, and a poll by Republican pollster Civitas had Obama winning African-Americans 89% to 2%. More importantly, an April PPP Poll [PDF] had Obama beating Romney 90% to 7% among African-Americans with an Approval/Disapproval rate of 91%/7%. The crosstabs from PPP’s May poll in North Carolina are unavailable for some reason but I would be their numbers are about the same.
PPP concedes that this is “an unrealistically low share” of the African-American vote for Obama.
So what’s going on here? Is this the first sign that the President’s shift on same-sex marriage has really hurt him among religious African-Americans? Well, that’s undoubtedly how some on the right will spin it, but there are several caveats here. African-Americans constituted some 22% of total poll respondents. While this is roughly consistent with the state’s demographics, it means that, of the 810 poll respondents, 178 of them were African-American (or at least identified themselves as such). This is a pretty small sample size and the margin for error for this individual demographic would be fairly high, at least in the 6-11% range one would guess. It’s also worth noting that PPP is an autodial poll that doesn’t use human operators so it relies on people responding via their telephone keypads, and doing so truthfully. Its possible that we’re seeing false responses here, or bad sampling. At the very least, unless we see something like this replicated in other polls of the Tar Heel State, there’s not really any reason to believe that it’s true, or that the President is really in trouble among members of his most loyal demographic group.
Update: Nate Silver weighs in on this poll and, while he does agree that losing in North Carolina is bad news for the President, he’s as skeptical as I am about the numbers for African-Americans:
The poll surveyed 810 North Carolina voters, and weighted its demographics such that 22 percent were African-American. If you do the math, that suggests that about 180 respondents in the survey were black.
The margin of error on a 180-person subsample is quite high: about 7.3 percent.
And that probably underestimates the real margin of error somewhat. The 22 percent African-American figure in the poll represents the results after demographic weighting. However, some demographic groups are considerably harder to get on the phone than others, African-Americans among them. Since the incidence of cellphone use is considerably higher among African-Americans than among white Americans, this difficulty will be especially true for a polling firm that does not include cellphone voters in its sample, as Public Policy Polling’s surveys do not.
In practice, it is possible that Public Policy Polling got something like 100 African-American voters on the phone rather than 180, which would be associated with a margin of error of about 10 percent. In addition, all of these African-Americans were reached on landlines, and they may have different demographic characteristics (for instance, being older) than the cellphone voters that the survey missed.
A more robust sample of African-American voters comes from the Gallup poll, which interviews about 3,500 people a week for its national tracking poll and does include cellphones in its sample.
The Gallup poll suggests that Mr. Obama’s ratings among African-American voters may have dropped by a percentage point or two over the past few weeks, but probably not more than that. In the four full weeks of polling since Mr. Obama announced his support for same-sex marriage — which had been unpopular in the African-American community until recently — his approval rating has averaged 86 percent among black voters. That compares with 87 percent for 2012 as a whole.
(…)
There just isn’t any indication that there is much going on here, other than statistical noise. Although Mr. Obama may not win 95 percent of African-American voters in 2012, he will surely win an overwhelming majority of their support. Mr. Obama has much bigger problems to worry about.
There is nothing theoretically wrong with looking at Mr. Obama’s support among different demographic subgroups. But doing so can lead to some bad analysis — sorry, there’s no polite way to put it — if you aren’t alert to this very basic issue of sample size.
Furthermore, keep in mind that in addition to the already-high margin of error related to polling demographic subgroups, the margin of error is calculated in such a way that it is supposed to cover 95 percent of all possible cases. But it won’t cover 100 percent. The results from 1 in 20 polls — or 1 in 20 subsamples within polls — will fall outside the margin of error.
