The Pew Research Center reports that pollsters are finding it much more difficult to get people to respond to their surveys:
For decades survey research has provided trusted data about political attitudes and voting behavior, the economy, health, education, demography and many other topics. But political and media surveys are facing significant challenges as a consequence of societal and technological changes.
It has become increasingly difficult to contact potential respondents and to persuade them to participate. The percentage of households in a sample that are successfully interviewed – the response rate – has fallen dramatically. At Pew Research, the response rate of a typical telephone survey was 36% in 1997 and is just 9% today.
The general decline in response rates is evident across nearly all types of surveys, in the United States and abroad. At the same time, greater effort and expense are required to achieve even the diminished response rates of today. These challenges have led many to question whether surveys are still providing accurate and unbiased information. Although response rates have decreased in landline surveys, the inclusion of cell phones – necessitated by the rapid rise of households with cell phones but no landline – has further contributed to the overall decline in response rates for telephone surveys.
A new study by the Pew Research Center for the People & the Press finds that, despite declining response rates, telephone surveys that include landlines and cell phones and are weighted to match the demographic composition of the population continue to provide accurate data on most political, social and economic measures. This comports with the consistent record of accuracy achieved by major polls when it comes to estimating election outcomes, among other things.
(…)
The study is based on two new national telephone surveys conducted by the Pew Research Center for the People & the Press. One survey was conducted January 4-8, 2012 among 1,507 adults using Pew Research’s standard methodology and achieved an overall response rate of 9%. The other survey, conducted January 5-March 15 among 2,226 adults, used a much longer field period as well as other efforts intended to increase participation; it achieved a 22% response rate.
Most political pollsters will tell you that a poll conducted over a longer time period tends to be less reliable than one conducted over a two or three day period. This is because the longer the poll is in the field, the less representative it becomes of public opinion at a given point in time. A poll conducted over at two month period may end up with a 22% response rate, but it’s not going to be very valuable to analysts or other interested in figuring out what might be going on in the minds of the electorate. That’s why the typical political poll is usually only in the field for two or three days at the most. The only exception you’ll see to that generally will be tracking polls, which are a different animal from regular political polls in many ways anyway. So, if you’re going to conduct a political poll, you have to accept the fact that you’re going to get about 9% response rate, and plan accordingly.
As Pew notes, there have been several reports in recent years of pollsters having difficulty getting responses to polls in the way that they did in the past. Thanks to Caller ID and cell phones, many people screen calls and decide not to answer those from numbers they don’t recognize. Additionally, it seems fair to say that Americans have become more concerned about their privacy and less willing to answer questions from some stranger who could be half a country away for all they know. What this usually means for pollsters, of course, is that they have to find a way to maintain an acceptable sample size with the correct demographics, and the usual solution to that is to start out with a larger list of phone numbers to call. Of course, this increases costs to the pollster and, perhaps, brings the integrity of the poll itself into question.
Pew notes in the survey that, despite the low response rates, pollsters are still able to come up with representative samples if they properly balance the demographic factors, especially those such as age, race, and gender that tend to have a very strong influence on political opinions. The question that this raises, though, is whether we’re reaching a sea change in the polling industry where polling by telephone, whether cell phone or land line, is going to become less and less reliable. One solution that some pollsters, like Rasmussen and PPP have come up with is to use automated response polls. However, as I noted last month, those polls completely leave cell phones out of the mix because they are not allowed to call them. This potentially results in non-representative samples as more and more people go to a cell phone only lifestyle. The obvious next step for polling, of course, would be to go online, and there are some polling companies that use this method. However, as many have noted, they tend to be less reliable than the telephone polls. It also requires some degree of trust that the response one is getting are reliable and truthful, and assumes that people willing to go online and answer Internet surveys are representative of the population as a whole. As we’ve seen from the “flash polls” that many media companies put on their websites, that most assuredly isn’t the case. If it were Ron Paul would be the Republican nominee for President. The pollsters using online polling do things somewhat more scientifically than that, but it strikes me that they still have the same problem.
All of this leaves us with a question, if response rates continue to fall precipitously are we witnessing the end of polling?
