Does Survey Mode Still Matter?

Over the past several years, pollsters have been working to understand the best approach to conducting survey research in a world where no single technology allows them to reach the entire public. Most reputable polling organizations have adapted their national telephone polls to call both landlines and cell phones in order to deal with the growing cell-phone-only population. Yet, these pollsters still must deal with the fact that response rates for telephone polls are generally under 25%, and adding cell phones to their samples have greatly increased the cost of polling.

At the same time, many firms have been developing technology with the aim of producing reliable and accurate opt-in Internet surveys. These surveys can generally be produced for half the cost of a telephone poll, but they diverge from traditional approaches to survey research because they do not rely on a probability sample. The proliferation of opt-in Internet surveys has generated some controversy within the survey research community (for example, here, here, here, and here). In 2010, the American Association of Public Opinion Researchers (AAPOR) issued a report warning against using opt-in Internet surveys to estimate population values, but also noting that significant evidence on this question was lacking. Indeed, many of the studies that AAPOR relied on when reaching their conclusions were based on outdated data.

More...