When asked about their news consumption habits and their views of the media in general, Americans give similar answers regardless of whether they are surveyed by phone or online, according to a Pew Research Center analysis of new data from a 2017 study and findings from a 2014 survey. Just one question with very general response options – how much people followed news – did not follow this pattern, yielding a noticeably higher news consumption estimate in phone surveys than web surveys.
The new analysis sheds light on concerns raised among pollsters that the medium by which a survey question is asked – its mode – can affect responses. (In this case, a telephone survey with an interviewer was compared with a self-administered survey on the web.) Under a phenomenon known as “social desirability bias,” some respondents might be inclined to give more honest answers online than they would on the phone because online surveys do not involve a human interviewer and therefore are inherently more private.
For example, respondents in the Center’s 2014 survey were more likely to say they were “very satisfied” with their lives when interviewed on the phone than online. Questions about news consumption might similarly be at risk of this mode effect if respondents believe that their responses about media habits would portray them in a more positive light to an interviewer.
A set of comparisons across seven different media questions, however, shows little if any of this sort of mode effect on six specific items. On questions about how much Americans enjoy keeping up with the news, how in touch they feel journalists are with the public, whether they have ever stopped talking to someone over political news, and whether they recently got news from the newspaper, TV, or radio, the Center’s analysis found no mode differences.
On a more general question, though, there was a difference between the two modes. When asked how much they followed the news, 63% of U.S. adults who were interviewed on the phone said “all or most of the time.” When asked the same question on the web, the number dropped to 49%.
The difference is especially strong among black U.S. adults: 66% said they follow news most of the time when asked on the phone, versus just 34% who answered this way on the web – a difference of 32 percentage points. By contrast, the difference among whites was just 11 points (65% on the phone vs. 54% on the web).
Mode also generally had a more pronounced effect on the responses from Democrats than on the responses from Republicans. Democrats, including independents who lean Democratic, were 17 points more likely to say that they followed news when asked on the phone (66%) than on the web (49%). Among Republicans and Republican leaners, by comparison, there was just a 7-point gap between those who were asked about their news habits on the phone (59%) and the web (52%).
Pew Research Center and other organizations have long asked how the presence or lack of an interviewer affects a broad range of survey questions. In the case of questions about news and media, the absence of mode effect on more specific questions – such as whether respondents got news from a newspaper “yesterday” – suggests that it may help to ask respondents to think more precisely about their news habits.
Note: Survey methodologies can be found here (2017) and here (2014), and 2017 topline is available here (PDF).
