In spring of 2020, Pew Research Center suspended international face-to-face data collection as COVID-19 emerged as a major global public health concern. At that time, we – and many within the wider survey research community – were uncertain about how the situation would unfold in the coming months. Would things be back to normal soon? Could we continue with phone surveying in countries where that is a commonly used mode? And how willing would respondents be to take surveys during a pandemic?
Roughly 18 months on, the pandemic is still with us. But some of our questions have been answered. We were able to quickly restart survey operations in summer 2020 by focusing on economically advanced countries and relying on the Center’s web-based American Trends Panel in the United States and phone polls elsewhere.
Since the initial disruptions of field operations last year due to COVID-19, we have been able to conduct 33 surveys in 17 countries and territories. Still missing from our survey coverage are the middle- and lower-income countries that we polled face-to-face prior to the pandemic. In some of these places, we are exploring a shift from in-person to telephone or online polling, but in many instances, we see little option but to wait for reduced health risks and a return to in-person polling. The wait could be substantial, depending on the virulence of coronavirus variants, immunization rates and public health restrictions.
Dialing into phone surveys
Relying exclusively on phone polling, even in countries where this had been our traditional survey mode, offered no guarantee that we could successfully achieve high-quality, nationally representative surveys. COVID-19 disrupted phone research, as well as in-person interviewing. But our experience proved positive, and even yielded several positive takeaways:
- Local research vendors in most countries were able to move quickly from centralized interviewing based at call centers to decentralized setups with staff calling from home. Once the Center was able to verify the security, quality and efficiency of the sample management and dialing systems, this adaptation overcame in-country restrictions such as lockdowns and periodic closures of nonessential businesses. In countries where remote interviewing was not used, vendors implemented a wide range of health and safety protocols, including reduced staffing, increased wellness checks for staff and enhanced call-center cleaning.
- Likewise, vendors were able to offer online training sessions for interviewers so Center staff could confirm the quality of interviewing. While this was an option in past projects, the pandemic was a catalyst for improving use of technology among local vendors, which in turn improved Center access and oversight.
- The sample performance relative to key demographics in all countries surveyed during the pandemic, such as age, gender, education and geographic location, met our standards for representativeness applied in past projects. Though our phone polling over the past two years was largely confined to countries where we previously used this mode of data collection, the methodology remained robust over the course of the pandemic.
- We did not find that respondents were more difficult to interview by phone during the pandemic. Despite some country-to-country variability and differences between landline and mobile respondents, our response rates were generally within a percentage point or two of pre-pandemic rates. Our monitoring of survey outcome trends will continue as the pandemic unfolds.
Surveying in 2022 and beyond
In the months ahead, we intend to expand our country coverage to a more economically and geographically diverse set of countries. In some cases, we are actively exploring a shift from face-to-face interviews to ones conducted by phone, or even online surveys. We’ve successfully made similar mode changes in the past (see here and here). As always, we are mindful not only of the effect that mode may have on sample quality and response patterns, but also the issue of operational reliability.
While we are optimistic about transitioning to phone surveys in additional countries, we assume that face-to-face interviews will remain our preferred method of survey administration in many middle- and lower-income countries. Here, too, we are already testing the waters to better understand how well local polling partners can implement surveys, while also adhering to public health requirements and safeguarding the well-being of both employees and respondents. Our approach to resuming in-person surveys will be gradual, given the uncertainty generated by new coronavirus variants, the variability of vaccination rates, government lockdowns and other measures. Of course, if the threat posed by COVID-19 were to substantially recede in the coming months, we would consider a swifter return to face-to-face polling.
The only constant for us is placing a premium on the safety and health of our in-country research partners and survey respondents around the globe.
Again, the good news is that our initial steps to mitigate the disruption caused by COVID-19 have proven effective. By solidifying phone-based polling in roughly a dozen countries, and then gradually expanding phone and online operations to additional locations, we’ve established a resilient survey research network and even made gains in quality control. There is no doubt that challenges remain as we navigate toward the “new normal” of comparative, international survey research. The only constant for us is placing a premium on the safety and health of our in-country research partners and survey respondents around the globe.
