The Pew Research Center’s study of the impact of response rates on survey quality was conducted in the summer and fall of 2003 and mirrored, in many respects, the Center’s response rate study in 1997.
Two separate surveys were undertaken at the same time with the same questionnaire. One was conducted with 1,000 adult respondents June 4-8, 2003 using the Center’s standard methodology for its national polls. The other study interviewed 1,089 respondents using a more rigorous set of techniques to achieve the highest feasible rate of response, and remained in the field for five months from June 4 until October 30. The standard survey also continued data collection after June 8 and eventually interviewed 1,370 people, with an overall response rate of 37%.
The methods used in the rigorous study differed from the standard survey in several ways. Letters announcing the survey were sent in advance when an address could be identified from the telephone sample (86% of households), with roughly half of these letters including a $2 incentive. In households where a respondent refused to participate, letters were sent to describe the survey process before additional attempts to complete an interview were made. If no contact had been established as of July 24, an additional letter with a $2 incentive was sent to all available addresses. Beginning on July 31, messages explaining the survey were left on answering machines in households where a survey had not been completed. Respondent selection within households used a version of random sampling whereas the standard survey used an “at home” selection method. The rigorous study also included Spanish language interviews (34 of the 1,089 interviews were conducted in Spanish).
