The data in this report are based on three independent survey administrations based on the same randomly selected, nationally representative group of respondents. The first is Pew Research Center’s largest survey on domestic politics to date: the 2014 Political Polarization and Typology Survey, a telephone survey of just over 10,000 Americans. A subset of these respondents has been impaneled into the American Trends Panel. Follow-up surveys have been conducted with them, and this report includes data from two of these follow-up surveys (the first by Web and telephone, the second by Web and mail). The main telephone survey and the panel surveys are described separately, in further detail, in the section that follows.
For the majority of this analysis, the samples we looked at included the same respondents or a highly overlapping group of respondents. We accounted for this overlap in all tests of statistical significance. As a result of the overlaps among the samples, the margin of sampling error reported in this methodological statement should not be used to assess the significance of differences between the various measures of racial identification tested here.
Overview of Telephone Survey Methodology
The telephone survey was conducted January 23-March 16, 2014 among a randomly selected national sample of 10,013 adults, 18 years of age or older, living in all 50 U.S. states and the District of Columbia (5,010 respondents were interviewed on a landline, and 5,003 were interviewed on a cellphone, including 2,649 who had no landline telephone). The survey was conducted under the direction of Abt SRBI. A combination of landline and cellphone random digit dial samples were used; both samples were provided by Survey Sampling International. Interviews were conducted in English and Spanish. Respondents in the landline sample were selected by randomly asking for the youngest adult male or female who was at home at the time of the call. Interviews in the cell sample were conducted with the person who answered the phone, if that person was an adult 18 years of age or older. For detailed information about our survey methodology, see http://pewresearch.org/pewresearch-org/politics/methodology/.
Data collection was divided equally into three phases (A, B, and C) with independent samples, non-overlapping interview dates and separate weighting. The questionnaire for each phase contained a core set of measures of political attitudes and values, political engagement and demographic characteristics, along with a set of unique questions about issues, lifestyle, media use and other topics. Additionally, most respondents to the survey were invited to join the newly created Pew Research Center’s American Trends Panel, described below.
The combined landline and cellphone sample is weighted using an iterative technique that matches gender, age, education, race, Hispanic origin and nativity and region to parameters from the 2012 Census Bureau’s American Community Survey and population density to parameters from the 2010 U.S. Census. The sample is also weighted to match current patterns of telephone status (landline only, cellphone only or both landline and cellphone), based on extrapolations from the January-June 2013 National Health Interview Survey. The weighting procedure accounts for the fact that respondents with both landline and cellphones have a greater probability of being included in the combined sample and adjusts for household size among respondents with a landline phone. Sampling errors and statistical tests of significance take into account the effect of weighting.
The following table shows the unweighted sample size and the error attributable to sampling that would be expected at the 95% level of confidence:
Sample sizes and sampling errors for subgroups are available upon request.
In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.
For a more detailed explanation of the telephone survey sampling, see: https://www.pewresearch.org/politics/2014/06/26/appendix-3-about-the-surveys/#telephone-survey-methodology-in-detail
The American Trends Panel Surveys (ATP)
The American Trends Panel (ATP) is created by the Pew Research Center and managed by Abt SRBI. The Abt SRBI panel team included Chintan Turakhia, Charles DiSogra, Courtney Kennedy, Nick Bertoni, Marci Schalk, Robert Magaw, Emily Martinez, Allison Ackermann and Sandra Hernandez.
The ATP is a nationally representative panel of randomly selected U.S. adults living in households. Respondents who self-identify as internet users and who provided an email address participate in the panel via monthly self-administered Web surveys, and those who do not use the internet, do not have an email address, or refuse to provide their email address participate via the mail.
Data in this report are drawn from the fifth and the seventh waves of the panel conducted in 2014. For analysis comparing the same respondents’ answers across multiple questions, the data from these two waves and the initial telephone survey were merged. A separate weight was created for analysis based on the subset of respondents who answered both wave 5 and wave 7 of the ATP (N=2,721).
The fifth wave of the panel was conducted July 7-August 4, 2014 among 3,351 respondents. This wave featured a mode experiment in which web panelists were randomly assigned to either Web or telephone (CATI) mode, resulting in 1,509 experimental web mode completes, 1,494 experimental telephone mode completes and 348 non-experimental phone mode completes (resulting in 1,842 total phone mode completes). The margin of sampling error for the full sample of 3,351 respondents is plus or minus 2.2 percentage points; the margins of sampling error for the web sample and the phone sample are plus or minus 3.3 percentage points and plus or minus 3.0 percentage points, respectively. The seventh wave of the panel was conducted September 9-October 3, 2014 among 3,154 respondents (2,811 by Web and 343 by mail). The margin of sampling error for the full sample of 3,154 respondents is plus or minus 2.7 percentage points.
All current members of the American Trends Panel were originally recruited from the 2014 telephone survey discussed in the last section of this methodology. At the end of that survey, respondents were invited to join the panel. The invitation was extended to all respondents who use the internet (from any location) and a random subsample of respondents who do not use the internet.16
Of the 10,013 adults interviewed, 9,809 were invited to take part in the panel. A total of 5,338 agreed to participate and provided either a mailing address or an email address to which a welcome packet, a monetary incentive and future survey invitations could be sent. Panelists also receive a small monetary incentive after participating in each wave of the survey.
The ATP data were weighted in a multi-step process that begins with a base weight incorporating the respondents’ original survey selection probability and the fact that some panelists were subsampled for invitation to the panel. Next, an adjustment was made for the fact that the propensity to join the panel and remain an active panelist varied across different groups in the sample. The final step in the weighting uses an iterative technique that matches gender, age, education, race, Hispanic origin and region to parameters from the U.S. Census Bureau’s 2013 American Community Survey. Population density is weighted to match the 2010 U.S. decennial census. Telephone service is weighted to estimates of telephone coverage for 2014 that were projected from the January-June 2013 National Health Interview Survey. It also adjusts for party affiliation using an average of the three most recent Pew Research Center general public telephone surveys and for internet use using as a parameter a measure from the 2014 Survey of Political Polarization. Sampling errors and statistical tests of significance take into account the effect of weighting. The Hispanic sample in the American Trends Panel is predominantly native born and English speaking.
The following table shows the field periods, unweighted sample sizes and response rates for each wave. The cumulative response rate takes account of the response rate for the 2014 Survey of Political Polarization (10.6%) and attrition from panel members who were removed at their request or for inactivity.
