by Scott Keeter, Director of Survey Research, Pew Research Center and Greg Smith, Research Fellow, Pew Forum on Religion & Public Life
Muslim Americans are a population of great interest to scholars, journalists and policy makers in the U.S. Yet because Muslims make up a very small percentage of the total U.S. public, it is extremely difficult to interview a large enough sample to provide a reliable picture of their views, experiences and demographic characteristics. This week the Muslim West Facts Project, a partnership between Gallup and the Coexist Foundation, released a survey of American Muslims. Like the Pew Research Center’s 2007 survey of Muslims in the U.S., “Muslim Americans: Middle Class and Mostly Mainstream,” the Gallup study is based on a nationally representative probability sample. But there are important differences in the methodological approach and findings of the two studies.
A principal difference between the studies is how much of the Muslim American population is covered by the study. The Pew Research study includes Muslim Americans who speak English, Arabic, Urdu and Farsi, while the Gallup study covers only those who speak English and Spanish. This difference in approach is critical because about two-thirds of American Muslims are foreign born, and a significant minority of them may not speak English well enough to complete an interview in that language. Based on the languages used in the interviews, Pew Research’s study estimates that at least 17% of Muslim Americans fall into this category.
In part because of this difference, the two studies reach significantly different conclusions about the racial composition, educational attainment and employment status of the population. Gallup estimates that more than one-in-three American Muslims (35%) are black, compared with Pew’s estimate of 24%. Gallup estimates that 40% of Muslim Americans have obtained a college degree; Pew estimates that 24% have done so. And Gallup estimates that more Muslims are currently employed than does Pew Research (70% vs. 57%).
On race, education and employment, Muslims interviewed in English by Pew Research are roughly similar to Muslims interviewed by Gallup. Muslims interviewed by Pew Research in Arabic, Urdu or Farsi, by contrast, exhibit very different characteristics than those interviewed in English. For instance, only 8% of Muslims interviewed in these languages describe their race as black, compared with 27% of Muslims interviewed by Pew Research in English and 35% of Muslims interviewed by Gallup. Only 14% of Muslims interviewed in these languages report having graduated from college, about half the number of college graduates among those interviewed by Pew Research in English and about one-third the number of Muslims interviewed by Gallup. And only 25% of those interviewed in Arabic, Urdu or Farsi are currently employed, which is less than half the employment rate seen among Muslims interviewed in English by Pew Research or Gallup.
Additionally, the two studies sought to learn different kinds of things about Muslim Americans and took very different approaches to finding and interviewing them. Pew’ Research’s questionnaire and sampling approach was tailored specifically to this population. In addition to questions about demographic characteristics, economic status and political behavior, the Pew Research interview also included a large number of questions about the experience of Muslims in the U.S. in the post-9/11 period, immigration, religious attitudes and customs specific to Islam, religious conversion, concern about Islamic extremism and attitudes about foreign policy.
By contrast, the Gallup study is based on responses given by Muslim respondents identified in the course of political, economic and health-tracking surveys of the general public conducted in 2008. Out of more than 319,000 interviews, 946 Muslim respondents were identified. Gallup’s study covered basic demographics, health, employment, economic status, political and civic engagement and community satisfaction, among other topics.
The Pew Research study used two sample sources to reach a broad cross-section of Muslim Americans. One involved re-contacting Muslim respondents identified in previous Pew Research Center interviews; this part of the sample is the most similar to Gallup’s, in that all telephone households had a roughly equal chance of being included in the sample. The second source was a new random-digit dial (RDD) sample divided into five strata. Four of these strata were based on the estimated density of the Muslim population in each county of the United States, with a disproportionate number of interviews conducted in counties estimated to have larger concentrations of Muslims. The fifth stratum consisted of a commercial list of approximately 450,000 households believed to include Muslims, based on an analysis of first and last names common among Muslims. The phone numbers associated with these households were matched against the RDD sample and the duplicate cases removed, which allows the list-based sample to be treated as a random stratum.
Pew Research offered respondents a modest monetary incentive for completing the interview, while Gallup respondents received no incentive. The incentive was offered largely because the Pew Research survey was very lengthy. The average length was more than 30 minutes, significantly longer than most national political surveys. The use of incentives in lengthy or complex surveys is quite common; among the many well-known studies that employ them are the National Immunization Survey, the National Health and Nutrition Examination Survey, the University of Michigan’s Survey of Consumer Attitudes, the California Health Interview Survey and the National Survey of Family Growth.
One strength of the Gallup study is that it included interviews with cell-phone only respondents. The Pew Research study, by contrast, was conducted using landline phones only. This is important since it is likely that many recent immigrants rely only on a cell phone. Of course, some immigrants reached by cell phone may have been unable to complete an interview in English or Spanish. And it is important to note that some Muslim American immigrants speak languages used in neither survey.
Despite the differences in approach and some divergent findings, both studies share the same strength, namely, that they are each based on nationally representative, probability samples of American Muslims. This means that each study can provide estimates with a known degree of sampling error, which is the key for any sample survey that purports to be representative of some larger population. Because federal government surveys do not ask about religious affiliation or background, there is little information about this population from the Census or other large government studies. Thus both the Gallup and Pew Research studies, like those that are sure to follow, help increase our understanding of the Muslim community in the United States.
For more information, see the Pew Reseach Center report on Muslim Americans.
