Methodology

Between Oct. 14, 2014, and May 21, 2015, Pew Research Center completed 5,601 face-to-face interviews with non-institutionalized adults ages 18 and older living in Israel. The sample includes interviews with 3,789 Jews, 871 Muslims, 468 Christians and 439 Druze. An additional 34 respondents belong to other religions or are religiously unaffiliated. Five groups were oversampled as part of the survey design: Jews living in the West Bank, Haredim, Christian Arabs, Arabs living in East Jerusalem and Druze.

Interviews were conducted under the direction of Public Opinion and Marketing Research of Israel (PORI). Surveys were administered through face-to-face, paper and pencil interviews conducted at the respondent’s place of residence. Sampling was conducted through a multi-stage stratified area probability sampling design based on national population data available through the Israel’s Central Bureau of Statistics’ 2008 census.

The questionnaire was designed by Pew Research Center staff in consultation with subject matter experts and advisers to the project. The questionnaire was translated into Hebrew, Russian and Arabic, independently verified by professional linguists conversant in regional dialects and pretested prior to fieldwork.

Following fieldwork, survey performance was assessed by comparing the results for key demographic variables with population statistics available through the census. Data were weighted to account for different probabilities of selection among respondents. Where appropriate, data also were weighted through an iterative procedure to more closely align the samples with official population figures for gender, age and education. The reported margins of sampling error and the statistical tests of significance used in the analysis take into account the design effects due to weighting and sample design.

In addition to sampling error and other practical difficulties, one should bear in mind that question wording also can have an impact on the findings of opinion polls.

Questionnaire design

The questionnaire was divided into six sections. The first section included introductory questions and general questions which were asked of all respondents. The questionnaire then branched into four different sections – Jewish, Muslim, Christian and Druze. The final section asked demographic questions of all respondents.

All respondents who took the survey in Russian or Hebrew were branched into the Jewish questionnaire. Arabic-speaking respondents were branched into the Muslim, Christian or Druze questionnaire based on their response to the religious identification question.

Note that not all respondents who took the questionnaire in Hebrew or Russian are classified as Jews in this study. For more details on the definition of religious groups used in this report, please see the sidebar “How religious groups are defined.”

The report is accompanied by a topline survey questionnaire.

Sample design and weighting

Nationally representative surveys in Israel are usually broken down by ethnicity: a Jewish (including non-Jews but no Arabs) and an Arab portion. This approach is also used by the Israeli Census.

Pew Research Center’s survey was designed to achieve a nationally representative sample of both ethnicities – Jews and Arabs. The survey was also designed to obtain large enough samples for analysis of key religious groups – Jews, Haredim, Muslims, Arab Christians and Druze – as well as of key regions – Jews living in the West Bank and Arabs living in East Jerusalem. To do so, the sample design included separate base samples of the two ethnic groups, Jews and Arabs, as well as oversamples of Arab Christians, Druze, Haredim, the West Bank and East Jerusalem.

Base sample:

The base samples were stratified by region, size of locality and socioeconomic index (SES) and each sample was distributed proportionally among the strata based on population data from the 2008 census. The primary sampling units (PSU) were statistical areas (SA) and were randomly sampled. The starting point for each PSU was randomly chosen from coordinates on a map (http://govmap.gov.il).

Households were selected via a random-route procedure. Within a household, interviewers selected the respondent who had the most recent birthday. At every address, up to five calls were made to complete an interview with the chosen respondent.  Attempts to contact the respondent were carried out on different days and hours, unless the chosen respondent, or another person (household member, neighbor, etc.), suggested a specific time to return for an interview.

Oversamples:

To increase the number of respondents in the West Bank and East Jerusalem, additional SAs within the relevant regional strata were sampled. For Haredim, SAs with a homogeneity index of 1 or more were sampled. Data on the homogeneity index was provided by the CBS, 2009. For Arab Christians and Druze, SAs with at least 2% concentration of the targeted group were sampled. For the religious subgroup oversamples, the relevant SAs were stratified by region, size of locality and SES. Each sample was distributed among the strata proportional to the population and the correct number of SAs was sampled to achieve the target sample size for each group.

Weighting:

As a result of these oversamples, the unweighted sample has more respondents belonging to these groups than would normally occur in a nationally representative sample. However, the oversampled religious groups and the higher number of respondents in the West Bank and East Jerusalem do not represent a disproportionately large share of the final, weighted sample that was used for analysis. The oversamples were addressed through statistical adjustment, or weighting, to ensure that Jews and Arabs, along with the various groups that were oversampled, are represented in their proper proportions in the weighted estimates.

Because the Jewish and Arab samples were drawn separately, the weighting also was done separately for each group. The two subsamples were then combined in the third stage of the weighting process. Overall, three different stages of weighting were needed to account for the oversamples and to bring the samples in line with national characteristics of the Jewish and Arab adult populations in Israel.

Step one:

In the first step, a design weight corrects for the oversamples of PSUs in certain regions and with particular demographic characteristics. The design weight takes into account the probabilities of selection of PSUs from within a stratum, selection of households in a PSU and selection of an adult within a contacted household. Respondents living in areas that were oversampled were weighted down due to the fact that they had a higher chance to be interviewed. In addition, respondents residing in households with fewer adults were weighted down because they have a higher probability of being selected than people residing in households with many adults.

For the Jewish sample, the base sample and oversamples were merged and weighted with a single design weight. The same approach was employed for the Arab base sample and oversamples. The design weights for the Jewish and Arab samples corrected the disproportional regional design in each sample and also balanced the samples with respect to the distribution of religious subgroups within each group.

Step two:

In the second step, a post-stratification weight was created for the Jewish and Arab samples separately. A raking technique was used, which adjusts the characteristics of respondents to match known characteristics of the target population for gender, age, region and education. All population characteristics are based on the latest population estimates as published by the Central Bureau of Statistics of Israel.

The Jewish sample was weighted by an interlocking weight of gender and region (West Bank, not West Bank) as well as an interlocking weight of age and region. The sample was also weighted by education and region to match the characteristics of Jews overall in Israel. This weighting approach adjusts the demographic profile of the West Bank sample separately from the full Jewish sample.

Given the lack of parameters for the different Jewish religious subgroups, the sample could not be weighted for age, gender, education or region for the Haredi subsample or any of the other Jewish religious groups. Nonetheless, the sample looks reasonably similar to the demographic breakdowns for the religious subgroups among Jews in the Israel Social Survey.

In the Arab sample, interlocking weights of the three religious groups – Muslims, Christians, Druze – were used along with age, gender and region to adjust the sample. In addition, the Arab sample was weighted on education for the general Arab population in Israel. This weighting approach adjusts the demographic profile separately for each of the religious subgroups among the Arab sample.

Step three:

In the third step, the Jewish and Arab samples were combined and a post-stratification weight was created to match the demographic profile of the national sample to known characteristics of the Israeli population. The full sample was weighted by region and by an interlocking weight of religion and ethnicity (Jews, including non-Jews but not Arabs; Arab Muslims; Arab Christians; and Arab Druze). The other demographics for the combined sample – age, gender, education – were all within acceptable limits and weighting was not necessary.

Sample size and margin of error

This table shows the total, unweighted sample size and associated margin of sampling error for each subgroup. For results based on these subsamples, one can say with 95% confidence that the error attributable to collecting data from some, rather than all, adults within the population is plus or minus the margin of error. This means that in 95 out of 100 samples of the same size and type, the results obtained would vary by no more than plus or minus the margin of error for the population in question.

Response rates

The overall response rate for this survey is 57%. The response rate for different groups included in this survey vary somewhat, but are generally clustered within the range of 50% to 70%.

Mode effect considerations in comparing the results for American and Israeli Jews

One of the important challenges in international survey projects is the decision about the survey design and mode. This decision is influenced by research objectives, local research practices and experience, population coverage expectations and the availability of reliable sampling frames, timelines and costs. Conducting the survey in the same mode across countries, such as by telephone or by face-to-face interviews, is often not feasible in international research, since a mode that is optimal for one country may be a poor choice for another country.³⁵

Telephone surveys in the United States can provide a high-quality nationally representative sample at a lower cost and with a shorter fieldwork period than a face-to-face survey. The situation in Israel is considerably different. Recent research has shown that available telephone frames underrepresent people with a lower socioeconomic status as well as Arabs and other subpopulations in Israel.³⁶ At this point, it does not seem feasible to conduct a nationally representative survey with oversamples of key religious groups by telephone in Israel, which is why the survey was conducted via face-to-face interviews.

Since a goal of this research project is to compare the attitudes of Jews in the U.S. and Israel, it is important to address whether results collected through two different modes are comparable. Face-to-face and telephone interviews are, by and large, administered in very similar ways. Both are conducted by interviewers, meaning that the interviewer reads the questions and response options to respondents. This mode of communication allows interviewers to help respondents stay focused and to provide clarification or encouragement at difficult junctures during the interview.

Nonetheless, the physical presence of the interviewer in face-to-face interviews can lead to a closer rapport between the respondent and the interviewer and to a more engaging experience than in a telephone interview.³⁷ Given these dynamics, it is possible that respondents interviewed by telephone will be more likely than people interviewed face-to-face to give answers that are considered socially acceptable, to avoid being seen in a negative light by the interviewer, and will be more likely to satisfice, meaning they will not be as thoughtful in formulating their responses to the survey questions. However, research has shown that the differences due to the mode in random-probability surveys tend to be small and limited to very sensitive questions.³⁸

While it is important for the reader to be aware that differences observed in this study between American and Israeli respondents might be affected by the modes used, these effects should be considered minimal and most likely do not drive the results in this report.