This report on global restrictions on religion follows the same methodology as the baseline report, with one major difference. This report also tracks changes over time in the extent to which governments and societal groups around the world restrict religious beliefs and practices.
Measuring Change
As part of its original study on global restrictions on religion, the Pew Forum developed two indexes – the Government Restrictions Index (GRI) and the Social Hostilities Index (SHI) – that were used to rate 198 countries and self-governing territories on their levels of restrictions during the two-year period from July 1, 2006, to June 30, 2008. (For more information on the indexes, see below.) Using the original study as a baseline, this study assesses increases and decreases in government restrictions and social hostilities involving religion by comparing each country’s baseline scores on the two indexes with its scores for an overlapping two-year period from July 1, 2007, to June 30, 2009. Comparing rolling averages for overlapping time periods reduces the impact of year-to-year fluctuations and helps identify significant trends (see below).
Determining When Changes on an Index Indicate a Substantial Change
This report highlights changes in a country’s average scores on the Government Restrictions Index and Social Hostilities Index that are deemed to be “substantial.” Unlike in public opinion surveys, there is no statistically determined margin of error that can be used to determine whether observed differences in index scores are significant. Instead, this study defines a change in a country’s score as substantial according to two criteria.
First, to be characterized as a substantial change in this study, the change had to be at least 1.5 standard deviations above or below the mean amount of change among all 198 countries or territories on each index. As shown in the chart on page 74, 16 countries had changes on the Government Restrictions Index that were 1.5 standard deviations or more above the mean amount of change, and 14 countries had changes that were 1.5 standard deviations or more below the mean amount of change.
As shown in the chart above, 16 countries had changes on the Social Hostilities Index that were 1.5 standard deviations or more above the mean amount of change, and 11 countries had changes that were 1.5 standard deviations or more below the mean amount of change.
Second, to be characterized as “substantial,” the change also had to be consistent throughout the full three-year period studied, meaning the change had to be in the same direction both in the period from mid-2006 to mid-2008 and in the overlapping period from mid-2007 to mid-2009. This criterion helps to avoid highlighting short-term spikes that might be due to variability in the coding or information in the primary sources. (For information on the primary sources, see Primary Sources below.)
When both criteria were applied to changes on the GRI, the study found that government restrictions on religion rose substantially in 14 countries and decreased substantially in eight. The scores stayed roughly the same in most (176) countries. When both criteria were applied to changes on the SHI, the study found that social hostilities involving religion rose substantially in 10 countries and decreased substantially in five. The scores stayed roughly the same in most (183) countries.
Forum staff carefully reviewed the coding and primary sources for all countries that did not meet the consistent change criterion on either the Government Restrictions Index or the Social Hostilities Index. The review confirmed that countries eliminated by that criterion were ones where the changes were not drastic, dramatic or substantive.
Advantage of Using the Mean Amount of Change as the Point for Comparison
As noted above, the starting point for measuring changes on the indexes for individual countries is the mean (average) amount of change among all 198 countries and territories. Using the mean for all countries as the starting point is a more conservative approach than using “0” as the starting point, especially for assessing increases in restrictions. This approach does not assume that the overall global increase of .12 points on the Government Restrictions Index and the .02 increase on the Social Hostilities Index were necessarily true increases. Instead, this approach takes into account that those increases might have resulted from variability in the methodology or more detailed information in the primary sources.
In practical terms, using the mean amount of change as the starting point from which changes are assessed means that a country’s index score had to have numerically increased by more than 1.5 standard deviations above its previous score to be considered substantial. In the case of the Government Restrictions Index, a country’s score had to have increased by at least .68 points, which is 1.5 standard deviations (.56) above the mean amount of change (.12). Similarly, a country’s score on the Social Hostilities Index had to have increased by .82 points, which is 1.5 standard deviations (.80) above the mean amount of change (.02) on that index. By this standard, the majority of the increases that met the standard-deviation criterion for a substantial change also met the criterion that it be a consistent change. Indeed, 14 of the 16 countries met both criteria for a substantial increase on the Government Restriction Index, and 10 of the 16 countries met both criteria for a substantial increase on the Social Hostilities Index.
Conversely, for a decrease to be counted as substantial, the decline needed to be less than 1.5 standard deviations below its previous score, since the average change over time was positive (+.12). In the case of the Government Restrictions Index, a country’s score had to have decreased by at least .44 points, which is 1.5 standard deviations (.56) below the mean amount of change (+.12). A country’s score on the Social Hostilities Index had to have decreased by least .78 points, which is 1.5 standard deviations (.80) below the mean amount of change (+.02). By this standard, only eight of the 14 countries that met the first criterion for a substantial decrease on the Government Restriction Index also met the second criterion. Five of the 11 countries that met the first criterion for a substantial decrease on the Social Hostilities Index also met the second criterion.
Advantage of Using Rolling Averages
This three-year study averages the middle year with the first and third years in order to give greater weight to consistent patterns of change on the individual questions that make up the indexes. For example, consider a hypothetical example using one of the 20 questions that make up the Government Restrictions Index: GRI Q.11, “Was there harassment or intimidation of religious groups by any level of government?” In hypothetical Scenario 1, there was a consistent increase in GRI Q.11 across the three individually coded years of data from the lowest intensity score (“No” = 0) in the first year, to the middle level of intensity (“Yes, there was limited harassment” = .250) in the second year, and then to the highest intensity (“Yes, there was widespread harassment” = .500)28 in the third year. The average of the second and third years in Scenario 1 is .375, an increase of .250 points from the average of .125 for the first and second years.
The same scores are present in Scenario 2 in the table above, but in an inconsistent order. Instead of consistently increasing from low to high across the three years as in Scenario 1, the first year in Scenario 2 has the middle level of intensity (“Yes, there was limited harassment” = .250), then drops to low intensity (“No” = 0) in the second year, but ends with high intensity (“Yes, there was widespread harassment” = .500) in the third year. This inconsistent pattern of change results in an overall smaller amount of change between rolling averages (an increase of .125 in Scenario 2 compared with .250 in Scenario 1). This is because the lowest level in Scenario 2 occurred in the second year instead of the first year in Scenario 1, making the average for the period ending in June 2009 in Scenario 2 (.250) smaller than the average in Scenario 1 (.375). However, the rolling average in both Scenarios for the period ending June 2008 (.125) is unaffected because the middle level of intensity on GRI Q.11 still occurred, just in the first year instead of the second.
Similarly, the importance of a consistent pattern of change can be seen when looking at a hypothetical example using one of the 13 questions from the Social Hostilities Index (SHI Q.2, “Was there mob violence related to religion?”).29 In SHI Q.2 Scenario 1, when the intensity of hostilities consistently increased across the three individually coded years of data, the amount of change was greater than when the same scores were present but in an inconsistent order (.385 versus .287).
Overview of Procedures
The methodology used by the Pew Forum to assess and compare restrictions on religion was developed by Senior Researcher and Director of Cross-National Data Brian J. Grim in consultation with other members of the Pew Research Center staff, building on a methodology that Grim and Prof. Roger Finke developed while at Penn State University’s Association of Religion Data Archives.30 The goal was to devise quantifiable, objective and transparent measures of the extent to which governments and societal groups impinge on the practice of religion. The findings were used to rate 198 countries and self-governing territories on two indexes that are reproducible and can be periodically updated.
This research goes beyond previous efforts to assess restrictions on religion in several ways. First, the Pew Forum coded (categorized and counted) data from 18 published cross-national sources, providing a high degree of confidence in the findings. The Pew Forum’s coders looked to the sources only for specific, well-documented facts, not for opinions or commentary.
Second, the Pew Forum’s staff used extensive data-verification checks that reflect generally accepted best practices for such studies, such as double-blind coding (coders do not see each other’s ratings), inter-rater reliability assessments (checking for consistency among coders) and carefully monitored protocols to reconcile discrepancies between coders.
Third, the Pew Forum’s coding took into account whether the perpetrators of religion-related violence were governmental or private actors. The coding also identified how widespread and intensive the restrictions were in each country.
Fourth, one of the most valuable contributions of the indexes and the questions is their ability to chart change over time, as discussed earlier in the methodology.
Countries and Territories
The Pew Forum study covers a total of 198 countries and territories. These include all 192 states that were members of the United Nations during the period under examination (mid-2006 to mid-2009) with the exception of North Korea, for which sufficiently precise and timely data was not available. In addition, the study includes seven territories: Hong Kong, Macau, Taiwan, the Palestinian territories, Kosovo, Western Sahara and Northern Cyprus. These are treated as separate entities, for various reasons, by some or all of the primary information sources for this study. The U.S. State Department, for example, reports separately on Northern Cyprus because it has been administered by Turkish Cypriot authorities since 1974.
Although the 198 countries and territories vary widely in size, population, wealth, ethnic diversity, religious makeup and form of government, the study does not attempt to adjust for such differences. Poor countries are not scored differently on the indexes than wealthy ones. Countries with diverse ethnic and religious populations are not “expected” to have more social hostilities than countries with more homogeneous populations. And democracies are not assessed more leniently or harshly than authoritarian regimes.
Information Sources
The Pew Forum identified 18 widely available, frequently cited sources of information on government restrictions and social hostilities involving religion around the world. This study includes three sources that were not used in the baseline report on religious restrictions. (See below for more details on the new information sources.)
The primary sources, which are listed below, include reports from U.S. government agencies, several independent, nongovernmental organizations and a variety of European and United Nations bodies. Although most of these organizations are based in Western countries, many of them depend on local staff to collect information across the globe. As previously noted, the Pew Forum did not use the commentaries, opinions or normative judgments of the sources; the sources were combed only for factual information on specific policies and actions.
Primary Sources
1. Country constitutions
2. U.S. State Department annual reports on International Religious Freedom
3. U.S. Commission on International Religious Freedom annual reports
4. U.N. Special Rapporteur on Freedom of Religion or Belief reports
5. Human Rights First reports in first and second years of coding; Freedom House reports in third year of coding
6. Hudson Institute publication: Religious Freedom in the World (Paul Marshall)
7. Human Rights Watch topical reports
8. International Crisis Group country reports
9. United Kingdom Foreign & Commonwealth Office annual report on human rights
10. Council of the European Union annual report on human rights
11. Amnesty International reports
12. European Network Against Racism Shadow Reports
13. United Nations High Commissioner for Refugees reports
14. U.S. State Department annual Country Reports on Terrorism
15. Anti-Defamation League reports
16. U.S. State Department Country Reports on Human Rights Practices
17. U.S. National Counterterrorism Center’s Worldwide Incident Tracking System
18. Uppsala University’s Uppsala Conflict Data Program, Armed Conflict Database
U.S. government reports with information on the situation in the United States
19. Dept. of Justice Report on Enforcement of Laws Protecting Religious Freedom 2000-2006
20. Department of Justice “Religious Freedom in Focus” newsletters
21. FBI Hate Crime Reports
As noted above, this study includes three sources that were not included in the Pew Forum’s first report on global restrictions on religion: Freedom House reports; Uppsala University’s Uppsala Conflict Data Program, Armed Conflict Database; and the Worldwide Incident Tracking System (WITS), a publicly available database maintained by the U.S. National Counterterrorism Center, a U.S. government organization that is part of the Office of the Director of National Intelligence. The new sources were used for the most recent year of coding included in this study, July 1, 2008, to June 30, 2009.
The Freedom House reports replaced the Human Rights First reports, which have not been updated since mid-2008. The Uppsala Armed Conflict Database provided a more reliable source of information on the number of people affected by religion-related armed conflicts. While other primary sources report the effects of such conflicts, they sometimes provide only ballpark estimates on the number of people affected by such conflicts.
Some of the apparent increase in religion-related terrorism noted in this study could reflect the use of the WITS database, which provided greater detail on the number of people affected by religion-related terrorism than the State Department’s International Religious Freedom reports or the U.S. State Department’s annual Country Reports on Terrorism, which were the primary sources used to code terrorist incidents in the baseline report.
As explained in more detail below, the Pew Forum’s staff developed a battery of questions similar to a survey questionnaire. Coders consulted the primary sources in order to answer the questions separately for each country. While the U.S. State Department’s annual reports on International Religious Freedom generally contained the most comprehensive information, the other sources provided additional factual detail that was used to settle ambiguities, resolve contradictions and help in the proper scoring of each question.
The questionnaire, or coding instrument, generated a set of numerical measures on restrictions in each country. It also made it possible to see how government restrictions intersect with broader social tensions and incidents of violence or intimidation by private actors. The coding instrument with the list of questions used for this report is shown in the Summary of Results.
The coding process required the coders to check all the sources for each country. Coders determined whether each source: provided information critical to assigning a score; had supporting information but did not result in new facts; or had no available information on that particular country. Multiple sources of information were available for all countries and self-administering territories with populations greater than 1 million. More than three-in-four of the countries and territories analyzed by the Pew Forum were multi-sourced; only small, predominantly island, countries had a single source, namely, the U.S. State Department reports.
Coding the United States presented a special problem since it is not included in the State Department’s annual reports on International Religious Freedom. Accordingly, the Pew Forum’s coders also looked at reports from the Department of Justice and the FBI on violations of religious freedom in the United States, in addition to consulting all of the primary sources, including reports by the United Nations, Human Rights Watch, Freedom House, the International Crisis Group and the U.K. Foreign & Commonwealth Office, many of which do contain data on the United States.
The Coding Process
The Pew Forum employed strict training and rigorous coding protocols to make its coding as objective and reproducible as possible. Coders worked directly under a senior researcher’s supervision, with additional direction and support provided by other Pew Forum researchers. The coders underwent an intensive training period that included a thorough overview of the research objectives, information sources and methodology.
Countries were double-blind coded by two coders (coders did not see each other’s ratings), and the initial ratings were entered into Microsoft Excel spreadsheets. The coders began by filling out the coding instrument for each country using the information source that had the most comprehensive information, typically the U.S. State Department reports. The protocol for each coder was to answer every question on which information was available in the initial source. Once a coder had completed that process, he or she then turned to the other sources. As new information was found, this was also coded and the source duly noted. Whenever ambiguities or contradictions arose, the source providing the most detailed, clearly documented evidence was used.
After two coders had separately completed the coding instrument for a particular country, their scores were compared by a senior researcher. Areas of discrepancy were discussed at length with the coders and were reconciled in order to arrive at a single score on each question for each country. The Excel worksheets for each country were then combined into a master file, which was imported into SPSS.
Throughout this process, the coding instrument itself was continually monitored for possible defects. The questions were designed to be precise, comprehensive and objective so that, based on the same data and definitions, the coding could be reliably reproduced by others with the same results.
Pew Forum staff generally found few cases in which one source contradicted another. When contradictions did arise – such as when sources provided differing estimates of the number of people displaced due to religion-related violence – the source that cited the most specific documentation was used. The coders were instructed to disregard broad, unsubstantiated generalizations regarding abuses and to focus on reports that contained clear, precise documentation and factual detail, such as names, dates and places where incidents occurred.
Inter-rater reliability statistics were computed by comparing the coders’ independent, blind ratings. The Pew Forum took scores from one coder for the 198 countries and compared them with another coder’s scores for the same questions, computing the degree to which the scores matched. These measures were very high, with an average score of .8 or above on the key variables. Scores above .8 on a 0-to-1 scale are generally considered very good, and scores around .7 are generally acceptable. The Pew Forum’s overall inter-rater reliability average across all the variables coded was greater than .8 for each year.
The data-verification procedures, however, went beyond the inter-rater reliability statistics. They also involved comparing the answers on the main measures for each country with other closely related questions in the dataset. This provided a practical way to test the internal reliability of the data.
Pew Forum staff also checked the reliability of the Pew Forum’s coded data by comparing them with similar, though more limited, religious restrictions datasets. In particular, published government and social regulation of religion index scores are available from the Association of Religion Data Archives (for three years of data) and the Hudson Institute (for one year of data), which makes them ideal measures for cross validation. The review process found very few significant discrepancies in the coded data; changes were made only if warranted by a further review of the primary sources.
Restriction of Religion Indexes
The Government Restrictions Index is based on 20 indicators of ways that national and local governments restrict religion, including through coercion and force. The Social Hostilities Index is based on 13 indicators of ways in which private individuals and social groups infringe on religious beliefs and practices, including religiously biased crimes, mob violence and efforts to stop particular religious groups from growing or operating. The study also counted the number and types of documented incidents of religion-related violence, including terrorism and armed conflict.
Government Restrictions Index
Coding multiple indicators makes it possible to construct a Government Restrictions Index of sufficient gradation to allow for meaningful cross-national comparisons. An additional advantage of using multiple indicators is that it helps mitigate the effects of measurement error in any one variable, providing greater confidence in the overall measure.
The Pew Forum coded 20 indicators of government restrictions on religion (see the Summary of Results). In two cases, these items represent an aggregation of several closely related questions: Measures of five types of physical abuses are combined into a single variable (GRI Q.19); and seven questions measuring aspects of government favoritism are combined into an overall favoritism scale (GRI Q.20 is a summary variable showing whether a country received the maximum score on any one or more of the seven questions). These 20 items were added together to create the GRI.
A test of whether the 20 items were statistically reliable as a single index produced a scale reliability coefficient of greater than .9 for each year. Since coefficients of .7 or higher are generally considered acceptable, it was appropriate to combine these 20 items into a single index.
The GRI is a fine-grained measure created by adding the 20 items on a 0-to-10 metric, with 0 indicating very low government restrictions on religion and 10 indicating extremely high restrictions. This involved two general calculations. First, the 20 questions that form the GRI were standardized so that each variable had an identical maximum value of 1 point, while gradations among the answers allowed for partial points to be given for lesser degrees of the particular government restriction being measured. Second, the overall value of the index was proportionally adjusted so that it had a maximum value of 10 and a possible range of 0 to 10 by dividing the sum of the variables by 2.
Social Hostilities Index
In addition to government restrictions, violence and intimidation in societies also can limit religious beliefs and practices. Accordingly, Pew Forum staff tracked more than a dozen indicators of social impediments on religion. Once again, coding multiple indicators made it possible to construct an index that shows gradations of severity or intensity and allows for comparisons between countries. The Summary of Results contains the 13 items used by Pew Forum staff to create the Social Hostilities Index.
As with the Government Restrictions Index, various types of violence and intimidation were combined. A test of whether these 13 items were statistically reliable as a single index produced a scale reliability coefficient of .9 or higher for each year. Since coefficients of .7 or higher are generally considered acceptable, it was statistically appropriate to combine these items into a single index.
The SHI was constructed by adding together the 13 indicators based on a 0-to-10 metric, with 0 indicating very low social impediments to religious beliefs and practices and 10 indicating extremely high impediments. This involved two general calculations. First, the various questions that form the index were standardized so that each variable had an identical maximum value of 1 point, while gradations among the answers allowed for partial points to be given for lesser degrees of the particular hostilities being measured. Second, the indicators were added together and set to have a possible range of 0 to 10 by dividing the sum of the variables by 1.3.
Levels of Restrictions
The Pew Forum categorized the levels of government restrictions and social hostilities by percentiles. Countries with scores in the top 5% on each index are categorized as “very high.” The next highest 15% of scores are categorized as “high,” and the following 20% are categorized as “moderate.” The bottom 60% of scores are categorized as “low.” Readers should note that since the indexes measure the accumulated impact and severity of restrictions, distinctions among the scores of the countries in the bottom 60% of scores are less significant than distinctions made at the upper end of the indexes, where differences in the number and severity of restrictions between countries are greater. This is evident by the fact that the range of difference between scores of countries in the entire bottom 60% (0.1-2.4 on the GRI and 0-1.8 on the SHI) is about the same as the range of differences between scores in just the top 5% (7.2-8.3 on the GRI and 7.2-9 on the SHI).
Notes on Changes in Questions
Readers should be cautioned that some differences on individual measures may not be as significant as they appear due to minor fluctuations in coding procedures. This was especially the case for GRI Q.3 and SHI Q.5. As shown in the Summary of Results for GRI Q.3 (“Taken together, how do the constitution/basic law and other national laws and policies affect religious freedom?”), the number of countries where coders answered “0” on that question increased from 53 countries in the two-year period ending in mid-2008 to 75 countries in the two-year period ending in mid-2009.31 However, as noted in the Summary of Results, this change was likely attributable to a slight variation in the coding procedures across the years. During the first year coded (July 1, 2006, to June 30, 2007), the coders were more likely to give a “1” on this question than in either of the two subsequent years.32 A post-coding analysis suggests that this was because the coders in that year were more likely to code the presence of a few restrictions on religious freedom by the government as a “1”. In subsequent years the coders had a higher bar for coding “1”: the presence of restrictions alone was not sufficient, there also had to be clear harassment or abuses toward religious groups or their members.
As noted earlier in the methodology, some of the increase in religion-related terrorism (SHI Q.5) found in this study could reflect the use of new source material that provided greater detail on terrorist activities than the sources used in the baseline report. Specifically, in coding terrorist activities during the period from July 1, 2008, to June 30, 2009, coders used the Worldwide Incidents Tracking System (WITS) database, which is the U.S. National Counterterrorism Center’s database of terrorist incidents.
Changes in two other questions are important to note. The first two variables in the Government Restrictions Index on the presence of laws protecting religious freedom (GRI Q.1) and constitutional qualifications or contradictions of those protections (GRI Q.2) were back-coded so that the coding protocols for the period from mid-2006 to mid-2008 matched the protocols for the period from mid-2007 to mid-2009. This resulted in small changes to the mid-2008 Government Restrictions Index scores reported in this study from those reported in the baseline study (the mid-2008 index scores reported in this study reflect the revised coding). While some countries changed or amended their constitutions during the three-year period covered by this study, none of those changes affected the results of the coding, and any differences between the revised results and those reported in the baseline study do not represent changes but rather a slight revision in the methodology.
Finally, it is important to note that situations within countries may have changed since the end of the periods studied. One significant change that occurred outside the time frame of this study, for example, is the division of Sudan into two separate countries. Subsequent Pew Forum reports on global restrictions on religion will assess the South and the North separately; in this report, however, Sudan’s score represents the overall country in which the government in Khartoum was dominant.
New Questions Added to the Study
This report includes one new question that is not part of the indexes and was not included in the baseline report: the number of countries with laws penalizing blasphemy, apostasy or the defamation of religion. This question was coded only in the last year of the study (mid-2008 to mid-2009), but all the laws and policies coded predated July 1, 2008. This question is analyzed separately in the section beginning on page 67.
Additionally, this study reports on the number of countries where specific religious groups faced government or social harassment. This is essentially a cross-tabulation of GRI Q.11 (“Was there harassment or intimidation of religious groups by any level of government?”) and SHI Q.1a. (“Did individuals face harassment or intimidation motivated by religious hatred or bias?”). For purposes of this study, the definition of harassment includes any mention in the primary sources of an offense against an individual or group based on religious identity. Such offenses may range from physical attacks and direct coercion to more subtle forms of discrimination. But merely prejudicial opinions or attitudes, in and of themselves, do not constitute harassment unless they are acted upon in a palpable way.
As noted above, this study provides data on the number of countries in which different religious groups are harassed or intimidated. But the study does not assess either the severity or the frequency of the harassment in each country. Therefore, the results should not be interpreted as gauging which religious group faces the most harassment or persecution around the world.
Religion-Related Terrorism and Armed Conflict
Terrorism and war can have huge direct and indirect effects on religious groups, destroying religious sites, displacing whole communities and inflaming sectarian passions. Accordingly, the Pew Forum tallied the number, location and consequences of religion-related terrorism and armed conflict around the world, as reported in the same primary sources used to document other forms of intimidation and violence. However, war and terrorism are sufficiently complex that it is not always possible to determine the degree to which they are religiously motivated or state sponsored. Out of an abundance of caution, this study does not include them in the Government Restrictions Index. They are factored instead into the index of social hostilities involving religion, which includes one question specifically about religion-related terrorism and one question specifically about religion-related war or armed conflict. In addition, other measures in both indexes are likely to pick up spillover effects of war and terrorism on the level of religious tensions in society. For example, hate crimes, mob violence and sectarian fighting that occur in the aftermath of a terrorist attack or in the context of a religion-related war would be counted in the Social Hostilities Index, and laws or policies that clearly discriminate against a particular religious group would be registered on the Government Restrictions Index.
For the purposes of this study, the term religion-related terrorism is defined as premeditated, politically motivated violence perpetrated against noncombatant targets by subnational groups or clandestine agents that have some identifiable religious ideology or religious motivation. It also includes acts carried out by groups that have a non-religious identity but target religious personnel, such as clergy. Readers should note that it is the political character and motivation of the groups, not solely the type of violence, that is at issue here. For instance, a bombing would not be classified as religion-related terrorism if there was no clearly discernible religious ideology or bias behind it unless it was directed at religious personnel. Religion-related war or armed conflict is defined as armed conflict (a conflict that involves sustained casualties over time or more than 1,000 battle deaths) in which religious rhetoric is commonly used to justify the use of force, or in which one or more of the combatants primarily identifies itself or the opposing side by religion.
Potential Biases
As noted in the report, the primary sources indicate that the North Korean government is among the most repressive in the world, including toward religion. Because of independent observers’ lack of regular access to North Korea, however, the sources are unable to provide the kind of specific, timely information that forms the basis of this report. Therefore, North Korea is not included on either index.
This raises two important issues concerning potential information bias in the sources. The first is whether other countries that limit outsiders’ access and that may seek to obscure or distort their record on religious restrictions were adequately covered by the sources. Countries with relatively limited access have multiple primary sources of information that the Pew Forum used for its coding. Each is also covered by other secondary quantitative datasets on religious restrictions that have used a similar coding scheme, including earlier years of coded data from U.S. State Department reports previously produced by Grim at Penn State’s Association of Religion Data Archives (ARDA) project (three datasets); independent coding by experts at the Hudson Institute’s Center for Religious Liberty using indexes also available from ARDA (one dataset); and content analysis of country constitutions conducted by the Becket Fund for Religious Liberty (one dataset). Pew Forum staff used these for cross-validation. Contrary to what one might expect, therefore, even most countries that limit access to information tend to receive fairly extensive coverage by groups that monitor religious restrictions.
The second key question – the flipside of the first – is whether countries that provide freer access to information receive worse scores simply because more information is available on them. As described more fully in the methodology from the baseline report, Forum staff compared the length of U.S. State Department reports on freer-access countries with those of less-free countries. The comparison found that the median number of words was approximately three times as large for the limited-access countries as for the open-access countries. This confirms that problems in freer-access countries are generally not overreported in the U.S. State Department reports.
Only when it comes to religion-related violence and intimidation in society do the sources report more problems in the freer-access countries than in the limited-access ones. However, the Social Hostilities Index includes several measures – such as SHI Q.8 (Did religious groups themselves attempt to prevent other religious groups from being able to operate?) and SHI Q. 11 (Were women harassed for violating religious dress codes?) – that are less susceptible to such reporting bias because they capture general social trends or attitudes as well as specific incidents of violence. With these limitations in mind, it appears that the coded information on social hostilities is a fair gauge of the situation in the vast majority of countries and a valuable complement to the information on government restrictions.
Data on social impediments to religious practice can more confidently be used to make comparisons between countries with sufficient openness, which includes more than nine-in-ten countries covered in the Pew Forum’s coding. An analysis by Grim and Richard Wike, Associate Director of the Pew Research Center’s Global Attitudes Project, tested the reliability of the State Department reports on social impediments to religious practice by comparing public opinion data with data coded from the reports in previous years by Grim and experts at Penn State. They concluded that “the understanding of social religious intolerance embodied in the State Department reports is comparable with the results of population surveys and individual expert opinion.”33
Finally, the 2010 population figures used in this report are estimates from the United Nations World Population Prospects, 2008 Revision. A new revision was released in April 2011, after most of the analysis for this report was completed.
Footnotes:
28 Note that the maximum value for a question on the 20-question, 10-point Government Restrictions Index is .500 (20 x .500 = 10). (return to text)
29 Note that the maximum value for a question on the 13-question, 10-point Social Hostilities Index is .770 (13 questions x .770 = 10 points). (return to text)
30 See Brian J Grim and Roger Finke, “International Religion Indexes: Government Regulation, Government Favoritism, and Social Regulation of Religion,” Interdisciplinary Journal of Research on Religion, Volume 2, Article 1, 2006. (return to text)
31 0 = “National laws and policies provide for religious freedom, and the national government respects religious freedom in practice.” (return to text)
32 1 = “National laws and policies provide for religious freedom, and the national government generally respects religious freedom in practice; but there are some instances, e.g., in certain localities, where religious freedom is not respected in practice.” (return to text)
33 See Brian J Grim and Richard Wike, “Cross-Validating Measures of Global Religious Intolerance: Comparing Coded State Department Reports With Survey Data and Expert Opinion,” Politics and Religion, Volume 3, Issue 1, pages 102-129, April 2010. (return to text)
