Abortion remains a contentious issue among U.S. Christians. But when millions of churchgoers file into the pews each week, what do they really hear? A new Pew Research Center analysis finds that just 4% of sermons shared on U.S. church websites in the spring of 2019 discussed abortion even once – and when they did, it was rarely mentioned repeatedly. Still, pastors who broached the topic were nearly unanimous in their opposition.
Although they are relatively rare, these mentions of abortion add up over time: Christian churches that shared their sermons online posted an average of nine sermons each during the eight-week study period – and roughly one-in-five of those congregations (19%) heard at least one sermon that mentioned abortion. To arrive at these conclusions, the Center analyzed nearly 50,000 sermons shared online or livestreamed by more than 6,000 U.S. churches and delivered between April 7 and June 1, 2019 – a period that included Easter.
While the database is not representative of all U.S. Christian sermons, it offers a window into what many Americans hear each week from the pulpit. (See this report for details about how the analysis was conducted.)
How we did this
This report used computational methods to measure whether – and how often – pastors discuss abortion in their sermons. To arrive at these findings, Pew Research Center harnessed computational tools to identify, catalogue and analyze 49,719 sermons shared online or livestreamed by more than 6,431 U.S. churches. These sermons were delivered between April 7 and June 1, 2019. Some findings using these data were originally published in the report “The Digital Pulpit: A Nationwide Analysis of Online Sermons.” To find these sermons, researchers used a computer program that navigated through U.S. church websites in search of the sermons they shared online – these sermons were then downloaded, transcribed to text (if necessary) and analyzed. The congregations were found using Google Places, a close relative of Google Maps. For more information, see the original report’s methodology.
To determine what share of sermons discussed abortion, researchers used workers on Mechanical Turk to label 250-word-long segments of sermons for whether they did or did not mention abortion. Researchers labelled segments, rather than whole sermons, because whole sermons are generally too long for an individual worker to read in one sitting. Researchers then used these data to train a machine learning model, which determined whether the remaining segments mentioned abortion. Segments were then reaggregated to determine whether sermons mentioned abortion or not. For more information, see this post’s Methodology.
The findings suggest that roughly one-in-five congregations in the evangelical Protestant, historically black Protestant and Catholic traditions heard at least one message that touched on abortion during the study period. Meanwhile, only one-in-ten mainline Protestant congregations heard about abortion during the same period.
Sermons that mention abortion rarely focus entirely on the topic. When sermons are broken into smaller segments of 250 words (the median sermon runs 5,502 words), three-quarters of all sermons that mention abortion do so in just one segment. As a result, only 1% of all sermons across the whole database discuss abortion in more than one segment.
Congregations with fewer members tend to hear more references to abortion than those with larger memberships. For instance, 13% of mainline Protestant congregations with 200 or fewer members heard at least one reference to abortion over the eight-week study period. But that share falls to 8% among mainline Protestant congregations with more than 200 members.
In online sermons that do reference abortion, the pastors’ perspectives are almost unanimously in opposition to the practice. When researchers individually examined 60 sermons that discussed abortion, only four seemed to support access to abortion services. As a result, sermons that express a positive view of abortion rights were too rare to identify across the whole database.
Across religious traditions, clergy use distinctive terms in sermons that discuss abortion
Clergy in each major Christian tradition tend to use certain words disproportionately in sermons that mention abortion, compared with sermons by clergy in the same tradition that do not mention the topic. For instance, evangelical sermons that mention abortion are more likely than those that do not to include words like “womb,” “heartbeat” and “pornography.”
[of]
One evangelical pastor argued in favor of a focus on life both before and after birth, saying, “I like what somebody said recently: ‘from the womb to the tomb.’ It’s not just enough to care about children in the womb, but to care about all life all the way to the grave.”
Meanwhile, a mainline Protestant pastor argued that religious leaders had shied away from speaking against what he perceived as social ills: “Why go to church? They don’t really condemn anything. … Should we take a strong stand against sin, against abortion, against the gay agenda, against divorce, against the love of money, against pornography, against drugs, against hate?”
Sermons that discuss abortion also are more likely to mention specific books of the Old Testament (also known as the Hebrew Bible) by name, even as they reference the New Testament (Christian biblical texts originally written in Greek) at roughly the same rate as other sermons. Indeed, 72% of all sermons that mention abortion reference a book of the Old Testament, compared with 60% of sermons not mentioning abortion.
Correction: This post has been updated to reflect the fact that 60% of sermons not mentioning abortion reference the Old Testament.
Methodology
To collect the sermons analyzed in this study, researchers followed a process outlined here. To determine whether each sermon contained a reference to abortion, researchers segmented each sermon into 250-word snippets, then hired workers on Amazon’s Mechanical Turk to examine them. Each segment was independently coded by five separate workers, and in order to label a sermon as mentioning abortion, researchers required at least three of the five to identify it as such (this threshold was chosen because it maximized agreement between Mechanical Turk workers and researchers on a set of 151 documents). The Cohen’s Kappa – a measure of inter-coder reliability – between Mechanical Turk workers and researchers was 0.96.
The documents coded by workers were oversampled based on the presence of the following key phrases: “abortion,” “fetus,” “pregnant,” “unborn,” and “womb.” Many documents that contained these phrases mentioned abortion, a fact that allowed the model to learn other key phrases in addition to the ones used to oversample documents. All model statistics are “weighted,” meaning that they adjust for this oversampling.
This process generated 3,151 labeled segments, 675 of which mentioned abortion and 2,476 of which did not. Researchers then trained an extreme gradient boosting classifier on those labeled segments, which performed well: Precision was 0.89 and recall was 0.92 when calculated using fivefold cross-validation. Researchers also calculated Cohen’s Kappa between the model’s decisions and those of human judges by comparing its performance on the 151 segments that were coded in house. The Kappa value was 0.90, indicating that the model largely agreed with human decisions. This model was then applied to 250-word segments of every sermon in the database, which were then aggregated back up to the sermon level for analysis.
