Methodology

This study, “Nonprofit journalism: a growing if fragile part of the U.S. news system,” involved three major research components. The first was thorough landscape audit to identify nonprofit news outlets launched since 1987; the second was a detailed analysis of the websites of nonprofit news organizations; the third was a survey of editors and managers of those news organizations.

Landscape Audit

The first element of this study was to indentify landscape of the field of digital, nonprofit news outlets around the U.S. launched between 1987 and 2012.

To do this, researchers took several steps to get as close as possible to the total number of digital nonprofit news outlets in the U.S. In April 2012, researchers aggregated lists and databases of digital news operations. A total of nine such lists yielded 1,810 nonprofit news websites.

Lists and databases consulted for sample definition:

J-Lab Knight Community News Network Directory of Community News Sites: 1,266 sites.
Michele McLellan’s list of promising local news sites: 145 sites.
Nieman Journalism Lab Encyclo: 109 sites.
Columbia Journalism Review Guide to Online News Startups: 100 sites.
American University Investigative Reporting Workshop: 76 sites.
Investigative News Network member list: 61 sites.
Project for Excellence in Journalism interactive guide: 46 sites.
Harvard’s Hauser Center for Nonprofit Organizations list of nonprofit news orgs: 39 sites.
Statehouse News Network: 7 sites.

Next, researchers performed exploratory research on the web to search for any remaining non-profit outlets that had not been picked up by any of the above lists. This yielded an additional 31 sites, for a total of 1,849 sites.

Researchers then eliminated duplicate listings and commercial/for-profit operations, as well as websites that may have been noncommercial, but not formally affiliated with a 501c organization.

In addition, the following criteria were implemented to further define the sample:

Outlets must be domestic. They may cover international affairs, but must be based in the U.S. or a U.S. territory.
Outlets must be active, though they do not need to be publishing new content on a regular basis to be included in the study. Outlets that, at the time of the audit, had not published new material in the previous 90 days were eliminated.
Outlets must be young. The study included outlets launched in the previous 25 years, or between 1987 and 2012. This rule was established to guide the focus of the study toward a newer generation of digital-first and digital-native news outlets, though there are reputable legacy nonprofit news institutions that exist outside the purview of this study, such as the Associated Press, Texas Monthly and Mother Jones magazine, as well as public broadcasting. (An exception-the Center for Investigative Reporting-was included in the website analysis even though it was launched in 1977. CIR’s identity and finances are intertwined with two younger outlets, California Watch and Bay Citizen, and thus eliminating CIR would have made it difficult to study the latter two outlets.)
Outlets must be primarily digital. While some outlets in the study may produce a print newsletter, or create video content that is distributed on television networks, to be included, they must be primarily a digital operation.
Outlets must produce original reporting. Websites based entirely on aggregation or purely opinion content were eliminated.
Outlets sponsored by an institution must produce journalism that focuses beyond its own walls. There are nonprofit organizations such as churches, universities or advocacy groups that also publish a news service, but one that reports exclusively on the activities of the parent organization. Such outlets were removed from the study.

This process produced a refined universe of 172 nonprofit news outlets.

Recognizing there could be a few sites that were missed even in the exhaustive steps listed above, researchers took an additional step to gauge whether the universe we had identified was relatively comprehensive. This step was akin to snowball sampling-a method used in sociology to identify members of an unknown population by first contacting known members who assist in identifying others who are unknown to the researcher.

In October 2012, researchers identified 23 nonprofit news outlets-five from each region of the country and three focused on national or international subject matter. Outlets with the biggest monthly web traffic (using data from Compete.com) were selected for contact. These 23 outlets were contacted by researchers asking them to name any additional nonprofit news outlets that they were aware of in their state, or in the case of national-level outlets, around the country. Of those 23, 10 responded. Most of those listed outlets that had already been identified by Pew Research. Two of them identified a total of four outlets that had not been identified by researchers, one of which upon inspection was commercial in nature, and another had not yet launched.

The results of all these measures suggest that the Pew Research sample is thorough and robust, even if it is not 100% exhaustive.

Website audit

Once the universe was identified, three trained researchers conducted analysis of each of the 172 nonprofit news websites selected for study. The ‘audit’ was conducted between May and July of 2012, as well as in September for additional sites identified by Pew Research. All data were then re-evaluated and updated as needed in April of 2013.

Researchers evaluated the websites based on the following series of variables:

Nonprofit tax arrangement. This variable allowed research to identify whether the outlet is an independent nonprofit (501c3 in most cases), or a subsidiary of another institution such as a foundation or think tank. Researchers did further analysis using the Guidestar nonprofit database to verify information that was not clear from the outlet’s website.
Year launched. Researchers determined the year of launch by searching the ‘about’ section of an outlet’s website, supplemented by additional web searches and in some cases, using survey data for outlets that participated.
Editorial focus. Researchers determined the primary type of content an outlet produced by evaluating how the outlet describes its own work in its mission statement or ‘about’ section. If the focus of the site was still unclear, researchers would make a determination based on scanning the content on the outlet’s home page.
Geographic focus. This variable denotes an outlet’s reach, such as hyperlocal, metropolitan, state or beyond. Researchers deduced the geographic focus from the outlet’s name and/or its ‘about’ section or mission statement.
Content production. This variable assesses how much content an outlet produced in a 14-day time span. Researchers tallied (within various range categories)the number of original, reported straight news stories; the number of long-form investigative or enterprise pieces; the number of non-reported opinion pieces; and the number of non-text products such as videos and databases.

The results of this analysis were tabulated and discussed in the section of the report titled “The Landscape of Nonprofit Journalism.”

Survey of nonprofit executives and editors

The survey of executives, directors and editors of nonprofit news organizations was conducted from September 21, 2012-November 17, 2012.

Survey administration

The administration and data coding of the survey was handled by Princeton Survey Research Associates International (PSRAI). The survey was sent to all of the 172 nonprofit news outlets identified for the study. Of those, representatives at 93 of the outlets completed the survey, a response rate of 54%. The target sample was contacted several times to ensure maximum participation.

The schedule of survey administration was as follows:

An invitation e-mail message was sent to the sample on September 21, 2012, which included a link to the online survey questionnaire.
A first reminder e-mail was sent on September 27. Additional reminder messages were sent on October 4, 11 and 17.
Pew researchers sent e-mail messages and phone calls to the remaining unresponsive outlets, requesting participation.
Survey was taken offline on November 17, 2012.

In the end, representatives from 93 of the organizations completed the survey. Researchers compared the audit characteristics of the 93 responding organizations with the full 172. On the vast majority of measures, the responding organizations were very similar to the full population. Differences were observed on three characteristics. Compared with the full population of 172 sites, somewhat fewer of the responding organizations were sponsored by non-news organizations, fewer focused on state level news, and fewer focused on news about government.

Questionnaire design

The questionnaire was designed by the Pew Research Center in consultation with the staff at PSRAI. All respondents were granted anonymity in their responses to survey questions.

The questions in the survey were designed to solicit a mixture of hard financial data from the nonprofit news outlets, as well as attitudes about financial health. The survey targets were also asked questions about their staffing and operations, and about their organizational mission. (See the complete topline questionnaire here.)