As a special report for PEJ’s weekly News Coverage Index (NCI), the Quarterly Report is based on the aggregated data collected from April 1, 2007 – June 29, 2007 (the 2nd quarter of 2007).
Examining the news agenda of 48 different outlets in five media sectors, including newspapers, online, network TV, cable TV, and radio, the NCI is designed to provide news consumers, journalists and researchers with hard data about what stories and topics the media are covering, the trajectories of major stories and differences among news platforms. Following a rotation system, 35 outlets each weekday were selected as well as 7 newspapers each Sunday.
For its News Index PEJ monitors 48 different news outlets each week (35 per week-day) from five different media sectors
Newspapers (Thirteen in all, Sun-Fri)
NY Times every day
Code 2 out of these 4 every day
Wash Post
LA Times
USA Today
Wall Street Journal
Code 2 out of these 4 every day
The Boston Globe
Star Tribune
Austin American-Statesman
Albuquerque Journal
Code 2 out of these 4 every day
The Sun Chronicle
Star Beacon
The Chattanooga Times Free Press
The Bakersfield Californian
Web sites (Five in all, Mon-Fri)
CNN.com
Yahoo News
MSNBC.com
Google News
AOL News
Network TV (Seven in all, Mon-Fri)
Morning shows
ABC – Good Morning America
CBS – Early Show
NBC – Today
Evening news
ABC – World News Tonight
CBS – CBS Evening News
NBC – NBC Nightly News
PBS – Newshour with Jim Lehrer
Cable TV (Fifteen in all, Mon-Fri)
Daytime (1–1:30 pm) – code 2 out of 3 every day
CNN
Fox News
MSNBC
Nighttime CNN – code 3 out of the 4 every day
Lou Dobbs Tonight
Situation Room (7 pm)
Paula Zahn Now
Anderson Cooper 360
Nighttime Fox News – code 3 out of the 4 every day
Special Report w/ Brit Hume
Fox Report w/ Shepard Smith
O’Reilly Factor
Hannity & Colmes
Nighttime MSNBC – code 2 out of the 4 every day
Tucker (6 pm)
Hardball (7 pm)
Countdown w/ Keith Olbermann
Scarborough Country
Radio (Eight in all, Mon-Fri)
Headlines every day
ABC Radio headlines at 9am and 5pm
CBS Radio headlines at 9am and 5pm
NPR Morning Edition every day
Talk Radio
Rush Limbaugh every day
1 out of 2 additional conservatives each day
Sean Hannity
Michael Savage
1 out of 2 liberals each day
Ed Schultz
Randi Rhodes
From that content, PEJ analyzes all stories with a national or international focus that appearing as follows:
- On the front page of newspapers
- In the entirety of commercial network evening newscasts.
- The first 30 minutes of network morning news, the PBS evening news, and all cable programs
- The top 5 stories on each website at the time of capture
The resulting universe of stories was coded by a team, which is made up of 8 trained coders, a coding administrator, and a senior research methodologist. The complete methodology for the weekly NCI has further details on the coding system and coder reliability.
This report aggregates the NCI from April 1, 2007- June 29, 2007. The resulting universe totals 18,010 news stories, 459 hours of broadcast content (148 hours from network TV, 224 hours from cable, 87 hours from radio), 2.16 million words in newspapers, and 1.10 million words from news websites.
The following is some additional methodology information that applies specifically to the quarterly report.
Intercoder Reliability Testing
In order to continue to assure that our coding is performed with a high level of reliability, we have performed multiple tests of intercoder agreement with all of our coders.
During the 2nd quarter of 2007 we have had 8 professional coders work on the Weekly News Index project. All were included in our intercoder testing.
Housekeeping Variables
We first conducted a test of the “housekeeping” variables of the index in April. For this test, we selected a random sample of 151 stories from each of the 5 media sectors we cover. Each story was coded by two different coders. This represented more than 10% of the number of stories we code in a given week.
Of those stories, 32 were print stories (newspaper and online) and 119 were broadcast stories (television and radio). For our housekeeping variables, we achieved the following levels of agreement:
Print (32 cases)
Story Date: 100%
Source: 100%
Story word count (+- 20 words): 90%
Placement: 94%
Broadcast (119 cases)
Story Date: 100%
Source: 100%
Broadcast start time: 100%
Headline: 100%
Story start time (+- 6 seconds): 91%
Placement: 93%
Story end time (+- 6 seconds): 92%
Main Variables
Having demonstrated that we had a high level of agreement for all of our housekeeping variables, we then had the coders participate in an additional test to determine the level of agreement for the main variables of the index.
We randomly selected 116 stories from both print and broadcast mediums, which represent about 8% of the stories we code in a typical week. The level of agreement for each of our key variables was as follows:
Format: 89%
Big Story: 91%
Sub-storyline: 87%
Geographic Focus: 91%
Topic: 85%
We will continue to conduct further tests of intercoder agreement in the coming months in order to assure continued quality in our coding processes.
Lexis-Nexis Search for Candidate Mentions
Note: The following methodology applies only to PEJ’s search for candidate names (LINK) and not to the majority of data included in this report and the weekly News Coverage Index reports. For a detailed methodology about how the weekly News Index data is compiled, go here.
The Lexis-Nexis database search was conducted for each of the three leading candidates, from both the Republican and Democratic fields; and also for Fred Thompson and Mayor Bloomberg.
The newspapers included in this search were:
New York Times
Washington Post
Los Angeles Times
USA Today
Boston Globe
Star Tribune (MN)
Chattanooga Times Free Press
Albuquerque Journal
Austin American-Statesmen
We also searched news transcripts from the three major network television stations. The transcripts for the network television stations include a number of programs that are not included in PEJ’s weekly sample. Also, Lexis has transcripts for the entire programs so some segments that do not appear in the weekly index coding would be included in this Lexis search.
The television programs included in the search were:
ABC
World News Tonight
This Week
20/20
Good Morning America
CBS
60 Minutes
CBS Evening News
The Early Show
Face the Nation
NBC
Meet the Press
NBC Nightly News
Today
PBS
NewsHour with Jim Lehrer
For both newspapers and the television transcripts, searches were conducted using the last names of the candidates that appeared in either the headline or lead paragraphs of the story. The reason for searching for the names in the headline or lead paragraphs was to determine the number of stories that focused on the candidates. Another option would have been to search for mentions anywhere in the article or transcript which would have yielded more results, but would have also included many stories where the candidates were not central to the story.
The exceptions for using the candidate’s last name were Hillary Clinton; John Edwards Fred Thompson, and Mayor Bloomberg. Because of former President Bill Clinton, a search of the name “Clinton” would yield many articles about him and not the current Senator from New York. In addition, many articles refer to the candidate as “Hillary Clinton” while others refer to her as “Hillary Rodham Clinton”. Therefore, we searched for both options in order to get the proper total number of stories. In locating stories on “John Edwards” and “Fred Thompson”, full names of the candidates were used due in part that Edwards and Thompson are both widely used last names in the US. For Mayor Bloomberg, “Mayor Bloomberg” was used as a search term to avoid including stories on Bloomberg industries or individuals with the same last name. Many articles refer to Michael Bloomberg as “Mayor Bloomberg.”
