Methodology

A number of people at the Pew Research Center’s Project for Excellence in Journalism worked on Pew Research’s “The Crisis over Syria: How Al Jazeera America Tackled Its First Major Story.” Associate Director Mark Jurkowitz and Director Amy Mitchell wrote the report. Research Analyst Katerina-Eva Matsa analyzed the statistics and created the charts. Coding and data analysis were conducted by Researchers Monica Anderson, Paul Hitlin, Katerina-Eva Matsa and Nancy Vogt. Jesse Holcomb copy edited the report. Dana Page handles communications for the project.

Methodology

This special report by the Pew Research Center’s Project for Excellence in Journalism examined news media coverage of Syria from August 26 through August 31, 2013.

Data regarding the coverage of Syria in the main cable news media sample were derived from the Pew Research Center’s in-house coding operation.

Human Coding of News Media

Sample Design

The sample of mainstream cable news included the following programs from a total of 5 channels, which followed the schedule listed below.

Cable TV (14 in all, Monday-Friday)

Morning (6:00 to 7:00 a.m.) coded one hour of programming every weekday CNN Fox News MSNBC

BBC

Morning (7:00 to 8:00 a.m.) coded one hour of programming every weekday Al Jazeera

Nighttime CNN – coded one hour of the following every day

Situation Room (6 p.m.) Anderson Cooper 360

Nighttime Fox News – coded one hour of the following every day

Fox Report w/ Shepard Smith O’Reilly Factor

*On August 30, 2013 Bill O’Reilly did not host his show. Instead, Greta Van Susteren was on. Researchers still coded this segment.

Nighttime MSNBC – coded one hour of the following every day

Hardball (7 p.m.) The Rachel Maddow Show

*On August 30, 2013 the Rachel Maddow show did not air. Instead, Hardball was repeated. Therefore researchers did not code this segment.

Nighttime Al Jazeera – coded one hour of the following every day

America Tonight (9 p.m.)

Consider This (10 p.m.)

Nighttime BBC America – coded a half hour of the following every day

BBC World News America (5:30 p.m.)

Cable TV (4 in all, Saturday)

Daytime (2:00 to 3:00 p.m.) coded one hour of programming after the Obama address CNN Fox News MSNBC

Al Jazeera

Story Selection

Stories were included in the study if 50% or more of their content was on Syria. To find the relevant stories in all outlets researchers watched the full hour of programming. Then, researchers read or watched each story to see if the 50% of the story or more was on Syria. Commercials, promos, and teasers of upcoming stories were all excluded from the sample.

This process resulted in a sample of 327 stories.

Human Coding of Stories

The data in this study were created by a team of four experienced coders.

The unit of analysis for this study was the story. If a story contained two or more frames, the frame that was given the most attention (in terms of seconds) in the story was assigned.

Each story was coded for the messages it conveyed. While watching a story, coders checked all the messages that appeared about Syria. Direct and indirect quotes were counted along with assertions made by journalists themselves.

Intercoder Testing

We have conducted intercoder testing on all of the housekeeping variables (such as source, date, opinion vs. reporting and format) on numerous occasions, and reached a level of agreement of 80% or higher on each of them.

In addition to the intercoder testing conducted on all housekeeping variables, supplemental testing was conducted on the additional variables specific to this study. For the following variables, 20 randomly selected stories were coded by all members of the coding team in two rounds -10 stories were coded in the beginning of the study and 10 in the end.

The percent of agreement for each variable was as follows, which was derived from the final testing: