Methodology

Data collection was conducted in two phases. In phase one, Pew Internet conducted two online and one in-person focus group with middle and high school teachers, as well as two in-person focus groups with students in grades 9-12. Focus group findings were instrumental in shaping the development of a 30-minute online survey, which was administered in phase two of the research to a national sample of middle and high school Advanced Placement and National Writing Project Summer Institute teachers, drawn from sample files provided by the College Board and the National Writing Project, respectively.

Phase One — Focus Groups

Focus group discussions were designed to elicit from teachers and students their perceptions of the different ways digital technologies such as the internet, search engines, social media, and cell phones are impacting and shaping students’ research and writing habits and skills. Teachers were also asked to speak in depth about their experiences teaching research and writing to middle and high school students today, any challenges they encounter, how they incorporate digital technologies into their classrooms and assignments, as well as how these technologies play a role in their professionalization.

In-person focus groups were led by two Pew Internet researchers, and were held on-site at a College Board school in the northeastern United States, immediately following school hours. Each discussion was approximately 1.5 hours in duration, and students and teachers were provided with Barnes & Noble gift cards as a token of appreciation for their participation ($25 for students, $50 for teachers). Focus group discussions were recorded and transcribed.

Two online focus groups were also held with teachers. The first utilized an existing online research community of 150 Advanced Placement teachers. A series of discussion questions or “exercises” designed by Pew Internet were administered to this group by panel administrators using an online, asynchronous platform. Panel administrators then provided Pew Internet with transcripts of responses to the exercises.

The second online teacher focus group was administered by Pew Internet staff using an online focus group platform. Participants in this group consisted of 30 National Writing Project teachers recruited by NWP site administrators. The asynchronous 3-day focus group was moderated by Pew Internet staff, and all participants were able to see and respond to one another’s comments as well as questions and comments from the moderators. Transcripts of the focus group were downloaded following completion of the discussion.

All focus group sessions were analyzed to identify key themes, and discussion guides were revised and adjusted between groups to better probe emergent themes. All focus group discussion guides are available at the end of this section.

Phase Two – Online Survey

Following completion of focus group discussions, Pew Internet designed and fielded a 30-minute online survey with a national sample of middle and high school teachers. The sample is not a probability sample of all teachers because it was not practical to assemble a sampling frame of this population. Instead, two large lists of teachers were assembled: one included 42,879 AP teachers who had agreed to allow the College Board to contact them (about one-third of all AP teachers), while the other was a list of 5,869 teachers who participated in the National Writing Project’s Summer Institute during 2007-2011. A stratified random sample of 16,721 AP teachers was drawn from the AP teacher list, based on subject taught, state, and grade level, while all members of the NWP list were included in the sample.

Email invitations containing a link to the survey and unique username and password were sent to members of the sample; teachers could not opt-in to the sample. The response rate was 12% for the AP teachers and 14% for the NWP teachers. Given the nature of the sampling frames used for the study, no margin of error has been computed.

The online survey was conducted from March 7–April 23, 2012, and was completed at least in part by 2,462 teachers (2,067 teachers completed the entire survey; partial completes were kept, and all percentages reported are based on those answering each question).

The teachers who participated in the survey

The survey results presented here are not based on a representative sample of U.S. middle and high school teachers, yet every effort was made to administer the survey to as broad a sample of educators as possible from the samples available. Overall, the 2,462 Advanced Placement and NWP Summer Institute teachers who participated in the survey comprise a fairly wide range of subject areas, experience levels, geographic regions, school type and socioeconomic level, and community type (full sample characteristics below).

The final sample includes teachers from all 50 states, Puerto Rico and the U.S. Virgin Islands. All teachers who participated in the survey teach in physical schools/classrooms, as opposed to teaching online or virtual classes. English/Language Arts teachers make up a significant portion of the sample (36%), reflecting the intentional design of the sample. Only 9% of the final sample is middle school teachers, with the other 91% reporting that they currently teach grades 9-12. While half of the teachers participating in the survey report teaching in a small city or suburb, there is fairly wide distribution across school size and students’ socioeconomic status. There is also a wide distribution in terms of teachers’ ages and experience levels.

Interpreting the results

There are several important ways the teachers who participated in the survey are unique, which should be considered when interpreting the results reported here. First, 95% of the teachers who took the survey teach in public schools, thus the findings reported here reflect that environment almost exclusively. Moreover, and perhaps more importantly, the majority of teachers in this sample (56%) teach AP, honors and accelerated courses, thus the population of middle and high school students they work with skew heavily toward the highest achievers. These accelerated classes may have resources and support at their disposal—particularly in terms of specialized training and access to digital tools—that are not available in all classrooms.

In addition, it is critical to keep in mind that almost a third of the sample (NWP teachers) has received extensive training in how to effectively teach writing in today’s digital environment. The National Writing Project’s mission is to provide professional development, resources and support to teachers to improve the teaching of writing in today’s schools. The NWP teachers included here are what the organization terms “teacher-consultants” who have attended the Summer Institute and provide local leadership to other teachers. Research has shown significant gains in the writing performance of students who are taught by these teachers.¹¹

Thus, the population of teachers participating in this research might best be considered “leading edge teachers” who are actively involved with the College Board and/or the National Writing Project and are therefore beneficiaries of resources and training not common to all teachers. It is likely that teachers in this study are developing some of the more innovative pedagogical approaches to teaching research and writing in today’s digital environment, and are incorporating classroom technology in ways that are not typical of the entire population of middle and high school teachers in the U.S.