How focus groups informed our study about nationalism and international engagement in the U.S. and UK

Pew Research Center surveys regularly reveal divergent opinions about globalization, national identity and international engagement, both in the United States and other countries. Political affiliation and ideological orientation help explain some of these differences in opinion, but we wanted to explore more fully how local context and national identity also shape opinions about globalization.

To do this, we designed a cross-national, comparative qualitative research project. We recruited 232 people to participate in 26 focus groups in seven cities in the United States and United Kingdom. We asked participants about their views of local, national and international issues. Unlike in our surveys — where respondents typically answer closed-ended questions — we asked open-ended questions, allowing participants to discuss their responses in more depth. And being in a focus group allowed participants to engage with one another and the topics posed. After the focus groups concluded, we developed a strategy to systematically analyze verbatim transcripts by theme. We then assembled a matrix of distinct, yet sometimes intersecting, frames of reference that people employed when discussing the nature and consequences of globalization, national identity and international engagement. In addition to forming the basis of our analysis for a data essay published today, the findings of the focus groups helped guide us in developing survey questions which will be fielded soon in the U.S. and UK, as well as other countries.

In this post, we’ll explain how we developed and carried out this research project, and what we learned. To navigate directly to the different components of this analysis, you can use the links below:

Designing the project

From the start, we planned to use the focus group findings to inform future opinion surveys. Our research design aimed to depart from the structured interview format of quantitative surveys, while maintaining a systematic, rigorous approach to analyzing the focus group transcripts. The number and composition of focus groups in the U.S. and the UK were intended to exhaust the range of views on globalization, international engagement and national identity — providing us with enough disparate opinions about these topics from people in different walks of life, geographies and more that we achieved a “saturation” of sorts. Saturation is a difficult standard to demonstrate; however, we can report that over the course of the study we did observe an increasing repetition of themes, if not experiences, and gradually fewer views that could be described as novel or innovative. This suggests that the research design did, in fact, achieve a reasonably robust degree of saturation.

Building our focus groups

We drew on extensive background research and expert consultations to organize focus groups in the U.S. and UK using similar but distinct criteria and variables, such as party affiliation, ideology, vote in the referendum to leave or remain in the European Union, and 2016 U.S. presidential vote. Focus group composition in each country was designed to maximize variability — e.g., across different political ideologies — while creating optimal conditions for open, active engagement by participants — e.g., speaking with “leavers” in one group and “remainers” in another. Importantly, we found that people with certain characteristics in common were more likely to feel comfortable speaking openly about their opinions.

United States

In the U.S., research indicates that partisanship plays a role in how people feel about the themes we wanted to explore, but that the experience of being a Republican in a primarily “blue state” might vary from that of a Republican in a “red state.” As a result, we selected three cities — Seattle, Houston and Pittsburgh — in states that can generally be described as Democratic-leaning (Washington), Republican-leaning (Texas) or a “battleground” (Pennsylvania).

Our research also suggested that Americans’ views about globalization and immigration differ by their community type — rural, suburban or urban — as well as by their race and ethnicity. But we’ve learned from our past focus groups on this topic that it’s important to create homogeneous racial and ethnic groups so as to minimize participants’ discomfort speaking about sensitive issues and encourage more open sharing of personal experiences and perspectives. To that end, we created one focus group with each of three distinct racial and ethnic communities: a group composed of Asian and Pacific Islander Americans in Seattle, one comprised of Hispanic Americans in Houston and one consisting of Black Americans in Pittsburgh. All other groups were comprised of White Americans exclusively.

Taking all of these factors into account, we organized a total of four focus groups in each city: 1) a group of White Republicans and Republican-leaning independents living in rural areas surrounding each city; 2) a group of suburban White independents; 3) a group of ethnic minority independents who hailed from suburban or urban locations; and 4) a group of White urban Democrats and Democratic-leaning independents.

As an additional “screen,” or question we used to help sort participants into particular groups, we asked every participant a previously used survey question that related to our topic of interest: whether America’s openness to people from all over the world is “essential to who we are as a nation,” or ”makes us risk our identity as a nation.” We wanted to ensure that everyone had an opinion on this question and that Groups 1 and 4, as described above, were of like mind about it (the former saying America risks losing its identity, the latter saying openness is essential) and that Groups 2 and 3 had some variation.

United Kingdom

In the UK, we again focused on geography and ideology, but the Brexit vote also played a key role. In particular, we wanted to include two cities that largely voted to remain in the EU and two that largely voted to leave, while also capturing the diversity of views among people living in those areas.

We chose London and Edinburgh because they voted resoundingly to remain in the EU — and Edinburgh, specifically, to understand the Scottish perspective and explore the attitudes of those who do and do not support Scottish independence. We chose Birmingham and Newcastle because both voted to leave the EU and were historic manufacturing hubs, while Newcastle also offered a northern perspective.

Using a seven-point left-right ideological orientation scale, we created four focus groups in each of the three cities in England: 1) those on the ideological right who voted to leave the EU; 2) those on the ideological left who voted to leave; 3) those on the ideological right who voted to remain; and 4) those who on the ideological left who voted to remain. In addition, all of the ideologically left-leaning focus groups included at least two participants from racial and ethnic minority groups, while the right-leaning focus groups were entirely White Britons.

Participants were also asked to what degree they identified with their local area, their city, with England or Scotland (depending on the location of the group), with the UK, as European, or as a global citizen, each using a 10-point scale, where 0 meant not at all and 10 meant very strong identification. People were then recruited into different groups based on whether they gave higher values to being European or a global citizen or whether they gave higher values to local areas or their country.

Other variables

The variables we considered when putting these groups together are not the only ones that affect attitudes toward globalization and national identity. Things like education, gender, employment and age are also prisms through which views about these issues can be forged, and while these were not necessarily the primary variables we considered when putting our groups together, we did consider them.

Some of the groups had specific education cut-offs. In the U.S., no one in the White rural Republican groups attended college, while all other groups were comprised of a mix of educational attainment. And in the UK, different groups in different cities had different educational cut-offs. For example, the “remain”-voting groups in London and Newcastle all had an undergraduate degree or less education, while the participants in the “remain”-voting groups in Birmingham all completed undergraduate degrees or pursued graduate degrees. We took a similar approach for the “leave”-voting groups, with those in London and Newcastle having undergraduate degrees or more education and those in Birmingham having undergraduate degrees or less education. And in Scotland, the people we spoke with who supported Scottish independence had less education, while those who favored staying in the UK had more. We did this to examine how education might affect attitudes within each type of group.

With regard to gender and age, we made sure that all groups were roughly balanced, with about half of participants men and half women (there were no gender nonbinary or non-conforming individuals in our sample), and that all groups had a mix of adults ages 18 and older.

Developing a discussion guide

Our discussion guide — essentially a list of topics we planned to cover — included questions that aimed to explore our research interests and prompt conversations that examined the impacts of globalization and national identity at the local, national and international levels.

Since our screening questions already identified whether people were more nationally or internationally oriented, we included questions designed to uncover why people felt as they did, the crux of qualitative research. We wanted to see how they described and reacted to change affecting their community, their country and the world.

Asking about the local effects of globalization

At the local level, we asked the following questions:

Describe where you live.
How has your neighborhood changed?
Is your neighborhood closely knit?
Are there opportunities for people where you live?
How would you describe the kind of place where you’d like to live?

Asking about the national effects of globalization

At the national level, we asked the following questions (not all questions included in the focus group moderator guides were analyzed in the final report, but analyses of some of these data may be released at a later date):

What does it mean to be American/British?
What things make you proud to be American/British?
What makes you embarrassed or ashamed to be American/British?
When was the U.S./Britain at its best?
Can people from other cultures be American/British?
Is what it means to be American/British the same today as it was in the past?
To what extent to you feel at home in the U.S./Britain today?
Where is a quintessentially American/British place?
Who is a patriot/someone with a strong British identity?
Who is a cosmopolitan?
Is there common ground between patriots/people with strong British identities and cosmopolitans?
What are the issues facing our country today?

Asking about the international effects of globalization

At the international level, we asked the following questions:

What is globalization?
Has our country benefited from globalization?
Who or what has benefited in particular?
What are the biggest problems with individual countries working together?
Is our country open or closed to people from other countries?
What would be the impact of our country having fewer connections with other countries?

Analyzing the data

Once the focus groups wrapped up, we were left with 26 roughly 40-page transcripts, totaling over 1,000 pages of text. And even though our researchers had attended the focus groups, debriefed about them with moderators and had access to the transcripts, we sought to develop a method to identify key themes in a systematic way without needing to pore over more than a thousand pages.

To do this, we used a multistep method of data reduction. First, a team of researchers entered short-form text from the transcripts into a spreadsheet, or data display. In this display, the discussion guide questions were columns and individual participants were rows. Researchers entered short-form text deemed relevant to each discussion guide question into a cell. This short-form text was either a direct quote or paraphrased statement that we considered to directly answer the questions the participant was responding to, related to the key themes we initially set out to study or related to new themes we identified. If a respondent did not discuss the topic, we left the cell blank. We created a separate column of “coder’s notes” for researchers to note any patterns or themes within the group. We also bolded and color-coded paraphrased text or direct quotes that captured the themes that occurred across multiple questions.

Quality control

We conducted a number of quality control checks during this data entry phase. To ensure the essence of each data reduction spreadsheet captured the same general information, each researcher initially coded the same transcript. We noted and discussed any discrepancies (such as omitted key quotations, missing responses by participants, insufficiently condensed text, or significant paraphrases or direct quotations that were not bolded and/or color-coded). After coding the same transcripts, each researcher then proceeded to enter data onto a separate spreadsheet for each group on their own. Each researcher had at least two of their displays checked for errors by another researcher. No researcher entered data for more than eight transcripts.

After we entered all the transcript data, we combined the spreadsheets for each group into one “master” display for each country. At this stage, we completed an additional quality control check to ensure that the data in each cell properly aligned with others (i.e., that each column contained only data relevant to the discussion guide question “variable” header).

Creating ‘toplines’

After the quality assurance measures had been finalized, a team of three researchers further reduced the data displays into outlines or summaries, akin to the “toplines” we use to summarize quantitative data. Each researcher was assigned a section of the focus group instrument — either local, national or global — and outlined the key themes identified under each question in that section. They also noted any differences in the themes that came up between groups or between certain demographics within groups (e.g., groups consisting of people who voted to leave the EU were less likely to see multiculturalism in a positive light). These toplines were also careful to note when certain questions were not asked in all groups to indicate that there are limitations to the findings from these questions.

Key takeaways

We believe this qualitative research project was effective at exploring and understanding some of the key frames of reference people employ when answering questions about globalization, international engagement and national identity. Over the course of 26 focus groups, our guided discussions gradually achieved a high level of “saturation,” leading us to feel confident with the depth and breadth of the study’s findings. These findings, in turn, suggest new ways for us to ask about and engage survey respondents on topics related to the experience and consequences of increasing global interconnectedness. For example, we learned a great deal about how ideas of political correctness, national history and law and order factor into how people conceptualize their national identities. Without conducting these groups, we may not have known to ask questions on these topics when exploring people’s views of globalization.

But in addition to helping us better conceptualize how to ask questions about the topics we explored, we also learned about why people felt certain ways — adding a richness to our bread-and-butter quantitative work.

However, we also recognize the challenges of qualitative research. The process of developing this project, collecting the data, cleaning the data and conducting the analysis was extremely time consuming for our team. While we can typically write a survey and identify a sample, conduct fieldwork, and analyze our data within the span of less than a year (and sometimes, within less than six months), this project was over a year in the making.

In addition, when we conduct a survey, we have processes in place to manage its progress, from questionnaire development to reporting. But for this project, we were creating new systems. If we pursue this kind of project again, we now have a stronger foundation to build on, in addition to some new strategies we explored, such as using a QDAS, that may help truncate our timeline. Beyond the time costs, there are monetary costs to doing enough focus groups to feel like one has reached “saturation” and heard about perspectives from enough different types of people, especially if one is interested in cross-national, comparative work.

Despite these challenges, the research team is excited about the findings from this project — both on their own and in terms of how they can inform our future survey work — and we believe they were valuable enough to justify the costs.