By Meredith Dost and Kyley McGeeney
Each year about 36 million Americans move residences, according to the Census Bureau. And they quite often take their cellphone numbers with them. Others have not moved but bought their cellphone in a different state. The net result, according to new Pew Research Center estimates, is that 10% of U.S. adults have a cellphone number that doesn’t match the state where they actually live. For example, a potential survey respondent may live in Maryland but have a cellphone with a New York area code. For urban dwellers, four-in-ten have a number that doesn’t match the city where they live.
This trend has become a headache for researchers doing state- and local-level telephone polling. While some technical solutions are in the works, there is presently no surefire way to capture people with out-of-state or out-of-area numbers in sub-national polls. At the state level, the trend is most pronounced in Washington, D.C., where 55% of residents with a cellphone have a number from outside the District. In some parts of the country, by contrast, this is almost a nonissue. In Michigan, for example, just 5% of adults with cellphones have an out-of-state number.
These results are based on an analysis of 30 national political surveys conducted by Pew Research Center from 2013 to 2015. The pooled data consist of more than 32,000 cellphone respondents1 including nearly 18,000 respondents who do not have a landline in their home. To determine the share of geographic mismatch, researchers compared respondents’ self-reported ZIP code with the location associated with their cellphone number. A cellphone’s location is based on its area code, exchange and thousand-block, e.g. 917-593-2xxx, which are assigned according to the location where the phone was purchased. It was necessary to go back to 2013 to amass enough data to report results at the state level. This may be an underestimation of this phenomenon, especially as more people acquire cellphones and as it becomes the norm to keep them for a longer period of time and retain the same number when moving.
Determining how big of a problem out-of-area cellphone numbers are for pollsters entirely depends on the geographic population pollsters are trying to target. For national polls, including nearly all those conducted by Pew Research Center, it is not a problem. By dialing nationwide, researchers reach the target population of U.S. adults regardless of any internal mismatch across cities and states. If the goal is then to report survey findings by geography, researchers use respondents’ self-reported locations based on ZIP code. But for pollsters conducting surveys at the state or local level, especially in areas where the mismatch rate is high, out-of-area cellphone numbers are a serious concern because self-reported location is typically not available when selecting the sample of numbers to call.
In state-level polling, researchers typically draw a sample of phone numbers from a state in order to survey its residents. If people living there have a cellphone number from a different state, they won’t be included in that sample. This phenomenon is known as under-coverage. Excluding a group of people, especially one that’s shown to be demographically distinct, has the potential to bias survey estimates. It’s therefore important to understand just how large the under-coverage rate is, as well as the characteristics of who is excluded, in order to assess its potential effect on the final data.
Nearly half of adults living in urban areas have a cellphone number from elsewhere
Trying to infer where someone lives based on their cellphone number is clearly subject to error, and this error increases as the geographic unit of analysis gets smaller. For regional analysis (e.g., Northeast, South, Midwest, West), this is a minor concern; the new analysis finds that the cellphone number’s region and self-reported region match for 95% of adults, similar to findings in a 2009 Pew Research Center analysis.
At the state level, the geographic accuracy rate tends to be lower. Overall, about 10% of adults who live in a particular state have a cellphone number associated with a different state. Even more striking is the fact that, at the metropolitan level, four-in-ten adults have a number from outside the metropolitan area in which they live.
Unfortunately for pollsters, the mismatch rates are somewhat higher for respondents who are cellphone-only – those without a landline telephone in their home – than for cellphone respondents who have a landline. This is a problem because cellphone-only individuals can only be reached via their cellphone, whereas people who also have a landline can be reached on their landline even if their cellphone is excluded from the sample due to a geographic mismatch. At the regional level, 93% of cellphone-only respondents report living in the region that matches their sample information, compared with 96% of respondents who also have a landline. A similar pattern exists for state and metropolitan-level geographic data. The remainder of this analysis focuses on all cellphone respondents regardless of landline telephone ownership.
Share of adults with out-of-area cellphone numbers varies widely by state
The mismatch between where people live and their cellphone number location has the potential to be an issue in state polls, and the risk to pollsters is greater in some states than in others. In states such as Michigan, Ohio, Iowa and California the mismatch rate is 5% to 7%. By contrast, Washington, D.C. (55%), Maryland and New Hampshire (21% each) all have relatively high shares of cellphone users with out-of-state numbers.
The flip side of this mismatching is also problematic for pollsters. If a pollster attempts to conduct a cellphone random-digit-dial (RDD) survey in Maryland, for example, about one-in-five respondents (18%) will report living elsewhere. Such interviews are typically treated as ineligible for the survey, wasting interviewer time and increasing the survey cost. This phenomenon, sometimes called over-coverage, can be measured as the proportion of adults with a cellphone number associated with a given state but who live elsewhere. Washington, D.C. (61%), New Hampshire (25%) and Hawaii (21%) all have relatively high rates of over-coverage. In other words, approximately six-in-ten cellphone respondents with D.C. cellphone numbers actually live in a different state.
Geographically mismatched respondents younger, more highly educated
Cellphone respondents whose cellphone number does not match the state where they live (i.e., mismatched) are demographically distinct from cellphone respondents whose phone number and state of residence match. Mismatched respondents are younger, more highly educated and have higher incomes than matched respondents. A majority (55%) of those with mismatched geography are college graduates, compared with 38% of respondents who currently live in the state associated with their cellphone number.
The mismatched respondents are also more likely to be non-Hispanic whites and religiously unaffiliated and are less likely to be married and to be parents of children under age 18. Just 20% of those with mismatched states are parents of under-18 children, while about three-in-ten adults (31%) whose sample and self-reported states match report being parents. These findings are generally consistent with research by other research teams studying the characteristics of adults with out-of-area cellphone numbers.2
Strategies to include people with out-of-area numbers in telephone polls
Several strategies are available for including people with out-of-area numbers in telephone polls, but all of them have limitations. The most straightforward approach is to sample cellphone numbers from surrounding geographies when conducting a poll of a small area, such as a city or even a state. For instance, a telephone survey of District of Columbia residents would likely need to sample Maryland and Virginia telephone numbers; however, people living in the District with cellphone numbers from California would still not be captured in the cellphone sample, and D.C. phone numbers belonging to individuals now living in California would need to be screened out.
In a previous report, Pew Research Center discussed alternative strategies that employ new products telephone sample vendors offer to address this issue. For example, it is now possible to design a telephone sample using the billing address associated with a cellphone number. That helps to improve accuracy in many cases, but it is not a perfect solution. Billing address information is still not available for a large fraction of cellphone numbers, and even when it is available, it is sometimes not indicative of where the user actually lives. For example, the billing address for a college student included on a family plan would reflect a parent’s location, not the student’s.
Notably, pollsters who draw their samples from voter files rather than RDD frames do not have this problem. Polls built off voter files are based on the residential address on file for each person. Telephone numbers, including those of cellphones, are included in those files, but the sampling is based on the residential address reported by the adult when they registered to vote. While voter file samples are good option for pollsters looking to survey voters, it is not clear that they are a good option for organizations like Pew Research Center that study the attitudes and experiences of all U.S. adults, regardless of registration status.
