Methodology

Ipsos delivers affordable, statistically valid online research through KnowledgePanel®. KnowledgePanel is the first and largest online research panel that is representative of the entire U.S. population. Panel members are randomly recruited through probability-based sampling, and households are provided with access to the Internet and hardware if needed.

Ipsos recruits panel members using address-based sampling (ABS) methods to ensure full coverage of all households in the nation. Once household members are recruited for the panel and assigned to a study sample, they are notified by email for survey taking, or panelists can visit their online member page for survey taking (instead of being contacted by telephone or postal mail). This allows surveys to be fielded quickly and economically. In addition, this approach reduces the burden placed on respondents, since email notification is less intrusive than telephone calls and the self-administered mode minimizes social desirability bias and positivity effects that can be present with an interviewer. Many respondents find answering online questionnaires more interesting and engaging than being questioned by a telephone interviewer. Furthermore, respondents have the convenience to choose what day and time to complete their assigned survey.

KnowledgePanel methodology

KnowledgePanel provides probability-based samples with an “organic” representation of the study population for measurement of public opinions, attitudes and behaviors. The panel was first developed in 1999 by Knowledge Networks, an Ipsos company. Panel members are randomly selected so that survey results can properly represent the U.S. population with a measurable level of accuracy and a calculable response rate, features that are not obtainable from nonprobability or opt-in online panels.

KnowledgePanel’s recruitment process was originally based exclusively on a national random-digit dial (RDD) sampling methodology. In 2009, in light of the growing proportion of cellphone-only households, Ipsos migrated to an ABS recruitment methodology via the U.S. Postal Service’s Delivery Sequence File. ABS not only improves population coverage, but also provides a more effective means for recruiting hard-to-reach individuals, such as cellphone-only households, non-internet households, young adults and persons of color. Households without an internet connection are provided with a web-enabled device and free internet service.

After initially accepting the invitation to join the panel, participants are asked to complete a short demographic survey (the initial Core Profile Survey); answers to this survey allow efficient panel sampling and weighting for future surveys. Upon completing the Core Profile Survey, participants become active panel members. All panel members are provided privacy and confidentiality protections.

Adults from sampled households are invited to join KnowledgePanel through a series of mailings, including an initial invitation letter, a reminder postcard and a subsequent follow-up letter. Moreover, telephone refusal-conversion calls are made to nonresponding households for which a telephone number could be matched to a physical address. Invited households can join the panel by:

Completing and mailing back a paper form in a postage-paid envelope
Calling a toll-free hotline phone number maintained by Ipsos
Going to a designated Ipsos website and completing the recruitment form online

KnowledgePanel LatinoSM recruitment

In 2008, KnowledgePanel Latino^SM was developed to provide researchers with the capability to conduct representative online surveys with U.S. Hispanics, including both English- and Spanish-dominant Hispanics. With the advent of KnowledgePanel Latino, the first U.S. online panel representative of Hispanics was established to include those without internet access and those who only speak Spanish. Hispanic members recruited through KnowledgePanel’s traditional ABS sampling methodology described above are supplemented with recruitment using a custom dual-frame RDD sampling methodology targeting telephone exchanges associated with census blocks that have a 65% or greater Latino population density (this density level covers just over 50% of the U.S. Hispanic population). Moreover, cellular numbers from rates centers with high concentration of Hispanics are also used to improve the representation of samples. With this telephone recruitment, households are screened in the Spanish language to only recruit those homes where Spanish is spoken at least half the time.

Household member recruitment

During the initial recruitment survey, all household members are enumerated. Following enumeration, attempts are made to recruit every household member who is at least 13 years old to participate in KnowledgePanel surveys. For household members ages 13 to 17, consent is collected from the parents or the legal guardian during the initial recruitment interview. No direct communication with teenagers is attempted before obtaining parental consent.

Survey sampling from KnowledgePanel

Once panel members are recruited and profiled by completing KnowledgePanel’s Core Profile Survey, they become eligible for selection for client surveys. Typically, specific survey samples are based on an equal probability selection method (EPSEM) for general population surveys. Customized stratified random sampling based on “profile” data can also be implemented as required by the study design. Profile data can also be used when a survey calls for pre-screening – that is, members are drawn from a subsample of the panel, such as females, Republicans, grocery shoppers, etc. (This can reduce screening costs, particularly for lower incidence subgroups.) In such cases, Ipsos ensures that all subsequent survey samples drawn that week are selected in such a way as to result in a sample that remains representative of the population distributions.

As detailed above, significant resources and infrastructure are devoted to the recruitment process for KnowledgePanel so that its active panel members can properly represent the adult population of the U.S. This representation is achieved not only with respect to a broad set of geodemographic indicators, but also for hard-to-reach adults (such as those without internet access or Spanish-language-dominant Hispanics) who are recruited in proper proportions. Consequently, the raw distribution of KnowledgePanel mirrors that of the U.S. adults fairly closely, barring occasional disparities that emerge for certain subgroups due to differential recruitment and attrition.

For selection of general population samples from KnowledgePanel, a patented methodology has been developed such that samples from the panel behave as EPSEM samples. Briefly, this methodology starts by weighting the pool of active members to the geodemographic benchmarks secured from a combination of the U.S. Census Bureau’s American Community Survey (ACS) and the latest March supplement of the Census Bureau’s Current Population Survey (CPS) along several dimensions. Typically, the geodemographic dimensions used for weighting the entire KnowledgePanel include the following dimensions, with additional nesting of dimensions as well:

Gender (male/female)
Age (18-29, 30-44, 45-59, and 60+)
Race/Hispanic ethnicity (White/non-Hispanic, Black/non-Hispanic, Other or 2+ races/non-Hispanic, Hispanic)
Education (less than high school, high school, some college, bachelor and beyond)
Census Region (Northeast, Midwest, South, West)
Household income (under $10K, $10K to <$25K, $25K to <$50K, $50K to <$75K, $75K to <$100K, $100K to <$150K, and $150K+)
Home ownership status (own, rent/other)
Household size (1, 2, 3, 4+)
Metropolitan area (yes, no)
Hispanic origin (Mexican, Puerto Rican, Cuban, other, non-Hispanic)
Language dominance (non-Hispanic and English dominant, bilingual, and Spanish dominant Hispanic) when survey is administered in both English and Spanish

Using the resulting weights as measures of size, a probability-proportional-to-size (PPS) procedure is used to select study specific samples. It is the application of this PPS methodology with the imposed size measures that produces demographically balanced and representative samples that behave as EPSEM. Moreover, in instances where a study design requires any form of oversampling of certain subgroups, such departures from an EPSEM design are accounted for by adjusting the design weights in reference to the census benchmarks for the population of interest.

Survey administration

Once assigned to a survey, members receive a notification email letting them know there is a new survey available for them to complete. This email notification contains a link that sends them to the survey. No login name or password is required. The field period depends on the client’s needs and can range anywhere from a few hours to several weeks.

Typically, after three days, automatic email reminders are sent to all nonresponding panel members in the sample. Additional email reminders are sent and custom reminder schedules are set up as needed. To assist panel members with their survey taking, each individual has a personalized member portal listing all assigned surveys that have yet to be completed.

Ipsos also operates an ongoing modest incentive program to encourage participation and create member loyalty. The incentive program includes special raffles and sweepstakes with both cash rewards and other prizes to be won. On average, panel members complete three to four surveys per month with durations of about 10 to 15 minutes per survey. An additional incentive is usually provided for longer surveys.

Response rates

As a member of the American Association of Public Opinion Research (AAPOR), Ipsos follows the AAPOR standards for response rate reporting. While the AAPOR standards were established for single survey administrations and not for multistage panel surveys, Ipsos uses the Callegaro-DiSogra (2008) algorithms for calculating KnowledgePanel survey response rates.

Omnibus survey completion rates

The field period and completion and qualification rates for this survey are presented below:

Ipsos KnowledgePanel weighting

Study-specific post-stratification weights

Once all survey data have been collected and processed, design weights are adjusted to account for any differential nonresponse that may have occurred. Depending on the specific target population for a given study, geodemographic distributions for the corresponding population are obtained from the CPS, the U.S. Census Bureau’s American Community Survey (ACS), or in certain instances from the weighted KnowledgePanel profile data. For this purpose, an iterative proportional fitting (raking) procedure is used to produce the final weights. In the final step, calculated weights are examined to identify and, if necessary, trim outliers at the extreme upper and lower tails of the weight distribution. The resulting weights are then scaled to aggregate to the total sample size of all eligible respondents.

For this study, the following benchmark distributions of U.S. adults age 18 and over from the 2022 CPS were used for the raking adjustment of weights:

Gender (male, female) by Age (18-29, 30-44, 45-59, and 60+)
Race/ethnicity (White/other non-Hispanic, Black non-Hispanic, Hispanic)
Education (less than high school or high school graduate, some college, bachelor and beyond)
Census region (Northeast, Midwest, South, West)
Household income (under $25K, $25K to <$50K, $50K to <$75K, $75K to <$100k, $100K to <$150K, and $150K+)

The margin of sampling error is plus or minus 3.2 percentage points at the 95% confidence level, for results based on the entire sample of adults. The margin of sampling error takes into account the design effect, which was 1.05. The margin of sampling error is higher and varies for results based on subsamples. In reporting of the findings, percentage points are rounded off to the nearest whole number. As a result, percentages in a given table column may total slightly higher or lower than 100%. In questions that permit multiple responses, columns may total substantially more than 100%, depending on the number of different responses offered by each respondent.