Pew Research Center Analyzes Polling Limitations in 2016 and 2020

Image Courtesy of Pew Research Center

By Anna Harvey

Following the tumultuous election of 2020, think tanks and research institutions were left with the challenging task of analyzing the mentalities and political identities of the American electorate. With this challenge, however, came startling realizations on the difficulties in accumulating accurate data.

On April 8, the Pew Research Center released an article on the current difficulties they experience with determining consistent research with the outcomes of elections. 

“Looking at final estimates of the outcome of the 2020 U.S. presidential race, 93% of national polls overstated the Democratic candidate’s support among voters, while nearly as many (88%) did so in 2016,” the Center said.

While the 2020 election did play in favor of the Democratic candidate, the Center nevertheless confirms that the amount of Republicans participating in surveys has dramatically decreased over the past 5 years. In the article, the Center analyzes its methods to acquire survey recruits, and how to amend their efforts in order to include a proportionate amount of Republicans.

Currently, the Center conducts surveys on its online panel, the American Trends Panel (ATP). 

“The ATP is recruited offline via random national sampling of residential addresses,” the Center said. “Each survey is statistically adjusted to match national estimates for political party identification and registered voter status in addition to demographics and other benchmarks.”

Dr. John White from the Politics department further stated that research centers will not only have to reexamine their research methods, however, but that they will also need to adequately determine the political composition of the American electorate. Until then, White stated, think tanks will be unable to properly amend their survey recruitment to obtain equivalent and realistic numbers of recruits.

In particular, the article states that this understatement of GOP support in recruits may be due to several reasons. Firstly, the article suggests that Republicans, specifically Republicans who voted for Donald Trump, are ceasing to participate in online panels. 

“This trend could reflect real-world change in participation (i.e., Republicans are increasingly resistant to polling) or real-world change in party affiliation (i.e., that there is a decline in the share of the public identifying as Republican), but it might also reflect methodological changes over time in how the ATP is recruited,” the Center said.

The Center states, however, that the number of U.S. adults identifying as Republican or Republican-leaning has been consistent since 2016, and consequently there has been little to no decline in Republican affiliation between 2014-2020.

In analyzing how this may further impact the midterm elections in 2022, White posited that Trump voters may likewise decline to participate in surveys or even vote in further elections, due to the fact that Trump will not be on the ballot or from fear of others’ judgment.

“It seems as though Republicans have been even more reluctant to participate in survey research questions,” White said. “We had a lot of evidence according to focus groups post-election in 2020 from Trump voters who did not want their vote for Donald Trump to be publicized, because of fear of retaliation from family members, from neighbors, etc… Trump is not on the ballot in 2022 and may not be on the ballot in 2024, so to what extent [with Trump’s absence] on the ballot are they willing to participate and vote?”

On the other hand, the Center also stated that voters from the pro- and most anti-Trump areas of the country were also more unlikely to join the ATP. 

“Survey participation has long been linked to individuals’ levels of education and social trust,” the Center said. “Now that the GOP is doing better attracting voters with lower levels of education and, according to some analysts, doing better than in the past attracting low trust adults, Republican participation in surveys is waning, increasing reliance on weighting as a corrective.”

Several suggestions that the Center gives to achieve better representation include retiring overrepresented panelists, using a relatively high response rate survey offering mail and online responses, using more offline methods of communication potentially with incentives, and creating new recruitment materials.

White states that survey responses by mail have generally been fraught with issues, and that it will be unlikely that mail surveys will help to correct representation disparities. 

“There have always been issues with mail surveys,” White said. “Mail surveys have traditionally been used when you’re dealing with a select group of people… [and] often require follow-up.”

A second explanation the Center posits is that Republicans are becoming increasingly reticent to participating in surveys. 

“This idea is not supported by GSS or Gallup poll trends,” the Center said. “However, it is consistent with one prominent interpretation of polling errors in the 2016 and 2020 elections: that they stem from certain types of Republicans not participating in polls.”

In particular, the article states that while weighted data for Republicans has remained consistent over the last six years—staying in the 42% to 45% range—trends in unweighted data still remain a concern. This disparity in weighted versus unweighted data gives the Center further incentive to stabilize participation numbers.

“The recent increases in the size of the weighting correction on partisanship suggests that the panel would benefit from shoring up participation among harder to reach groups,” the Center said. “If that is successful, then reliance on weighting will lessen.”

Along with the challenges of weighing the data, however, White also states that a major roadblock facing researchers is analyzing the composition of the American electorate.

“[There is] a lot of calculation that goes into the composition of what the electorate looks like,” White said. “And that’s hugely important in the midterm election coming up in 2022. Midterm elections depend a lot on the composition of the electorate.”

Concerning the severity of the situation of the polling industry’s methodology of data collection, White acknowledges that it is important to correct any polling concerns.

“The reputation of the industry and any individual polling firm rests on its accuracy. So they have a lot at stake in this process, and there’s a lot of concern within the industry as to are they getting it right,” White said.  

With the 2022 midterm elections, the Pew Research Center and other polling firms will likely reveal the effectiveness of their changing polling strategies, which will help to further prepare the American electorate for the next riveting presidential election.