Monday, July 29, 2024

Why You Can't Trust Political Polls

Historians generally trace presidential polling to the 1824 election.  A straw poll conducted by the Harrisburg Pennsylvanian newspaper predicted Andrew Jackson would win.  In what would become an all too familiar outcome, the poll was inaccurate. John Quincy Adams was elected president. 

Since that oops moment, there have been a procession of presidential polls that have spectacularly failed. In 1936, The Literary Digest polled its two million subscribers and concluded Republican Alf Landon would triumph over incumbent President Franklin D. Roosevelt.  Oh-Oh.

In 1948, the prestigious Gallup Poll reported that Thomas Dewey would beat President Harry Truman. Gallup predicted the margin of the vote would be 45% for Dewey and 41% for Truman.  The poll secured its place in infamy as Truman won 50% of the vote compared to Dewey's 45%.

A more recent whoops moment occurred in 2016 when polls showed Donald Trump trailing Hillary Clinton in 2016.  Two poll modelers put her chances at 99%.  Trump's stunning win left Clinton to claim the Russians had influenced the outcome. Hillary's problem was putting too much stock in polls.

The 2016 election has been the subject of an analysis by the Berkeley Haas School of Business at the University of California-Berkeley.  Their study found a steady decline the in accuracy of early polls.  Only 60% proved to be accurate including those conducted up to 10 weeks before the election.

Their analysis of 1,400 polls from 11 election cycles found the outcome lands within the poll results only about half the time. The Berkeley Haas study documents many reasons the election outcome could be different from polls, including the way pollsters compute confidence levels in their results. 

Confidence levels only take into account a sampling error, a statistical term that quantifies deviations from polling large voting populations. But Berkeley Haas concludes that it does not include other kinds of error, such as surveying the wrong set of voters.  As a result, there is more opportunity for errors.

Nonpartisan Pew Research Center has studied polling in depth for decades, shedding light on presidential surveys.  Pew has researched surveys from Gallup, Fox News, Associated Press and others, that conduct polls by telephone or online from randomly selected samples of adults. 

Pew documented the influence of party affiliation in national polls. There are 7% more registered Democrats than Republicans. Pollsters generally attempt adjust their data to compensate for this disparity. Surveying more Democrats would reflect a bias in results.

Pew researchers admit that is no single "correct" adjustment to the ratio of Democrats to Republicans for national polls.  Pollsters use their own modification, which explains why there can be differences between national polls.  The absence of a standard ratio explains why polls are often wrong.

As Pew points out, there is also a bias in people who register to vote. Compared with the public in general, registered voters tend to be older, wealthier, more likely to be non-Hispanic whites and homeowners, according to Pew.  

"Evidence suggests that the Democratic advantage is somewhat narrower among registered voters than the general public--and often even narrower among actual voters," Pew found.  What this means is that polls often have a sampling prejudice that tilts toward an oversampling of Democrats.

Democratic likely voters are also clustered geographically, more so than Republicans. Since national polls are designed to reflect geographic population centers where the majority of likely voters are located, polling will underrepresent the candidate preferences of Republicans. 

Sampling for "likely voters" is also less science than the polling organizations confess.  Pollsters screen for registered voters on the assumption they will cast ballots.  However, Pew has reported that in each election there are a myriad of factors that determine whether registered voters turnout.  

In some elections, Democrats have outperformed the 7% advantage in registered voters.  In others, robust Republican turnout has erased the registration margin. Turnout is the most difficult number to calculate yet it is most critical factor in determining the election outcome. 

Polling methodologies are also subject to variances.  Different polls may have sampling errors, different weighting practices for Democrats and Republicans, variations in the wording of questions and differences in the survey mode--whether by telephone or online, notes Pew. 

One flaw never mentioned in news coverage about poll results is telephone surveys, a staple of many polls including the Gallup Poll.  An estimated 73% of Americans, including most under the age of 40, do not have a landline telephone.  Those with landlines skew older, distorting results.

Pew reveals that all national polls use weighted data rather than raw data.  In other words, the actual survey numbers (raw data) may show one presidential candidate leading by five percentage points.  However, the data is adjusted to reflect the general population's age, race, gender and region. 

Most Americans have no idea that the polling numbers they are reading are subject to so many alterations, which have the potential to influence the outcome of the polling. 

Pew Research's extensive analysis of national polling uncovered another prevalent defect in surveys.  Pollsters often claim their polls have a three percent margin of error.  Pew found the real margin of error is often double the one reported.  That makes a huge difference in closely contested battleground states. 

Remember national polls reflect voter preferences from a sampling of states.  However, U.S. presidents are not selected by popular vote.  The tally of votes in the Electoral College determine the presidential outcome.  That makes national polls an unreliable predictor of the final Electoral College vote. 

State by state polls might potentially paint a true picture of  the electoral outcome.  However, those polls are often conducted by newspapers or state organizations. There are wide disparities in the quality of methodologies used at the state levels, including those by professional polling organizations.

Despite these cautions, polls are already popping up reflecting the head-to-head matchup between Vice President Kamala Harris and former President Trump.  In one poll, the election is a dead heat.  There are more than 100 days until the election.  As noted, early polls are notoriously wrong.

These polls are fodder for campaigns and the media.  But they are practically useless as predictors of the outcome of the election.  So much has happened in just the last few weeks--an assassination attempt on Trump; President Biden bowing out; and, the coronation of Harris as the Democrat nominee. 

The public has not had time to digest all these developments to form an opinion about voting in November.  The race is just beginning. And convulsions in the political landscape are likely to jolt the campaigns, including a Manhattan judge's ruling on presidential immunity.  

Even with overwhelming evidence, America's pollsters remain in denial about the accuracy of their predictions.  They owe Americans the truth about how the polls are conducted, including a breakdown of respondents age, gender, party affiliation and geographic representation. 

If pollsters election forecasts go south this election, they will rush in with a clever spin.  Their revisionist narratives will assert their polling was misrepresented or they were within the margin of error.  Don't fall for their phony excuses.  Put your faith in election returns not polls.   

No comments:

Post a Comment