Election predictions can’t be proven – and that’s not a problem

12 September 2024, 07:47

As the US presidential election approaches, millions of people are turning to election forecasting models that claim to predict the outcome.

The political scientist Justin Grimmer criticized these forecasts in a Politico piece, arguing it would take hundreds of years to validate election forecasts by the usual standards of social science. He’s right; but those standards are precisely the problem.

To differentiate skilled forecasters from lucky ones, we need many predictions from each forecaster.  Imagine flipping a coin and asking me to guess heads or tails.  I could call it right once just by chance.  But if I'm right five times in a row, which should normally happen about three percent of the time, you’ll probably believe I have a way to predict the result.

This approach, called hypothesis testing, has been the scientific standard since the early 1900s.  But presidential election forecasts can’t pass a hypothesis test, because presidential elections don’t happen often enough.  Even under favourable assumptions, a forecaster who is right 75% of the time would need almost a century to make enough predictions to pass.

Hypothesis testing is a valuable, if often misused, tool for scientific research. Science can't prove things beyond all doubt, but it does require a foundation of established facts to build new knowledge on.  Those facts need to clear a high bar before we trust them as scientific truth.

Despite that, we make most of our decisions without a scientific level of certainty.  Investors can’t wait until they know which companies will be successful—by then it’s too late to invest!

In his book On the Edge, election forecaster Nate Silver describes two communities he calls the Village and the River.  The Village consists of media and political elites, who move slowly and want certainty.  The River includes venture capitalists, tech entrepreneurs, and professional gamblers, who are comfortable taking risks.

Grimmer’s piece exemplifies the Village: seeking certainty and trusting scientific institutions.  But while Silver admits the Village has its strengths, his sympathies lie firmly with the River.

We have information about the credibility of forecasts like Silver's.  We can check his track record on other questions, like Congressional forecasts; we can critique the assumptions underlying his model.  That evidence can’t establish his forecast as scientific truth, but that's okay.

You shouldn't “trust” election forecasts.  But you might want to believe them.

________________

Dr. Jay Daigle is a professor of mathematics at the George Washington University. He studies math teaching, modeling, and metascience, and writes about the ways hidden assumptions shape our decisions and beliefs.  You can find him on twitter @profjaydaigle or on his blog Maybe-Mathematical Musings at https://jaydaigle.net/blog.

LBC Views provides a platform for diverse opinions on current affairs and matters of public interest.

The views expressed are those of the authors and do not necessarily reflect the official LBC position.

To contact us email views@lbc.co.uk