U.S. Election Models Have the Same Flaws as in 2016

(Bloomberg Opinion) -- One of the major stories of the 2016 Presidential election was how Donald Trump won despite most quantitative analysts giving him anywhere from a 1% to a 15% chance in the weeks leading up to the election. Today the Economist has Trump at a 7% chance. Other sites are more generous, but I can’t find any statistical models giving Trump even a 20% chance. Betting markets are still around 40%, as is the estimate from Professor Philip Tetlock’s Good Judgment Project, which I consider the most reliable available.

I raised an objection before the 2016 election to the quantitative models that was similar to points made by Nassim Taleb, Dhruv Madeka and others. It’s a subtle statistical point, but an important one, and we’re seeing the same flaws in 2020. The day-to-day moves in probability are too small to support a 47 percentage-point drop in Trump’s probability of winning since mid-March, to 7% from 54%, at the same time the actual day-to-day moves in the models are implausibly large. Trump’s probability of winning has to be closer to 50% than the quantitative models suggest.

It’s easier to see the points in the context of a casino. Suppose you’re the manager and one of your floor people reports that a customer is ahead $60,000. You ask to see the series of bets. If the customer won a single big bet, say betting $2,000 on the number five in roulette, there’s nothing suspicious. And if the customer made a series of big bets, say 50 hands of blackjack at $10,000 per hand, it’s also easy to be up $60,000 with a little luck.

But suppose the customer made 137 separate $1,000 bets at the craps table on the shooter making a point of five, and won 81 of them instead of the expected 55. The odds of that are less than one in 200,000, and you should look hard. The bets are too small and the odds too short to have any reasonable chance of getting ahead by $60,000.

But this is what the Economist says Joe Biden did. I’m not trying to pick on the Economist; I see the same pattern in all major daily probability forecasts of the election. It’s just that the Economist was kind enough to supply me with a spreadsheet of their daily predictions, and also the entire computer code and data for the model. Biden’s probability of winning the Presidential election went up 81 out of 137 days, and when it did go up it went up an average of 1.09 percentage points. When it went down it went down only an average of 0.73 percentage point, similar to the payouts on the craps bet. This is strong evidence the model has some bias. It’s not necessarily a bias in favor of Biden; it could be a bias in favor of whoever is ahead in the election, or something else.

In order to accept a 47 percentage-point decline in probability for Trump since March, I would need to see daily probability moves about three times as big as the ones shown in the Economist model. If the craps bettor made $3,000 bets instead of $1,000, his $60,000 gain could be normal good luck. But the daily probability moves in the Economist model are  far too large. It’s wildly implausible that the election probabilities move around 1 percentage point per day in Spring and early Summer.

Consider when information comes out about the 2020 election result. If we go back, say, 20 years, everyone would have to say the odds of the Republican candidate winning in 2020 were close to 50%. There wasn’t enough information to move far from those odds. When the actual electoral votes are cast on Dec. 14, 2020, we’ll know the result, and the probability will be either 0% or 100%. So there will be a 50% change in probability over the span of 20 years.

Much of the information about the 2020 election was already out by March 1, 2020, when the Economist starting running its model. We were pretty sure who the candidates would be, and we knew their records and the general tenors of their platforms. A lot more information is going to come between now and the election in November, with debates, Vice Presidential picks, conventions, ads, campaign events, polls and news. Finally, a lot of information will be revealed on election day itself.

For a rough guess we might say 30% of the information was known before March 1, 30% will come out between now and November, and 30% on election day. That leaves only 10% of all the information about the election to come out between March 1 and today, and I think that’s generous. The daily probability changes in the Economist model suggests that 30% of all information about the 2020 election outcome was revealed between March 1 and today.

To take the Economist probability estimate seriously, I would have to see daily probability changes three times as big as they have been, but I would only believe daily probability changes one-third as big. So I don’t believe the model. At a rough guess, its total move is about four times too big, so the 47 percentage-point drop for Trump should be more like 12 percentage points, putting Trump’s chances at 42%, which would bring the model more into line with betting markets and Tetlock’s Good Judgment Project.

The major forecasting models take different approaches. The Economist uses a machine-learning algorithm with Bayesian updating and considers the historical question: how often has a candidate won in the past given the current poll numbers, favorability ratings, incumbency status and economic indicators?

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Aaron Brown is a former managing director and head of financial market research at AQR Capital Management. He is the author of "The Poker Face of Wall Street." He may have a stake in the areas he writes about.

©2020 Bloomberg L.P.