Don’t Like What the Virus Models Say? Try Building Your Own

(Bloomberg Opinion) -- These are the best of times for disease modelers. They have the ear of presidents and prime ministers, and are getting huge amounts of news media attention. Their work is surely having greater impact now than ever before.

These are also the worst of times for disease modelers, because they have to model the behavior not just of diseases but also of the humans that carry them. This is always tough to get right, but far tougher when presidents, prime ministers and other people are listening to the disease modelers and in some cases rapidly changing their behavior in response.

This conundrum has delivered some big forecast shifts. On March 16, the Covid-19 Response Team based at Imperial College London’s MRC Centre for Global Infectious Disease Analysis was predicting that “in the (unlikely) absence of any control measures or spontaneous changes in individual behaviour,” the disease would cause “approximately 510,000 deaths in GB and 2.2 million in the US, not accounting for the potential negative effects of health systems being overwhelmed on mortality,” a frightening forecast that was said to have had a big impact at 10 Downing Street and the White House. Nine days later, the head of the team, Imperial College epidemiologist Neil Ferguson, was telling Parliament that U.K. deaths probably wouldn’t top 20,000.

This was not the embarrassing admission of error it was made out to be by armchair epidemiologists on Twitter and in the media. Ferguson was simply saying that the worst-case scenario his team had modeled — and dubbed “unlikely” — had become even more unlikely as the government shifted policy and Britons began to take the disease more seriously, and that the best-case scenario in which social distancing efforts and case-based tracing and isolation halted the spread of the disease until a vaccine arrived had become more likely. Still, his updated mortality estimate was a little puzzling even for those of us who had read the initial report, and will almost certainly turn out to be too optimistic, given that the U.K. has already reported more than 12,000 coronavirus deaths, with the daily death toll still rising, and that’s probably a significant undercount given that it doesn’t include most deaths from the disease outside of hospitals.

The model-based forecasts of coronavirus deaths and hospital-bed needs from the Institute for Health Metrics and Evaluation at the University of Washington haven’t produced quite that much whiplash, given that the first one on March 26 already assumed a significant amount of social distancing. But they too have produced rapidly changing estimates and a lot of head-scratching from observers by shifting within weeks from predicting that U.S. hospitals would soon be overwhelmed with Covid-19 patients to estimating now that “peak resource use” has passed while, with some local exceptions, coming far short of exceeding hospital or intensive-care-unit capacity.

So one can kinda sorta see where Republican U.S. Senator John Cornyn was coming from when he tweeted last week:

The Wikipedia page that Cornyn links to says the scientific method involves formulating hypotheses, making deductions based on them, testing those deductions against “experimental” or “measurement-based” evidence, and then refining or eliminating the hypotheses. Far from being an indictment of the disease modelers, though, this describes more or less what they’ve been doing: Formulating hypotheses about the behavior of the SARS-CoV-2 virus based on what they’ve seen of it so far and what they know about other viruses, making deductions about the course of the disease based on those hypotheses and some hypotheses about human behavior, and then refining those hypotheses as new evidence comes in.

No, these aren’t the controlled experiments of laboratory science, but such modeling is probably the most scientific way to tackle an emergent phenomenon like a pandemic or, for that matter, climate change — although the disease modelers have the advantage of getting useful feedback on their forecasts much more quickly than the climate modelers usually do. The results of the Covid-19 models shouldn’t be taken as the final word. Far from it. But they offer useful on-the-fly guidance that it would be ridiculous to ignore just because the eventual reality often turns out not to match the forecasts.

To use these forecasts intelligently, I wonder if it might help if more politicians, journalists and others tried their hand at disease modeling to get a sense of how it works. This is not hard to do these days, thanks to the excellent Covid-19 Scenarios site developed by scientists at the University of Basel in Switzerland and the Karolinska Institute in Sweden. Built around a simple Susceptible-Infected-Recovered model of the sort that epidemiologists use, it allows users to tweak parameters ranging from Covid-19’s reproduction rate (R0), a measure of how many people a person with the disease can be expected to infect, to its seasonality, its fatality rate and the expected length of hospital stays, then forecasts the disease’s spread and impact based on demographic data from just about every country on earth. The site also allows one to impose interventions of varying duration and effectiveness and see what kind of impact they have, although at this point it doesn’t say what exactly those interventions are (they’re working on that).

The horrifying (and by now seemingly quite unrealistic) intervention-free baseline of the model for the U.S. is that 95% of the population gets the disease and that a little over 1% of the infected, 3.3 million people, die. That fatality rate is not the unrealistic part, with estimates from epidemiologists so far seeming to converge around 1% or a little lower. A much higher 6.2% of confirmed coronavirus cases worldwide have died, but there’s ample to reason to believe that confirmed cases represent a fraction of infections. There have been suggestions that they represent a quite tiny fraction, meaning that the disease is already widespread, the fatality rate is lower than the seasonal flu’s 0.1% and it’ll all be over soon. But local experiments with widespread or randomized testing for the disease have not backed up this view. A recent University of Bonn study of the hard-hit German town of Gangelt, for example, found that 15% of residents were either infected with the new coronavirus or had antibodies indicating that they had been. This made for a fatality rate so far of 0.37%, which is (1) several times worse than the seasonal flu, (2) possibly skewed by undercounted deaths and (3) likely to go up with the passage of time.

The fatality rate could go down thanks to new treatments, or up if hospitals are overwhelmed. But 1% seems both reasonable and useful as a starting point, in part because it makes so clear that the key variable in most forecasts is how many people would be infected. A forecast of 60,000 deaths in the U.S. from the coronavirus, the current working assumption at the White House, implies at a 1% fatality rate that only about 6 million people, or 1.8% of the population, will get the disease. (Given the greatly varying severity of the disease by age, it could also imply that more younger Americans get Covid-19 and that relatively few over 65 do, but I’m trying to keep things simple here.) The 2.2 million-death forecast in the Imperial College worst-case scenario, meanwhile, implies that about two-thirds of the U.S. population would get it, at or near the oft-cited threshold for “herd immunity” where so many people have recovered from a disease and become at least temporarily immune to it that it can no longer spread. The H1N1 pandemic of 2009-2010 stopped well short of that, though, with just under 20% of the U.S. population infected and the disease in decline even before a vaccine became widely available. That sounds encouraging, but if Covid-19 were to infect 20% of the U.S. population, the expected death toll (again assuming the age distribution of cases is the same as that of the population) would be 660,000.

My point in sharing all this morbid information is that if you want to criticize one of the model-based coronavirus forecasts as too pessimistic, or too optimistic, you kind of need to have your own forecast of how many people will be infected — and be willing to adjust it as new information comes in. Not so easy, right?

This column does not necessarily reflect the opinion of Bloomberg LP and its owners.

Justin Fox is a Bloomberg Opinion columnist covering business. He was the editorial director of Harvard Business Review and wrote for Time, Fortune and American Banker. He is the author of “The Myth of the Rational Market.”

©2020 Bloomberg L.P.