Reasons to Wonder
Model Based Projections
So, even with all of the above described difficulties, how have the pandemic modelers performed in practice? Although there ate many models from which to choose, I’ll limit this discussion to those that appear to have been most influential to the formation of public policy.
The Imperial College Model
This model has been by far the most influential in the setting of government policy to fight COVID-19.
British scientist Neil Ferguson ignited the world’s drastic response to the novel Wuhan coronavirus when he published the bombshell report predicting 2.2 million Americans and more than half a million Brits would be killed.
Based on these terrifying predictions the United States and the United Kingdom, among many other countries have virtually shut down their economies. Tens of millions of people are losing their jobs. Tens of thousands of businesses may never reopen due to loss of revenue and financial barriers to reopening.
So, how has this model’s accuracy done as the pandemic played out. Let’s led the good doctor Ferguson, the model’s primary author explain (emphasis added).
But after tens of thousands of restaurants, bars, and businesses closed, Ferguson is now retracting his modeling, saying he feels “reasonably confident” our health care system can cope when the predicted peak of the epidemic arrives in a few weeks. Testifying before the U.K.’s parliamentary select committee on science and technology on Wednesday, Ferguson said he now predicts U.K. deaths from the disease will not exceed 20,000, and could be much lower.
That’s at least a 25-to-1 error in predicted deaths in the U.K.! Some have attempted to lessen the blow of this modeling fiasco by claiming the change is due to aggressive government policy. However this answer falls flat.
Former New York Times reporter Alex Borenson called Ferguson’s revised predictions a “remarkable turn,” …
“The UK only began its lockdown 2 days ago, and the theory is that lockdowns take 2 weeks or more to work,” Borenson said. “Not surprisingly, this testimony has received no attention in the US – I found it only in UK papers. Team Apocalypse is not interested.”
This is a stunning recantation of results that had driven governments across the world to choose highly aggressive policies. It is an example of gross professional incompetence at the very least. Dr. Ferguson and his team should suffer a complete loss in credibility and professional standing. However, as certified “elites” the likelihood of any real accountability for the massive failure is remote.
But it’s worse than that. In point of fact there already is a history for the performance of these models in predicting real-world infection results…and it’s very poor.
But the Times goes on to ask an important question that I have not previously seen raised: do disease modelers have a track record that deserves to inspire the remarkable confidence that is being reposed in them?
… and it was from Imperial that Ferguson and Anderson dominated the government response to foot and mouth.
That response, involving the slaughter of more than 11 million sheep and cattle at a cost of more than £8bn was based entirely on modelling and remains hugely controversial — with many believing the modellers got it wrong. They were modelling a fast-moving epidemic with little accurate data. A subsequent government inquiry was damning of the general approach and its conclusions may be relevant to the current crisis. It said: “The FMD epidemic in UK in 2001 was the first situation in which models were developed in the ‘heat’ of an epidemic and used to guide control policy . . . analyses of the field data, suggest that the culling policy may not have been necessary to control the epidemic, as was suggested by the models produced within the first month of the epidemic. If so it must be concluded that the models supporting this decision were inherently invalid.”
The Imperial modellers’ next big public challenge came eight years later when swine flu swept the world — fortunately killing few Britons because older people tended to be immune and younger ones were strong enough to fight it off. Britain was, however, left with 34 million doses of unused and expensive vaccines. Again there was an inquiry — which concluded that ministers had once again treated modellers as “astrologers”, asking them to provide detailed forecasts when they had too little data.
“Modelling did not provide early answers,” it concluded. “The major difficulty with producing accurate models was the lack of a relatively accurate idea of the total number of cases . . . This is not to reject the use of models, but to understand their limitations: modellers are not ‘court astrologers’.”
The tragic fact is that we had no valid reason to base our social and public health policies on these modelers and their models. The argument could be made that, at the very beginning of a novel form of pandemic, we had little choice but to use these crude, unreliable tools because nothing else was available (but I have my doubts). However, the argument cannot be made that, after having exposed the stunning failure of these modelers and their models we are bound to continue following then into ruin.