This article was written by Cindy Peng, Manager of the Sax Institute’s Decision Analytics team, and Dr Danielle Currie, Senior System Dynamics Modeller at the Sax Institute. It was originally published in The Mandarin.
A few short months ago, if you’d uttered the phrase “flattening the curve” in a public place, you’d have drawn blank stares. Nowadays, everyone knows it as a term used in modelling the COVID-19 outbreak, and barely a day goes past without some new pandemic modelling appearing in our news media. Modelling has a profile like it has never had before.
At the same time, you might have noticed that the models often come with the disclaimer “not for prediction”. Which may leave you wondering: what’s the point of these models if they aren’t for predicting? You may be further confused by the way researchers and the media sometimes use the term “predictive models”. What does it all mean?
Models are complicated – but they can essentially be broken down into two groups: those that are exploratory and those that are explanatory in nature. What we researchers call “predictive models” are those that explore relationships within existing data. From these relationships, we extrapolate possible future relationships. These models are exploratory in nature and driven by existing data.
The techniques we use to explore these relationships range from simple regression modelling – which is used to investigate the relationship between two variables – to advanced machine learning techniques that require highly specialised training.
One example of predictive modelling in the current pandemic is using Google searches to identify unconventional symptoms of COVID-19. Researchers found, for example, a spike in Google searches in Italy for being unable to smell, in the days before the pandemic struck that country. Weeks later, it transpired that the inability to smell was a relatively common symptom of the disease. The researchers discovered a relationship between the Google searches and a COVID-19 symptom.
Predictive models work out what is happening in a particular dataset, and “predict” that the same will happen outside the dataset, assuming the underlying mechanisms remain the same. In the above case, a spike in Google searches for the inability to smell in any country may “predict” the onset of a COVID-19 pandemic. The underlying mechanism only plays the secondary role of a safety check for us to make the prediction, i.e. whether it’s plausible that the inability to smell is linked to the onset of COVID-19.
But there are other types of models – the ones that their originators often label as “not for prediction”. These are the explanatory models, and they use underlying mechanisms as their building blocks. In other words, they begin with explicit assumptions based on our current understanding of how things work. Of course, in a situation like our current one, explicit assumptions about the COVID-19 pandemic may change, and we can adapt our models accordingly.
If we start with explicit assumptions about the pandemic, we can then ask important “what if” questions in our model. For example, we may wish to know what would happen if we implement social distancing measures at a certain time point, compared with a week later. We can then model the difference in infections or deaths, using our explicit assumptions about how the disease spreads.
This ability to ask and answer “what if” questions is what makes explanatory models such an important tool for governments. And although they may provide meaningful predictions, their real purpose is to allow decision makers to investigate alternative actions before they decide on policy.
A model developed by the Doherty Institute, for example, estimated that unless measures were taken to mitigate the effects of the pandemic, the capacity of our health system would be exceeded by 200% to 900%. Governments took note and acted. At the same time, by acting on this prediction, the future changes, and the prediction becomes “incorrect” for the right reason.
How can we know that the model is correct about the alternative futures that we avoided? To start with, the models have undergone rigorous processes to identify the best evidence available. And the details of the model are often published at the same time for transparency, so that they can be scrutinised by other experts.
In the UK, for example, a model from the Imperial College London proved decisive in the government’s decision-making. The model featured multiple scenarios based on different types and intensities of social distancing strategies; it found that strong suppression strategies would have to be maintained for 18 months or more as multiple waves of the pandemic occurred.
This finding caused huge public anxiety – and yet soon after publication of the model, other modellers pointed out some flaws in its explicit assumptions. One was that the model hadn’t accounted for the possibility of contact tracing to prevent a second wave of pandemic. Professor Neil Ferguson, who led the Imperial College London model and until recently advised the UK government in their response, acknowledged in later interviews that testing and contact tracing could be the way out of strict social distancing.
It’s worth mentioning a well-known saying in the modelling world: “All models are wrong, but some are useful”. In other words, there is no absolute correctness, but we can still be pragmatic and assess the model’s usefulness in our decision-making.
Our team at the Sax Institute are working on a number of COVID-19 models: some of them aim to inform containment of the spread of COVID-19 in the communities, and some of them aim to help manage hospital resources in this unusual time; all of them have the humble goal being a useful tool to support decision-making.
Models can be incredibly useful tools by allowing decision makers to glimpse into possible futures. Nonetheless, we would do well to remember that they aren’t crystal balls.
Read the original article (published on The Mandarin).
If you’d like to know more about the Sax Institute’s knowledge brokering service, please contact the Knowledge Exchange team.
