Guided by the Numerology

numerology noun: the use of numbers to try to tell somebody what will happen in the future [Oxford learners dictionary online]

Of all of the miserable contortions and distortions wrought by the covid-19 pandemic, there are at least three that are sufficient to cause Dr No to come back from retirement. The first two have to do with the numbers, and the third has to do with the government response to those numbers. This post covers the first of those three concerns, and still to come posts will cover the other two concerns.

If there is one phrase that has shouldered the burden for justifying the government’s reaction to covid-19, it is surely that the government has above all and throughout been guided by the science. This sounds sensible and reassuring, on a par with a doctor who, as he recommends a painful and debilitating treatment, says he is doing so guided by the science. The problem is that, while the doctor may have some more or less robust science tucked up his white coat sleeve, the government has not. Instead they have some numerology, dressed as science. They have what long term readers of Bad Medicine will know is one of Dr No’s gravest bête noires of pseudo-science, the anything goes Alice in Wonderland world of computer modelling.

Real science is about observing the world, coming up with ideas, and testing them. We might observe that older children appear to be taller than younger children, and come up with the idea that as children get older, they grow, and so become taller. We can then test this hypothesis by taking a group of young children and measuring their height at intervals over time. When we have enough data, we can plot the changes in height over time, and measure the rate of growth by drawing a line that fits best through the points on the chart. Had we only measured their heights on birthdays, we can even use that line to estimate their heights at intermediate ages, by the process known as interpolation, and be reasonably sure our estimates are reliable. So far, so good. We are still in the realms of science, but even at this early stage there are important caveats.

Suppose our group of children was a sample from a Scandinavian country, where on average people tend to be taller. Could we use our data to predict the heights of children in Asia, where people tend to be shorter? Of course not, because our sample of the Scandinavian children is not representative of Asian children. Were we to make predictions guided by the science about the heights of Asian children, we would soon find we were wrong, despite being guided by the science.

So we need to be sure that when we use data from one group — say residents of Wuhan, where air quality it terrible, or Northern Italy, where the air quality is also terrible, and the population is elderly to boot — are representative of the group — say UK residents — where the air quality may on average be better, and the age distribution somewhere in the middle of our other two samples — to which we wish to apply our predictions. If they are not, we need to adjust our predictions to allow for the differences, and so long as we do this adjustment robustly, we are still just about in the realms of science.

Any such comfort very rapidly departs the moment we start to extend our ruled line beyond the data, even beyond the edges of our chart, into the unknown. If, as in our study of the heights of children, we failed to measure the heights of adults, we would fail to spot that children stop growing after a certain age, and so our predictions, based on growth rates in children, would become increasingly wild the further we extrapolated. Instead of wizened old men, we would quite wrongly predict octogenarian giants.

Yet this is exactly what computer modelling does, and then some. It extends the line into the unknown, and then, as it that wasn’t enough, it ‘tests’ the predictions by asking ‘what-if’ this or that happens. It is about as scientific as trying to predict the outcome of North Korean missile tests. You tilt the launch pad a bit this way, and see where the missile ends up. Whoops, just taken out our industrial heartland. Tilt it the other way, whoops, sayonara Japan. Put in a more powerful propellant, and whoops, mid air explosion, or a dud propellant, and whoops, nice try but no cigar. Or tweak the design so it goes faster, only the tweak also causes a boomerang flight, such that the missile comes back and bites you on the bum.

The essential point is that computer modelling is always a flight into the unknown, hugely dependent on those initial ‘what if’ assumptions. It is not science, it is a forecast, and the further we extend the forecast, the more uncertain it becomes, the wilder the predictions made by the model. This should come as no surprise, because we are all very familiar with the mother of all computer models, the Met Office weather forecast. Even with their staggering collection of data and super computers, the forecasters struggle to make predictions that stick for more than a few days.

So next time you hear politicians or government experts telling you they are guided by the science, always and only ever guided by the science, be aware that sometimes they are being economical with the truth. The guidance may have its roots in science, but it has since been transformed by the magical numerology of computer modelling into a forecast, and a forecast, however interesting it may be, is only ever a forecast.