Analysis

COVID-19: Four different death tolls, but scale of tragedy depends on which numbers you choose

There are various different ways of measuring the impact of the pandemic, one of the worst periods for mortality in modern history - but the scale of the tragedy depends on which number you choose and what you are counting.

Please use Chrome browser for a more accessible video player

Ed Conway looks at how the number of excess deaths compares to years before the pandemic
Why you can trust Sky News

Let's begin with a multiple choice question.

In the year 2021, a year in which the UK faced wave after wave of COVID-19, was mortality in England and Wales:

a) similar to the 2020 toll?

b) less bad than in 2020?

c) on a par with a severe flu epidemic or extremely cold winter?

d) one of the worst years in a century?

Before we get to the answer, let's go through some of the numbers. There are, you may recall, various different ways of measuring the mortality impact of COVID-19.

More from UK

Going through this data is not an especially enjoyable task, but thankfully we have statisticians and actuaries who deal with such things on a regular basis.

And since we now have pretty much all the data we need to look at how 2021 compared with 2020, this feels like an opportune moment.

Four different death tolls

The numbers being produced by these statisticians and actuaries can sometimes look very different - consider the total death toll from COVID-19. There are actually at least four different, legitimate tolls.

The first is the official toll you hear about each day from the government, which it gets from the UK Health Security Agency. Technically speaking, this number counts anyone who has died within 28 days of a positive test as a COVID death. You probably heard it recently surpassed 150,000 people.

However, the Office for National Statistics has an entirely different estimate for the death toll. Its measure counts any death certificates where COVID is mentioned as a factor. This number now stands at 175,000.

Now, not all of these deaths are directly due to COVID. However, the ONS reckons that roughly 90% of them tend to be (with the remaining 10% being people who died primarily from other causes but with COVID-19 as an additional factor). On that basis, that suggests the overall toll is around 157,000.

Then there's another measure altogether, one which eschews the vagaries of diagnoses and PCR tests. This is to look at the total number of deaths across the country (from all causes) and compare this number to a "normal" period, which typically means the previous five-year average.

Excess deaths a 'more definitive measure'

This number - excess deaths - is often considered a more definitive measure of mortality over a given period and over the pandemic it is running at 151,000 - very close to the official death toll.

However, the complexity doesn't end there because there are also some shortcomings to excess death statistics too. One problem is that such numbers make no adjustment for the fact that as each year goes on this country gets bigger and its population ages - and the older and bigger a country's population, all else equal the more deaths one can expect in any given year - pandemic or not.

This is why actuaries prefer to age- and population-adjust this number, to come up with a standardised mortality rate.

This adjustment actually makes quite a big difference. For instance, age-adjusted excess deaths during the pandemic are not 151,000 but 120,000. And while the latest ONS numbers suggest quite high levels of excess deaths, there's good reason to believe this is in part because the numbers are not age-adjusted: do this adjustment and many of those apparent excess deaths fall away.

Please use Chrome browser for a more accessible video player

Countries should plan for several surges of COVID a year, says WHO's David Nabarro

Each death toll differs

You see the issue here: one person's death toll can look quite different from another's. Except that pretty much every measure of mortality looks unusually high right now - at least compared with recent history.

So, back to that question at the start. Let's take the first multiple-choice answers: was 2021 similar or worse than 2020? It depends on which dataset you're looking at. If you're looking at the official death toll courtesy of the government dashboard, there were 74,000 deaths in 2021, which is not far short of 2020 (76,000).

But now let's look at age-adjusted excess deaths, as calculated by the Institute and Faculty of Actuaries, and you get a very different picture: 47,000 deaths in 2021 versus 73,000 in 2020.

So it turns out both a) and b) are correct - depending on which numbers you're looking at.

Pandemic deaths by year: Official toll against age-adjusted excess deaths
Image: Pandemic deaths by year: Official toll against age-adjusted excess deaths

Why the difference?

The most likely explanation is that the official government toll missed a lot of deaths in the early period of the pandemic; these deaths, many of them in care homes, were picked up by the excess death measure. In 2021 excess deaths were a lot lower than the official toll, in part because there was lower generalised mortality from other causes, such as flu.

How about c) and d)? Surely they couldn't both be correct too?

Well, let's have a look back through time, at changes in the age- and population-adjusted standardised mortality rate.

There is no definitive prism for this. Like gross domestic product, age-adjusted mortality tends to improve every year (or, to put it another way, people live a bit longer than they would have), so comparing absolute levels only tells you so much.

Image: This has been one of the worst periods for mortality in modern history

For instance, in 2021 about 1% of the population died (adjusted for age and population growth). That compares to 1.06% in 2020 and 0.94% in 2019. But, as recently as 2015, the mortality rate was 1.01% (in other words worse than 2021) and in 2008 it was 1.12%. So technically, mortality during the pandemic was actually lower than only a few years ago.

Except that this is not an especially meaningful statistic when you consider that such deteriorations are very, very rare. It's a little like saying we shouldn't pay any attention to recessions because, say, the absolute level of the economy following a recession like the one we had in 2008/9 only dropped back to the level it was a few years ago. It's technically true, but what matters here is not so much the level but the change.

The previous five-year average

So a better way of comparing mortality over time is to look at how the mortality rate compares to the previous five-year average.

Here we start to see that the current pandemic sticks out like a sore thumb. In 2020, the standardised mortality rate was 8.3% higher than in the preceding five years. That was the single biggest annual increase since 1940 - a year defined by WWII and the Blitz.

But how about 2021? In that year mortality increased by 1.8%. How unusual is that? Put it this way: in the past 50 years there were, save for this pandemic period, only two years when the mortality rate exceeded the previous five year average: in 2015 when it rose 0.4% and 1976 when it rose by 0.3%.

So while 2021 was less bad than 2020, it was still worse than any other year going back a long time. That said, there were a few other years in the post-war period where mortality increased more: the terrible winter of 1963 where temperatures dropped to the lowest levels in 200 years (2.5%) and the 1951 flu pandemic (8%), for instance.

So: on this basis 2021 was very bad, but not quite as bad as the 1951 flu or the blizzards of 1963. Except that, again, we're oversimplifying. For remember, the average against which 2021 is being measured is the previous five years; and since those five years include 2020, a very bad year for mortality, that means this number probably understates the comparable abnormality of death levels in 2021.

For instance, compare last year with the 2015-19 period and the increase in mortality is closer to 2.7%. So answer c) is sort of correct.

But still, it seems odd to look at 2021 in isolation given it followed such a severe year for deaths, so if we want a view of the death toll with more perspective we might do better to look at both 2020 and 2021, and here the verdict is quite clear.

Take an average of the mortality rate over these two years and we have never seen as big a deterioration since 1940/41.

And you have to go back to WWI and the Spanish Flu to find another period of two or more years which were all far worse the preceding five-year average.

All answers correct

By now you've probably twigged that all of the answers are correct.

Follow the Daily podcast on Apple Podcasts, Google Podcasts, Spotify, Spreaker.

This has been one of the worst periods for mortality in modern history, but the scale of the tragedy depends on which numbers you choose.

But no numbers can do justice to the hundreds of thousands of tales of personal loss and misery across the country.