National Statistics exaggerate the UK’s crisis

This is a complex story about how new methodology can deliver improved but very misleading statistics. The data might be “better” – perhaps even the best in a technical sense – but if the result is a distorted view of the country, users are not well served. I doubt they are, but if the UK’s GDP data is comparable with that from other countries, the nation’s public services are in a huge mess.  

I was fortunate to have an article published on Tortoise Media on 17 December: “What’s bad? Our public services or the data?“. It started: “Emerging GDP data in Britain is telling us one of two things – and both are shocking. Either the response of Britain’s public services to the Covid-19 crisis has been by far the worst in Europe. Or our GDP data, the main measure we have of tracking the economy, is distorted relative to many other countries.”

At the time of writing, Eurostat (see the screen shots below of the main GVA data page) showed that public sector activity in the UK fell by 23 per cent in the year to the end of the second quarter of 2020. A large fall was expected as that was when the pandemic took hold and lockdown arrived. Except, across the rest of Europe, only France and Hungary recorded falls of more than 10 per cent over the same period. Belgium, Italy and the Netherlands – also hit badly by Covid-19 – each fell by around 8 or 9 per cent. And Spain’s output was unchanged. This makes no sense. For it to make sense, the response of the UK’s education and health sectors would have to have been very bad, comparatively.

Alternatively the UK numbers could be compiled in a different way – more on this below – rendering them not comparable. This is despite, of course, all countries supposedly having their GDP production harmonised.

The case for the UK having different methods starts a decade ago, when poor management and relocation out of London harmed the data collection and compilation activities of ONS, leading to errors. The Bean Review, published in 2016, was a response to this mess and led to many changes. The effort to get things “right” in recent years may have meant that planned improvements to the measurement of public sector output – in essence, to rely more on outputs (such as operations in hospital) than inputs (such as staff employed by hospitals) – to have been implemented faster or more assiduously in the UK than in many other countries.

That’s good but leaves the UK figures no longer comparable with most other countries. Indeed, if a country uses purely inputs (money spent) to measure health output, it might well have risen in 2020 as additional funds were thrown at the problem. Indeed, the majority of European countries are showing higher output now than a year ago.

The impact on aggregate GDP of the UK’s “better”, but lower, numbers could be considerable. The ONS figures suggest that the fall in public sector output accounted for over 6 percentage points of the 22 per cent fall in total GDP in the first half of 2020. In other words, over a quarter of the UK’s reported fall in GDP in the first half of this year has, in effect, been attributed to closing schools and stopping hospital operations. This might be accurate, but had the ONS assumed that health and education output was unchanged in the first half of 2020, on the grounds that inputs – staff employed – were unchanged, the fall in UK GDP would have been 16 per cent, not 22 per cent. That’s still a large fall but no longer the worst when those shaming league tables are produced.

This problem – and it is a major problem – has been brewing since August when ONS published its first estimates of GDP data. There were some extraordinarily odd numbers in it, especially the deflators, but they’ve been picked up by very few commentators. Mike Haynes for UK in a changing Europe was one, along with the solid Julian Jessop and Shaun Richards. The ONS published some sketchy commentary and blogs, for example, this one from August but they do little more than hint at a (minor) statistical issue. Mainstream commentary (excluding David Smith) seems to have been absent (as has the ONS for months) but now that comparable data is emerging on the Eurostat site the issue will surely return.

More third quarter data will be released by ONS on 22 December but many countries (including the UK) have already posted provisional data on the Eurostat site. The UK’s output in Q3 is reported as 15% down on a year before. The next “worst” countries are Finland and Belgium recording a decline of 3% over a year. Some 20 countries are now showing a rise. The UK is a real outlier, but is it the data or the performance of the public services – or dodgy data from every other EU country?

The core questions remain and will become more pressing in the weeks ahead. Consider :

  1. Was it right of ONS to introduce new untested methodology into key data at such a critical time?
  2. How reliable are the new indicators that now drive GDP trends in the UK?
  3. To what extent did other countries follow suit in adopting new measures?
  4. Were sufficient warnings given by ONS about the distortion to UK GDP figures, that (probably) rendered them no longer comparable with other countries?
  5. Does the statistics regulator (OSR) think these GDP numbers are worthy of National Statistics status? (Surely they should be experimental to use their terminology.)
  6. Does the ONS think that some/most/all other countries produced sub-standard data by not incorporating accurate methodology? (They could choose to do this to hide the true fall in GDP as a result of Covid-19.)
  7. Or, if all the statistics are fine and comparable, the huge public policy question arises – why did the UK’s public services perform so very badly relative to other countries? What does the NHS, DHSC and DfE think about that and do they agree?
  8. More broadly, is this another major governance failing by UKSA? Is this a repeat of the 2010 RPI fiasco with the ONS statisticians naively rushing into untested changes without due consideration of the impact on users?

I have written to OSR (the statistics regulator) to seek their opinions on these issues. This blog will be updated when answers arrive. ** PLEASE NOTE THAT I DID GET A RESPONSE FROM OSR AND BLOGGED ABOUT IT ON 31 JANUARY 2021 **


ONS data:

GDP quarterly national accounts, UK: April to June 2020” 30 September 2020

Note tables B1 and B2 (see screen shot). See the large falls in “Health and social work”.

And table AA that shows the “contributions to growth” of the government sector.

GDP first quarterly estimate, UK: July to September 2020” 12 November 2020

Table B2 (see below) shows a rise in the first Q3 estimates.

GDP quarterly national accounts, UK: July to September 2020” 22 December 2020 – next release

Eurostat data

The relevant public administration sector data is readily available.

3 thoughts on “National Statistics exaggerate the UK’s crisis

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s