PHE: Experts’ report fails reasonable tests

The last week has shown the importance of the PHE “Technical Briefing” notes. The 14th was published on 3 June and steered the discussion about not removing lockdown measures on 21 June. The report fails the standards that might be expected of a document presenting vital information to politicians, media and individuals. Presentationally it is unclear, uses technical language and offers no context – it is not user friendly. It’s not in a format that allows most politicians easily to make their own judgment and for the public to understand decision making. Instead, it leaves experts (real and self-proclaimed) able to cherry pick to push the messages they like. The underlying data is not available making it impossible to say if the presentation is fair. The methodology of its regressions and charting is not given, and nor is a clear assessment of the impact of data weaknesses on conclusions. As it stands it would be possible to game both the data from the collection processes (by directing PHE resources to certain areas and activities) and then get suitable results from the black box modelling. It is not a report – in style or content – that would pass muster in many professional areas.

Continue reading PHE: Experts’ report fails reasonable tests

PHE dashboard data: no reason to delay “Freedom”

The few weeks has seen endless media stories peddling worries about the Indian variant suggesting that the 21 June easing of lockdown needs to be postponed (with elements put off for the long term). “Data not dates” was the mantra. By far the most used source of data for the public has been the gov.uk dashboard. If the much awaited “freedom day” is to be delayed there ought to be unambiguous evidence in the most used data source. The current fashion for “Forget the data, fear the worst” will not do. The evidence is not there to delay – indeed there were fewer deaths in the latest week than the week before.

Continue reading PHE dashboard data: no reason to delay “Freedom”

Sex in official statistics – the regulator’s review

The categorisation of “sex” in statistics used to be a non-issue but in recent years it has become highly charged and very confused. The UK statistics regulator, OSR, has launched a timely consultation on the issue. As it stands, the definitions used by the government machine – and others – are a real mishmash with the result that we have potentially very misleading statistics. This was shown last week when the Fair Play for Women crowdfunded legal case, to seek clarity about what “sex” meant in the Census 2021, was heard. Thankfully, the pathway for OSR to be clear and robust about sex is much simpler following the court case: the High Court stated (see pdf at the end of this blog) that “What is your sex” means sex “as recorded on a birth certificate or Gender Recognition Certificate”. The statisticians must now fall in line. The OSR must now step up and do its job to ensure that they do: users will expect that statistics that are not compiled in line with law should not be classified as National Statistics. It is essential that rules are set and adhered to or more and more statistics will become meaningless. The OSR’s draft guidance needs to be substantially strengthened.

Continue reading Sex in official statistics – the regulator’s review

“Giving a voice” – an unfortunate confession from ONS

The Office for National Statistics does surveys “because it gives people the chance to have a voice”. Oh no it isn’t, it’s about collecting and publishing statistics about the nation. This was just one noteworthy comment in a weird interview with the National Statistician yesterday. That said, the “giving a voice” comment might help to explain why there’s been a mess up with the sex question in the Census, where a judge stepped in last week to get the ONS to correct its ways. Diamond also got into forecasting a third wave of Covid-19 in the autumn: I really think that “statistics” and the ONS should be about the past, not forecasts (especially since the ONS does not do epidemiological models). The National Statistician might also have quoted data* that’s not in the public domain – breaking his own code of practice! And all before breakfast on a Sunday!

Continue reading “Giving a voice” – an unfortunate confession from ONS

Vaccine alert – no numbers!

The government and its advisers are throwing vaccination numbers around on a daily basis. We want to believe them because the numbers are big and it points to good news. Sadly, so often the numbers are not published. I have written to the statistics regular to point this out and demand that such critical, important statements only quote published data. It is time for the NHS to do this – as a matter of urgency.

Continue reading Vaccine alert – no numbers!

Invisible Covid-19 vaccine data

No data has been published about the number of vaccinations given to care home residents in England or the UK. Yet we wake this morning to headlines that the NHS is saying “all eligible, older residents at more than 10,000 care homes have been offered the vaccine.” According to the same article (FT, £) “NHS England said figures set to be published on Monday were expected to show ……” The same story is now on the BBC website. This is not to deny that the very clear majority of residents have been vaccinated (and that the speed of roll out more generally is impressive) but as no data has been published to date to support the care home vaccine claims, people have been given very little reason to trust the next statement. It’s so sad when the data could be there if the NHS could be bothered, or be willing, to share it.

Continue reading Invisible Covid-19 vaccine data

Care homes: no data = no knowledge

How care home residents and staff have been tested for Covid-19? We don’t know. How many have received the vaccination in recent weeks? We don’t know. How many have died of Covid-19? We don’t know. Spot the pattern? When it comes to care homes, it’s a matter of no data equals no knowledge.

Continue reading Care homes: no data = no knowledge

Regulator fails to seek minimum standards on GDP

The debate about the accuracy and comparability of the GDP numbers being produced under Covid-19 rumbles on. I was pleased therefore to get a response from OSR to my email. Sadly my questions (listed in the annex at the end of this blog) were mostly not answered and the OSR sees little worthy of mention. The regulator is doing too little too late and does not seem to have given enough attention to a good many lines in the code of practice (set out below). Statisticians producing GDP data are not: including a clear approach to quality management, being open about areas of improvement, giving clarity about data inputs and the impact of any changes, being transparent about the impact of data changes on statistics, providing advance notice about changes to methods, releasing new data as experimental statistics, providing any validation of data with other sources. OSR needs to look closely in particular at section Q3 of the code about quality assurance as ONS is not being sufficiently transparent, or giving guidance about use, notably in respect to the strengths and limitations, reliability, consistency and comparability of GDP data.

Continue reading Regulator fails to seek minimum standards on GDP

UK’s population estimates are “shockingly bad”

The country’s population estimates are latest statistical victim of Covid-19 as they are exposed as “shockingly bad”. A report in the last week has suggested that the latest ONS estimates have failed to spot the loss of over a million foreign born residents who left as a result of the pandemic. Sometimes the published numbers are so off beam, and so obviously implausible, one wonders why they get published. It’s a classic example of a phenomenon well known among data watchers: that the ONS cares more about production cycles and processes than what the numbers show. There is no “sense check” mechanism and it is a mystery as to why UKSA/OSR lets this carry on. It’s as if the regulator only responds when the producers own up to a mistake or an influential user demands change.

Continue reading UK’s population estimates are “shockingly bad”