The rain was pouring down outside so Bank Holiday Monday was a good time to read the blog from Met Office Chief Scientist Professor Dame Julia Slingo OBE FRS in which she reflects on this summer’s weather. The title – “So what happened to our summer?” – gives away the sense of mild embarrassment about the poor predictions for July and August. The Met Office has lost the confidence of the BBC which has terminated the weather presenting contract. (This Met Office blog covers that and criticism of its apps.) Beyond the accuracy of its forecasting, I see evidence that the Met Office has confused governance, is not being very open with its data, not focusing on the public’s needs and opinions, setting itself lower targets for forecasting accuracy, and is becoming less transparent in its operations. Perhaps it’s no surprise that the organisation is being reviewed by Government, not least as they were given the go-ahead last year to spend £100m on a new computer.
I have had my issues with the Met Office. Earlier this year I was writing about open data and the Met Office is a key player, or should be, as one of four trading funds crammed full of valuable data. Yet trying to find their policies and understand the reporting structure is hard. It has a board that “challenges and supports the Executive team”. (See page 31 of the annual report.) There is also the Shareholder Executive (reporting to the Chancellor not BIS as of May this year) and the now defunct Public Data Group and three “additional review bodies”. It was not obvious before the election who was doing what, and is even less clear now. It’s no surprise perhaps that the organisation is run with so little transparency. The demise of the PDG – one of several blows for open data transparency following the election – means there will presumably be even less transparency! That’s not good.
I was also interested in public perceptions of the service. It seems that the Met Office carries out two surveys. It is shocking that they do not publish the results of these surveys on its web pages, choosing just to show a few selected charts/tables over short time periods. They show the responses to two question:”Generally speaking, how accurate or inaccurate do you think most weather forecasts are?” and “Overall how useful would you say weather forecasts are these days?”. From the charts (one is copied below), it looks as if, within annual ups and downs, the trend is flat. But the charts show that the “benchmark” is falling over time – and that appears to be their target. So, how do you make yourself look better if you can’t improve your performance? Lower the expectations! The Met Office says that they are set “challenging annual performance targets to ensure that the PWS is meeting the needs of the public and public sector users.” Umm.
The survey pages are at least now up to date. As of last December they were over three years out of date. That gives a flavour of how important the public is to the Met Office.
The Met Office does publish some open, free to use data. Quite where you go to see their targets for better, more, different open data, I don’t know. I don’t think there are any.
The annual report published in June also said: “We (the Met Office) are currently conducting a General Review in collaboration with BIS, focusing on the role of the Met Office, and how this is underpinned by its delivery and funding model. It also aims to ensure Government as a whole understands the Met Office’s purpose, value-add and structure. A high-level summary of the Review is expected to be published in late summer 2015. A Commercial Review was also initiated during the year, looking at how we participate in our chosen markets, and what operating model is most efficient for our commercial activities. This review is ongoing.” This will presumably lead to the publication of a revised version of its Framework Document from 2013.
I was also curious about the accuracy of their forecasts. The Met Office’s web pages claim that “The Met Office is a world leader in forecast accuracy” and that they have “an open and transparent policy on how well our public weather forecasts are performing“. A Freedom of Information request for underlying data resulted in, err, no data of any consequence and a very thin explanation of the process of determining accuracy. So not so open! (Even more irritating was the incompetent way that the FoI requests were handled – muddled and garbled responses of the sort that is surprising given how many supremely well qualified people work there.)
So, what? I have at least a niggling concern that all is not well at the Met Office.
About the Met Office: Set up in 1854 in the Board of Trade, now a trading fund of BIS employing 1700 people around the world. More about the management structure of the office can be found here. Following some nagging from me (mainly FoI requests highlighting the inadequacies) the PDG web pages are more informative than they were a year ago. But now the PDG has been wound up. There is also an “independent” group (the PWSCG, one of three “additional review bodies”) that supposedly oversees the Public Weather Service from a customer point of view. Oddly it comprises virtually entirely people who are public sector employees of one sort or another. Quite how good they are at judging of the public’s demands and perceptions is a matter of debate. The Met Office is currently spending almost £100 million on a new computer to get the forecasts more accurate. (I looked for information on spending/accounts on their website – it’s probably there but I couldn’t find it where one might expect, like here. There is a blog from last October announcing the spend on the new computer. And the 2014/15 annual report is here.)