Why a learning culture is important (Part Two)


In this article we are going to expand on part one article. We are going to explore the role of forecasting. Statistics and probability in government.

It is ironic that nobody saw the coronavirus crisis coming. Because everyone saw it coming. A pandemic was top of the UK government risk register and still it seemed to take everyone by surprise.

Suddenly we are all living in a world of mathematical models. Projected curves and logarithmic scales.

Time for our first class. Anybody who is going to be making decisions based on evidence or data analysis. Needs to know what the common pitfalls are.

What they should be thinking about?
What questions to be asking? 
What are you seeing?

Professor Jennifer Rogers vice president Vice-President for External Affairs at the Royal Statistical Society. States everybody should be equipped with statistical data analysis skills. If you are going to be using information to make future decisions. You need to be making sure that you fully understand what you are seeing.

It is ironic Dominic Cummings himself. A humanities graduate called for this in a talk he gave at the Institute for Public Policy Research 2014. Just looking at basic probabilities. The Royal Statistical Society found. Only forty percent of MPs could correctly answer the question below.

What is the probability of getting two heads from flicking a coin twice? Three quarters of MPs said they got the right answer.

The assumption most make is fifty, fifty, heads up. Two heads are half of a half. One chance in four. But you knew that right. Unlike 60% of MPs. Luckily, the royal statistical society now runs training sessions for politicians.

Professor Rogers states the problem goes beyond basic probability. Politicians lack of statistical understanding. This factor was revealed when it came to the Coronavirus data.

Yes, we see government data presentation briefings. Professor Rogers states. It is the first time the general public has been given statistics on a daily basis by the government on the BBC. But what is interesting it illustrates the challenges of presenting data. The public never see the uncertainty they always see point estimates.

Point estimate

A point estimate is one single value. It is not the best guess as to what that value might be. For example, numbers at the start of the epidemic suggested 20,000 deaths would be a good outcome. It was a statement which we have now passed. The failure was the figure was a point estimate. The public were never given any kind of indication as to the degree of uncertainty. If you are not trained in thinking about such certainty. It is easy to conclude what might not necessarily be true. If you learnt to question certainty. You might make very different decisions based on what was shown.

Government may have thought the general public cannot manage uncertainty. The public want a leader to give a very clear message. They just want one simple number. A question should be asked do politicians understand it themselves. Personally, I think the public can sometimes see uncertainty as a deficiency. If a politician states a number with degree of uncertainty. Then the public may view them as unreliable.

Throughout the Corona 19 crisis statistics have quoted as certain. If understanding incorporated a degree of uncertainty this may help people to make better decisions.

Sir David Spiegelhalter has stated the public are hungry for details for facts for genuine information. Yet they get feed what he calls number theatre. Rather than genuinely trying to inform people as to what is going on. He just wished that the data being brought together and presented by people knew it strengths and limitations. Treat the audience with some respect the public might believe you.

Sir David Spiegelhalter thinks the way numbers are presented can completely change their impact. He says he could make any number big or small any fact look reassuring or frightening what do you want. The crudest thing of course is to talk about five percent mortality rate. That sounds frightening but change the ninety five percent survival rate that sounds a lot better to people. Numbers are not cold.

Sir David Spiegelhalter stated in May 2020 numbers can produce emotional reaction. He saw numbers that morning concerning Covid 19 that nearly had him in tears. He hadn't had such an emotional response to numbers for as long as he could remember. He thought the extra deaths may have been incorrectly labelled as Covid 19 on their death certificates. It is a huge, huge number. He found that shocking. But the numbers did not tell him that. A graph did and the sudden realisation hit him with a thump. The presentation style is absolutely vital.

Statistics are often presented as if they are objective but they are subjective. Generally, statistics are full of assumptions.

They are treated as exact measures of the world. In reality, statistics are a partial picture supplemented with maths. Statistics are approximations of reality for one specific purpose. Statistics are what somebody thinks. Generally, they are revered as an infallible Oracle. In fact, they are just another tool for human beings to try to make sense of the world.

More often than not politicians do not understand how they work. Statistics should come with caveats when making decisions. When you know what they say remember it is hard to make predictions especially about the future. Ask any weather forecaster.

And what do you need to forecast the weather maths.

Brier score

definition by Brier

David Spiegelhalter states when you get probabilities of rain. The forecast is being scored according to a squared score rule called the Brier score. This was developed by a weather forecaster called by Brier in the early 1950’s. This is massively used throughout the entire weather forecasting industry. And has been for decades. And it should be used everywhere else as well.

This rule helps forecasters make more accurate predictions. it does this by encouraging them to be more honest about their uncertainty. All probabilities are somewhere between zero and one.

A probability of one means you think something is definitely going to happen.

So, if the weather forecaster says seventy percent chance of rain. The probability for rain is 0.7 so take an umbrella. And this rule is used to give scores to weather forecasters. Where they lose marks for being wrong and gain marks for being right.

You might think I will give point seven probability to an event to occur. And then it occurs I have missed it by point three so you might say I lose point three. No, you do not lose point three. You lose point three squared. You lose point zero point nine. So, if it did not occur, I would be point seven out. I lose point four nine. So, notice I lost five times as much if it did not occur as if it did occur. So instead of losing two and a bit I loose five times as much.

This system awards you for not just getting the answer right. But for saying how certain you are about the answer you give. Are you, nought point nine sure it will rain, or do you think it is just a bit more likely than not?

Nought point five, five maybe. The squared score rule means. That in the long run you will do better. If you are good at predicting not only the outcomes but how likely those outcomes are to happen.

Nought point five, five maybe. The squared score rule means. That in the long run you will do better. If you are good at predicting not only the outcomes but how likely those outcomes are to happen.

As soon as you apply a square. You are encouraged mathematically to say what you really think and that minimises your expected penalty. These are called proper scoring rules. Beautiful, beautiful little bit of maths. I think you will agree that the above brings a whole new meaning to the phrase dressed like a dog’s dinner.

You may be thinking that’s great for weather forecasters but how does that' help us with politics or making decisions in organisations. Well there is another group specialise in this kind of thinking about prediction to use. They apply the same Bryan scoring system perfected by weather forecasters. They are called super forecasters. You may have heard Dominic Cummings being questioned outside his home. Have you got any more weirdos? You called for super forecasters. Do you know what you are talking about?

Tomorrows article will expand upon the first two articles.

I will attempt to ask whether super forecasters should make political decisions?
Whether politicians and ourselves are really equipped to use the data we have? 
Or is it time to send us all back to the classroom for extra Maths?

About the author 

Christopher Bird

Building your own Power App, BI solution, or automated workflow can be a mind-blowing experience. It can also be a nightmare. Particularly when you begin with a blank screen. My advice, get professional help as and when you need it. That's what successful people do.

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
Subscribe to get the latest updates