Saturday, November 23, 2024

The Art of Uncertainty by David Spiegelhalter review – a search for sense in probability and chance

Must read

In 2011, the psychologist (and Nobel laureate) Daniel Kahneman proposed that we humans are bimodal animals capable only of two modes of thought. One (which he called “System 1”) is fast, instinctive and emotional. The other (“System 2”) is slower, more deliberative and more logical. The first doubtless evolved when we were hunter-gatherers, and served us well in that reality. The second, involving slow, effortful, infrequent, logical, calculating, conscious thought, came later, as societies became more complex and uncertainty became an integral part of the human condition. Uncertainty is, says David Spiegelhalter, “all about us, but, like the air we breathe, it tends to remain unexamined”. Which is why he wrote a book about how to live with it.

Uncertainty, in Spiegelhalter’s view, is a relationship between an individual and the outside world. And, because of that, our personal judgments play an essential role whenever we are faced with it, “whether we are thinking about our lives, weighing up what people tell us, or doing scientific research”. And tolerance of uncertainty varies hugely among people: some are excited by unpredictability, while others are crippled by anxiety.

When dealing with uncertainty, Spiegelhalter argues, Kahneman’s System 1 is bad news. It “tends towards overconfidence, neglects important background information, ignores the quantity and quality of the evidence, is unduly influenced by how the issue is posed, takes too much notice of rare but dramatic events, and suppresses doubt”. Which actually is a good description of my Twitter feed, posted by people whose only form of exercise is jumping to conclusions.

Spiegelhalter is one of the country’s most distinguished statisticians, but he’s also one of academia’s best communicators (he’s professor of the public understanding of risk at Cambridge). His aim in this book is to impose some intellectual order on a subject area that is rife with incomprehension, imprecision, contradiction and creative obfuscation.

This is a pretty tall order, so it needs a big (512-page) title to cover the territory. Spiegelhalter’s target audience(s) include: students of statistics who want to go beyond the standard syllabus; anyone working with “risk”; scientists who want to learn how to communicate effectively about uncertainty in their research; and “the interested citizen who is largely reliant on ‘experts’ and wishes to assess their trustworthiness”. In other words, this particular reviewer.

He addresses the important question of how long it would take a group of monkeys to compose the works of Shakespeare

Spiegelhalter’s guiding principle is that if we are to think usefully about uncertainty then we need to use numbers. So no more loose talk about “chances”, “luck”, “probability”, “risk”, “likely”, etc. This inevitably implies some mathematics, but nothing very abstruse (in many cases simple arithmetic is all that’s involved). And sometimes it’s good for your morale to see how basic maths can sort out questions that regularly defeat contestants in pub quizzes, like: what are the chances that two people chosen at random will have the same birthday?

What makes Spiegelhalter such a good communicator is his willingness to record his own attempts at working things out. In his history of games of chance, for example, he goes back to the time when people gambled with knucklebones. So he goes to his local butcher, purchases a leg of lamb and asks that he includes the ankle joint – the astragalus, ie knucklebone – which has four faces on which it might land. He then starts tossing the bone around and after 200 throws tabulates the results.

At another point he addresses the important question of how long it would take a group of monkeys to compose the works of Shakespeare. He installed a Monkey Simulator program on his PC and left it running for days. After 113m imaginary monkey keystrokes (equivalent to about 26 days for 50 monkeys typing one character per second) the best the simians could do was “we lover”, which appears in Love’s Labour’s Lost, Act 2, Scene 1.

All of this is typical of the astute way that Spiegelhalter sweetens an important pill. Most of us (including journalists) are woefully ignorant about probability, chance and risk. Mass media consistently creates needless alarm about small risks (travelling by air) while ignoring real or more significant ones (driving to work). People are hoodwinked by anti-vax propaganda and spooked by pure coincidences. Politicians smugly repeat ancient tropes about “lies, damned lies and statistics”, and constituents nod complacently. It’s all stuff aimed at triggering Kanheman’s System 1. Faced with such wilful blindness, most academics would have shrugged and returned to their rooms. Fortunately, Spiegelhalter is made of sterner stuff.

Oh, and if you want to know how much TV you’d have to watch before you could expect to get a pulmonary embolism, he has an answer for you: five hours a night for 19,000 years. Roughly.

• The Art of Uncertainty: How to Navigate Chance, Ignorance, Risk and Luck by David Spiegelhalter is published by Penguin (£22). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply

Latest article