I hope you had a great weekend. I had a long one. One day extra makes it feel so much longer. Almost like you got two Sundays. I am a bit extra happy since yesterday I got my first covid shot. Yeah!
Last week I actually finished two different books. One of them is the The Data Detective by Tim Harford that I will talk about today.
This books is a book a about statistics. But don’t worry, there is almost no math at all in it. When Harford was young – or at least younger, he read Darrel Huffs book “How to lie with statistics” and loved it. This is a classical book talking about how statistics commonly is being used by fraudsters and tricksters to manipulate people. Statistics has gotten a pretty bad name and Harford claims this has had negative consequences on peoples life and wellbeing. The fight to prove that cigarettes where causing cancer was one of these classical evidences. Harfords intentions with this book is to persuade the reader that statistics can be used to illuminate reality with clarity and honesty.
The book is split up in ten main chapters or the 10 rules. I personally don’t like when books are being split like this. Why is it always an even and nice number – wouldn’t it make sense to have 12 rules or 8?
I have studied quite a bit of maths at the university and statistics being one of the subjects. Thus a lot of the things he discussed was things that wasn’t anything new to me. But if you are less well versed in math I think it gives a pretty good insight into the world of numbers and what do matter.
The chapter that spoke most to me was the fifth rule. This was addressing survivorship bias and related to this publication bias. And how these two biases impact modern science and especially the softer sciences like psychology.
Harford talks in the book about research being done and we only see the result, the successful experiments. We don’t see what was done and didn’t work. This sometimes leads to what is called HARKing – it means Hypothesizing After Results Known. And it means that a scientist does his experiments – looks at the data and then makes the hypothesize – and it will then of course be true. It is nothing wrong looking at data and making hypothesizes – but they then needs to be validated using different experiments.
Daniel Kahneman, nobel laureate and author of thinking fast and slow, and others have been pretty vocal about the state of the science and especially the experiments done. It is a “Train Wreck” to happen. And these discussions has lead to scientist pre-registering their experiments so that HARKing becomes harder. There is also a global effort to retest well known experiments and trying to replicate them. This has in some cases lead to some corner stone research to be questioned when it wasn’t possible to be replicated. The Ego Depletion theory is one of these. But in this specific case the debate is still ongoing since it wasn’t the original experiments that where being replicated. It is quite interesting to follow the discussion and se how the research is done.
A second section I like was when Harford talks about illustrated statistics and this in the form of graphs. He here talks a lot about Florens Nightingale – the British nurse that brought hygiene to military hospitals and completely changed medical history. What many people probably doesn’t know was that she was a statistician and created one of the first really successful graphs that is called the rose diagram. This was specifically to illustrate the severe state in the military hospitals. And it really worked. She got her message through. Which is an extraordinary feat since she was a woman and succeeded in a heavily man dominated space. A great roll model.
I don’t think the book will change my life in any major way. I learned a great deal about statistics – especially linked to the scientific space. Most of the other areas he talks about was not, to me, any major news. But if you think statistics is an area held in shadow or potentially a bit magical. This is a great book to give you some guide lines and things to think about when you are approached with statistics: watch your feelings, be curious.
