Thought I’d do a short review of ‘Noise’ by Kahneman, Sibony and Sunstein, based on a few notes I took.

Authors distinguish between system Bias (average error) and system Noise (standard deviation) when judgement accuracy is definable and desirable (e.g. medical diagnoses, sentencing).

Types of Noise

I) Level Noise — Variability of an individual’s average judgement (e.g. lenient/harsh on average)

II) Pattern Noise — Variations due to individual’s idiosyncratic responses to cases/people (e.g. harsh judge being unusually lenient with older defendants who shoplift).

Pattern Noise is then broken down into a stable and random component — Stable Pattern Noise vs. Occasion Noise. The former refers to repeatedly observed idiosyncrasies. Occasion Noise is more random — for example, the impact of mood or the weather.

The relative sizes of bias vs. noise will depend on the system. Further, decomposing different kinds of noise is tricky.

Authors argue for such things as ‘noise audits’ and simple decision rules or algorithms to reduce noise. Many of these suggestions make sense, with the caveat that we must select where and how to use them with great care.

Issues with book

The core message and Noise categories were interesting enough, at least conceptually. While I am no statistician, I noted some issues:

Authors do not seem to understand something as fundamental to their work as correlation, which only works under linear relationships. There was even a statement that causation cannot happen without correlation. This is simply false. Further, as Nassim Taleb teaches us, correlation is itself a random variable — so we must be very wary of gamed R-squared!

Authors seem to be unaware that much of the real world is non-Gaussian, and that decision making must account for ergodicity.

They also discuss forecasting with no reference to fat tails.

My review of noise by Kahneman et al

All this is very disappointing, but it does not undermine the simple point that noise exists and we can do something about it.

In short, there was some interesting signal, but Noise was very noisy. I was tempted to abandon the 380-page book mid-way through. I kept reading, because for all his flaws, I value Danny Kahneman’s contributions.

_____________________________________________________________________________________________

Harsha is a 1:1 coach and independent thinker based in London. He empowers people to find more clarity, confidence and focus in their lives — to cut through the noise, in a world so full of it. Harsha’s new book, Machine Ego: Tragedy of the Modern Mind, is now available in paperback and Kindle through Amazon.