By Sanjeev Kotnala
I pre-ordered the book ‘Noise – A Flaw in Human Judgment’ because of Daniel Kahneman, the Nobel Prize-winning star author of ‘Thinking, fast and slow’ and Cass R Sunstein; the co-author of ‘Nudge’ two books I remember liking. Kahneman co-authored ‘Noise’ with Olivier Sibony and Cass R Sunstein. The book Noise promised to explain, grade, identify and share how to reduce Noise.
WHAT IS NOISE?
For that, let me share something you can relate to. One often checks for the boss’s mood before discussing a business proposal because the mood can influence the decision. Or, if you had multiple choices to approach seniors, you pick one over the other. Technically, the proposal should be decided on its merit. All the seniors should take the same decision, or at least the same senior should take the same decision all the time. Or, when you dial into a call centre for a problem resolution, a lot depends upon is assigned the call. And that random, seemingly unbiased allotment of your call may actually decide if the problem will get resolved. We all know life is not idealistic. It does not happen that way. The reason for this is bias and noise. Noise is the “unwanted variability in judgments”, and consistency means fairness to all.
It seems so simple an explanation. And once you start reading the examples in the book- you realise we have allowed too much noise to creep in some areas at too high a cost. Bias and noise are omnipresent in all judgments. Noise is there in the court judgments, insurance approval, recruitment, analyst recommendations, medical diagnosis, child protection, performance appraisals, fingerprint experts and student grading. In fact, the list is unending. Noise is wherever there is a judgment.
JUDGMENT
The authors point out that judgment should not be confused with thinking. Judgment is a form of measurement in which the instrument is the human mind. Judgment informally integrates diverse pieces of information into an overall assessment. They are not computations, and they do not follow exact rules. That is where noise and bias slip in.
Some judgments are predictive, and some predictive judgments are verifiable; we eventually know whether they were accurate. Many are long-term and unverifiable. The quality of such judgments can be accessed only by re quality of the thought process that produces them.
They remind that the phrase ‘judgment call’ implies both the possibility of disagreement and the expectation that it will be limited. Matters of judgment are characterised by an expectation of bounded disagreement.
THE GENESIS.
In a world of no bias and noise, the judgment on a particular case should not depend upon the judge assigned or the date, day, and part of the day. This is hardly true. A 1974 study of 50 judges setting sentences for identical (hypothetical test case) cases found hardly any consensus. The sentences covered a large spectrum of possibilities. The same drug dealer was sentenced to anything between one and 10 years. A bank robber received sentences ranging between five and 18 years. An extortionist faced anything between three years with no fine to 20 years plus a $65,000 fine. Other studies in 1977 and 1981- showed the same result. These were hypothetical cases where the same set of evidence and short synopsis were presented to the judges. However, real life is a lot more complicated. A lot more information and inferences are on the show. The problem of varying sentences is real, and it is far more significant than what these researchers point out.
BIAS AND NOISE
In Noise, a shooting-range example is used to explain it.
Bias is when all the shots land systematically off-target in the same direction.
Noise is when the shots are all over the target sheet. The problem is not missing the target but a lack of consistency.
One has to know the right answer to detect bias, like knowing where the bull’s eye is. For noise, you do not need such details. You will know and identify it even if while looking at the target sheet from the other side. What is important is to know if there is variability.
ALGORITHMS AND NOISE
The book does make a strong case for mechanical judgment governed by rules. However, they rightly point out that the algorithms may not be a solution. As they can, in fact, end up amplifying the bias. More so, when the data they are working on is corrupt or has parameters that can generate a bias. The risk of bias in algorithms is discussed, explaining that it can be filtered out of algorithms.
In the book, the authors clarify that their goal “is to offer suggestions for the improvement of human judgment, not to argue for the ‘displacement of people by machines’”. Algorithms are not perfect, and at least as of now, their lead over human judgment is slender. Also, algorithms may never be perfect “in many domains”, so “human judgment will not be replaced. That is why it must be improved”.
TOO MUCH NOISE IN NOISE
The book is far too long. It is iterative. And it seems the authors have a shallow view of the readers’ capacity to understand a subject. Yes, the subject is serious and needs explanation, but keeping it so long and iterative is not something I love. Noise is tiring and hardly an engaging read.
Maybe I was flawed in my judgment. I was bitten by excessive coherence, thinking that I could possibly not go wrong with the star author of Thinking, Fast and Slow’ and the co-author of ‘Nudge’. Just because last was a superhit does not mean that the next one will be even good enough. Past performance is not a guarantee for future performances; reader discretion is demanded. In the case of John Gresham, I had this feeling when I read the Painted house.
In fact, in this book of some 454 pages, the book is 385 pages. Twenty pages of enriching appendix covering how to conduct a noise audit, a checklist for decision observer, bias observation checklist, and correcting predictions. Fifteen pages of Review and conclusions- titled ‘taking noise seriously’. And for most of the readers, reading these 15 pages is more than enough.
WHAT’S NEW?
You can say Nothing, and then you can say a lot. What is Noise is well known to all of us? As we are intrinsically creators, managers, and victims or beneficiaries of Noise. That every judgment is flawed in many ways is no news to us.
Noise tells you that not only individual but group decisions can also have noise.
What it does is give you tags to the noise. A possible explanation and maybe something more to blame. It introduces system noise, pattern noise, naïve realism, forensic confirmation bias, bias blind spot, informational cascade, stable pattern noise, occasion noise level noise, and psychological biases. I think it adds too much noise to the already complicated lives of average men.
NOISE tells you that some noise is essential and even good- primarily in case of artistic judgment- like movie review- onboarding a script- valuation of a painting. Moreover, controlling noise comes with a cost and the possibility of giving birth to a fresh set of noise.
It prescribes ways to control noise and, in the process, differentiate between clinical and mechanical decision-making.
OBJECTIONS TO REDUCING NOISE
Some of the major objections to efforts to reduce or eliminate Noise are;
• Reducing Noise can be expensive.
• Some strategies introduced to reduce Noise might introduce errors of their own.
• We have to tolerate some noise, if we wish people to feel they are treated with respect and dignity.
• Noise might be essential to accommodate new values.
• Some strategies designed to reduce Noise might encourage opportunistic behaviour.
• People do not want to be treated as if they are mere things or cogs in some kind of machine. Some Noise reduction strategies might squelch people’s creativity and prove demoralising.
THE IMPORTANCE OF CONTROLLING NOISE
Quoting from the book. ‘Most of the time, professionals have confidence in their own judgment. They expect that colleagues would agree with them, and they never find out whether they do so. In most fields, a judgment may never be evaluated against a true value. It will at most be subjected to vetting by another professional who is considered a respect-experts. ( experts who are respected by peers in their field). Only occasionally will professionals be faced with a surprising disagreement, and when that happens, they will generally find reasons to view it as an isolated case. The routine of the organisation tends to ignore or suppress evidence of divergence among experts in their midst. This is understandable from the organisational perspective, Noise is an embarrassment.’ And that is the reason enough for people to try controlling noise. Or the need to flow decision hygiene of independent decisions and aggregation.
The six principles that define decision hygiene are
• The goal of judgment is accuracy, not individual expression.
• Think statistically, and take the outside view of the case.
• Structure judgment into several independent tasks.
• Resist premature intuition.
• Obtain independent judgments from multiple judges, then consider aggregating those judgments.
• Favour relative judgment and relative scale.
I understand and am with the authors when they point out that enforcing decision hygiene can be thankless. Noise is an invisible enemy, and a victory against an invisible enemy can only be an invisible victory.
NET IMPACT
It does make you realise how you have been inconsistent in decision-making. Why you must structure your probe. How appraisals, recruitment, subject expert medical doctors, or to say any judgment can have noise.
It does make one aware of noise in their life and how one could attempt to be more objective. But then, I don’t know if this awakened phase would last.
SHOULD YOU PICK UP NOISE
You may read if you want to bring more objectivity to your judgment. If you are in a decision-making position and have been worried about why you are not confident of your decisions. Read it if your life is not already complicated and you want some new jargon and discussion points on human judgment’s flaws. OTHERWISE, JUST LEAVE THE BOOK ALONE. It is not for everyone. I know Vermajee, my friend and a brand marketing consultant would love it. Maybe he will start offering ‘Noise Audit’ to clients who never knew what the noise is all about.
Sanjeev Kotnala is a senior industry professional, brand and marketing consultant, a coach, trainer and author of three books. He writes on MxMIndia every Wednesday.
