Noise: A Flaw in Human Judgment
The authors investigate unwanted variability in judgments that should be consistent and propose methods for reducing it.
Noise examines a form of error that receives less attention than bias: people reaching different judgments when the relevant circumstances should produce similar assessments.
Two equally qualified applicants may receive different scores because interviewers interpret the same answer differently. A shared rubric can reveal that disagreement, although the rubric itself still needs scrutiny for relevance and fairness.
Daniel Kahneman, Olivier Sibony, and Cass Sunstein distinguish this unwanted variability from a systematic tendency to lean in one particular direction.
Insurance estimates, hiring decisions, medical judgments, and sentencing illustrate the concern. Differences among evaluators may be substantial even when they use the same available information.
Variability can also occur within one person across occasions. Mood, fatigue, and the sequence of earlier cases may influence a judgment without being relevant to it.
Because each decision usually arrives with a plausible explanation, an organization may not notice the pattern until it asks multiple people to assess comparable cases independently.
Independent estimates collected before a meeting preserve information that can disappear when everyone follows the first confident speaker. The discussion can then address genuine differences rather than mistaking social agreement for reliability.
The authors call for decision hygiene: structured procedures intended to reduce error before anyone knows exactly which influences would distort a particular judgment.
Examples include separating dimensions of evaluation, obtaining independent estimates before discussion, and delaying an overall impression until relevant evidence has been considered systematically.
Rules and models can sometimes improve consistency, but a consistent system can still be biased, inaccurate, or insensitive to morally important differences between cases.
Reducing noise therefore requires deciding which variation is undesirable and which reflects legitimate judgment. Standardization is a tool, not an automatic definition of fairness.
The book makes consistency visible as an institutional responsibility: people should not have to depend on an evaluator's passing mood when the consequences concern their livelihood or freedom.
Mood, fatigue, and the sequence of earlier cases may influence a judgment without being relevant to it.
Where would greater consistency improve fairness, and what meaningful differences might be lost if the same setting became too standardized?