Kahneman's core finding — that fast, confident judgment is the default and slow deliberate review is the exception — maps almost perfectly onto how language models behave in production. If you haven't designed the audit, it isn't happening.
Daniel Kahneman spent decades documenting how human minds substitute hard questions for easier ones, lock onto the first number they see, and systematically underestimate how long and costly anything new will be. These aren't edge cases or signs of low intelligence — they are the architecture. The fast system handles almost everything and the slow system gets called in far less often than people believe. Confidence, crucially, is not a signal of accuracy. It is a signal that the fast system found an answer.
The problem scales. An agent running hundreds of decisions per hour inherits every bias Kahneman identified — anchoring to whatever context appears early in a prompt, substituting a confident-sounding answer for genuine uncertainty, and planning future task sequences with the same rosy optimism that makes human project timelines fiction. What humans do occasionally, agents do at volume and without embarrassment. The design implication is structural: System 2 behavior doesn't emerge on its own, so founders have to build it in deliberately — through verification steps, explicit confidence thresholds, and human review gates positioned where the stakes actually concentrate.
- Confident agent output is not accurate output, treat fluency as a red flag worth checking
- anchor contamination is real at the prompt level, so audit what context your agents see first
- the planning fallacy lives in your agent's task estimates, build in skeptical review before those estimates drive real commitments.
