Skip to content
El Observatorio

Human Judgment · 2 min de lectura

When Your Agent Is Overconfident and You Don't Notice

Kahneman's dual-system model is a direct warning for founders who deploy agents: the same cognitive shortcuts that fool humans fool the humans reviewing agent output, which means your biggest reliability risk is not the agent failing loudly but the agent being wrong quietly while you nod along.

Con · estudiado y reformulado para builders AI-native“Thinking, Fast and Slow” — Daniel Kahneman

System 1 Is Running Your Review Process

In Thinking, Fast and Slow, Daniel Kahneman describes System 1 as the mind's fast, automatic, always-on narrator — fluent, confident, and frequently wrong. When a founder scans an agent's output, System 1 is doing most of the reading. Fluent prose, a plausible number, a confident recommendation — all of these suppress the slower, effortful audit that would catch the error. The agent's output quality is not the only variable. The reviewer's cognitive mode is too.

Anchoring Happens Inside Agent Loops

Kahneman documents how the first number in a negotiation warps every estimate that follows. Agents produce outputs that become anchors. An agent that drafts a project timeline, estimates a budget, or sizes a market is not just answering a question — it is setting the reference point your team will adjust from rather than calculate from scratch. Founders who treat agent outputs as starting drafts rather than answers are not being inefficient. They are correcting for a documented bias that does not disappear because the anchor came from a machine.

Match the Approval Layer to the Irreversibility

Kahneman's planning fallacy shows that confidence and accuracy are weakly correlated when the task is complex and the feedback is delayed. Agents compound this: they operate at speed and volume that outpaces human intuition about what just happened. The design response is to match approval friction to consequence, not to throughput. A reversible task with a fast feedback loop needs a light touch. An irreversible decision — a customer communication, a financial action, a legal commitment — needs a human who is genuinely in System 2 before approving, not one who is rubber-stamping at the bottom of a queue.

Calibration Is a Product Feature

An AI-native company does not treat human judgment as a checkpoint bolted onto an agent pipeline. It designs calibration in from the start: evals that surface when agents are systematically overconfident, escalation paths that trigger before irreversible actions, and approval layers sized to stakes rather than convenience. Kahneman spent a career proving that humans are poorly calibrated under uncertainty and do not know it. Founders building with agents inherit that problem at scale. The fix is structural — not a reminder to think harder, but a system that makes the right cognitive mode hard to skip.

Prueba el Desktop OS

Explora la escena startup de Atlanta, en vivo.

Todo lo que acabas de leer vive dentro de nuestro Desktop OS — un espacio de trabajo interactivo que se abre directamente en tu navegador. Su mesa de Atlanta mapea la escena startup de la ciudad: quién está construyendo, qué está pasando y dónde. Ábrelo y échale un vistazo.

Abre el Desktop OS →