Fluency Is Not Accuracy
An agent that returns a clean, well-structured answer feels correct in the same way a sentence that reads smoothly feels true. Kahneman called this cognitive ease, and it is a trap. Founders shipping agents this year must resist the pull of confident output. The agent's tone carries no signal about its reliability. Build evals that test for correctness independent of fluency, and treat a polished response with the same skepticism you would give a nervous one.
Anchoring Lives Inside Every Prompt
When a human sees an initial number, it warps every estimate that follows. Agents face the same structural problem from the other direction: whatever framing is in the prompt becomes an anchor the model reasons outward from, rarely against. If your workflow is built on stacked agent calls, early errors compound quietly. Design your harnesses so that each stage can challenge the premise it inherited, not merely refine it.
The Planning Fallacy Is a System Design Problem
Kahneman showed that humans systematically underestimate time, cost, and complexity by focusing on the best-case scenario and ignoring the base rate of similar projects. Agent workflows inherit this bias when scoped by overconfident founders. Humans must hold the judgment call on anything irreversible: a timeline promised to a customer, a budget committed to a vendor, a legal boundary crossed. Match the approval layer to the real stakes, and never let an agent set a commitment that a human has not pressure-tested against outside-view history.
Design Audit Into the System
System 2 does not activate automatically — it requires a trigger and a cost worth paying. In an autonomous company, that trigger must be designed in, not hoped for. Humans should own the narrow, consequential tip of the pyramid: the goals, the tradeoffs, the decisions you would defend in a room with a customer. Agents own the reversible, measurable base. The organizational design question is not whether to trust agents but where to place the mandatory pause — and making that pause real enough to catch what fluency disguises.
