Skip to content
El Observatorio

Building & Agents · 2 min de lectura

Evals Are the Mom Test for Your Agents

Rob Fitzpatrick's The Mom Test teaches founders that people will lie to you kindly — so ask about real behavior, never about your idea. The same discipline now applies twice: to your customers, and to the agents doing your company's work.

Con · estudiado y reformulado para builders AI-native“The Mom Test” — Rob Fitzpatrick

Compliments are data-free, from customers and from models

Fitzpatrick's core warning is that a compliment tells you nothing — your mom will say she loves your idea because she loves you. An agent's self-report is a compliment with the same problem: a model telling you the task went well is politeness, not evidence. The answer in both cases is to stop asking for opinions and start measuring behavior. That is what an evaluation harness is — the conversation that can't flatter you. Build the eval before you build the agent, and let the cases that fail drive the design, the way hard customer questions drive the product.

Ask about the past, not the hypothetical

The Mom Test says to ask what someone actually did last week, not whether they would use your product someday — hypotheticals are cheap and specifics are honest. Run your systems the same way. Don't ask whether an agent would handle an edge case; collect what it actually did across real runs and turn the failures into red cases in the harness. Vague autonomy is the hypothetical answer; clear tasks, scoped tools, and tests against recorded behavior are the specific one. A workflow you can replay beats an agent you can only ask about.

Commitment is the only signal that survives infinite software

Fitzpatrick's test for a real customer conversation is advancement: did they give up something — time, money, an introduction, access to their data? When software becomes cheap to create, this gets sharper, not softer. Anyone can ship the demo now, so a kind word about your product is worth even less than it was. What remains scarce is distribution, customer access, and trust — and commitment is how trust reveals itself. Design for the moment a customer trades something real, and treat everything before that moment as unvalidated.

The conversation stays human

Agents can draft the survey, summarize the transcripts, and cluster the feedback, but The Mom Test is ultimately a book about judgment — knowing which answers were polite noise and which were the truth slipping out. That reading of people is exactly the work autonomous companies still route to humans: setting the goals, weighing the tradeoffs, deciding what a customer's hesitation actually meant. Let the agents do the work around the conversation. The founder holds the conversation itself, and owns what the company decides to believe from it.

Prueba el Desktop OS

Explora la escena startup de Atlanta, en vivo.

Todo lo que acabas de leer vive dentro de nuestro Desktop OS — un espacio de trabajo interactivo que se abre directamente en tu navegador. Su mesa de Atlanta mapea la escena startup de la ciudad: quién está construyendo, qué está pasando y dónde. Ábrelo y échale un vistazo.

Abre el Desktop OS →