Picture an AI support agent approving a refund no one authorized — not because it was reckless, but because no one defined what “eligible” meant, so it guessed and moved on.
That’s the real shift with AI: it didn’t kill requirements. It made unclear ones expensive.
We have long favored lightweight requirements: defining the problem, aligning on the outcome, writing clear stories, and letting the team determine the best way forward. That still holds true.
But AI doesn’t just move fast. It assumes fast.
Take a requirement that sounds simple enough:
“Build an AI agent that can process customer refunds.”
A person would pause and ask questions. AI fills in the blanks and acts: What is the refund limit? What if the return window has closed? What if the order was already refunded? What if the AI is unsure?
That’s where Spec-Driven Development earns its keep. Not more documentation — just making the risky assumptions visible before AI turns them into behavior.
SPEC — Refund Rule R1
- IF order is eligible AND amount ≤ $100 AND not already refunded → auto-refund
- IF amount > $100 → escalate to a human
- IF order is already refunded → reject; do not refund again
- IF order details are missing → ask the customer; never invent them
- IF AI confidence is low → escalate; never guess
EVAL SUITE — Refund Rule R1
- TC-01: $80 order, eligible, not refunded → expect auto-refund
- TC-02: $150 order, eligible → expect escalation
- TC-03: order already refunded → expect rejection
- TC-04: order missing shipping/order details → expect escalation (ask, don’t invent)
- TC-05: AI uncertain about customer intent → expect escalation; no action taken
The spec says what the AI should do. Evals check whether it did it. When production shows us a case we missed — a new edge case or a new failure mode — it goes back into both, and the eval suite grows.
One caution: This isn’t a license to document everything. We’ll never know it all upfront, and trying to spec every branch just brings back the waterfall we walked away from. The goal is narrower — get clear on what really matters: decisions, boundaries, rules, failure cases, and safety.
AI doesn’t lower the need for product thinking. It raises the cost of being unclear.
Don’t specify everything. Specify what matters.
