The first agent I built lied to me. Cheerfully.
I'd asked it to pull a quote from an insurance portal. It clicked the wrong control, read the number off the wrong row, and handed back a clean confident result — a green check on top of a wrong answer. Not broken. It had done exactly what it understood. What it understood was wrong, and nothing in its world told it the difference.
That moment didn't feel like a bug. It felt like a design problem nobody had named yet.
Here's what I've found since.
✦ ✦ ✦
§ 01
The design job moved upstream.
Most teams haven't followed it.
Before agents, design was outcome-focused. SaaS workflows were standardized: form → action → result, the same way every time. The UI was a faithful map of a deterministic process. Design's job was to make the path to a known outcome clear and efficient. The outcome itself was reliable by engineering.
Agents broke that contract at the top.
The interface has thinned to plain language — text, voice, intent stated in a sentence. But language isn't neutral. It requires interpretation. "Find me the best carrier" could mean cheapest, fastest, most reliable, best for this client's specific risk history. The agent interprets the intent and acts on that interpretation. The outcome lives downstream of it — which means the design job moved upstream: not designing the path to a known outcome, but designing the communication of intent so the interpretation produces the right one.
I spent three years on this at Alibaba. The B2B platform I led connected SMB owners in Ohio and Munich to manufacturers in Shenzhen and Dongguan they'd never meet — six months of inventory spend committed on the strength of a screen. A wrong supplier choice meant broken delivery promises, frozen cash flow, a business that didn't come back from it. Every design decision answered one question at that moment: can I trust this enough to act?
The agent version of that question is: did this system understand what I actually needed?
The blank box fails here — not for aesthetic reasons but because it hands the user a problem they can't solve. They don't know how the agent reads language, where its competence ends, or what level of specificity produces reliable output. The result is tentative input, ambiguous reading, output that misses intent. Not a model failure. A communication design failure.
The fix isn't a smarter empty box. The blank box trains the wrong behavior: one-shot attempts instead of deliberate communication. Working with agents requires more planning than any SaaS workflow ever did. The quality of the output scales directly with the quality of the intent communicated. The best results don't come from better one-liners. They come from iteration — refining what you mean, calibrating what the agent understands, building a shared language over time. That's an intentional practice, not a prompt. And it's a different design discipline entirely.
✦ ✦ ✦
§ 02
One great run isn't a product.
Consistent runs are.
A model provider update changed the behavior of a system I'd shipped — no announcement, no changelog that reached the people who needed it most. The first users we lost were the power users: people who had run the system hundreds of times, built actual processes on top of it, staked their reputation on its reliability. Some didn't come back. Not because the new outputs were bad. Because depending on something that changes silently and confidently — without any signal that it changed — was enough.
Trust is a floor.
Users don't average their trust across good and bad runs. They anchor to the floor. One confident wrong answer — the kind that looks right until it causes a downstream problem — does more damage than ten correct ones repair.
Most teams racing to improve model accuracy are solving the right problem for the wrong moment. Accuracy is measured in aggregate, after the fact. It is invisible at the exact moment that matters — when a person decides whether to act on this output, right now. What they can see is consistency: does this run match what the system has reliably done before?
Designing for consistency means making reliability visible, not just possible. Confidence signals that tell the user whether this specific run is as reliable as the last one. Evidence and provenance that make regression legible before it propagates into someone's work. Traceability: when the system changes, the change can be seen and evaluated — not discovered after the fact by a power user who noticed their workflow broke.
Accuracy is what the model achieves. Consistency is what the design delivers. They are not the same thing.
Deterministic software fails loudly. An agent completes confidently — and may still be wrong. In a non-deterministic system, consistency can't be self-reported. It has to be externally designed: evals to define what "good" means, observability to catch when outputs drift, annotation to hold the edge cases. That's not engineering infrastructure. It's the design of what "consistent" means for this specific system. Most teams haven't built it.
✦ ✦ ✦
§ 03
Five handoffs in every agentic workflow.
The fifth one compounds.
The engineering literature calls the human decision point "human-in-the-loop" and gives it one line: insert a checkpoint for high-risk decisions. One line. That line is the entire design job. And it isn't one handoff. It's five.
Setting 01
Structured intake, confirmed interpretation, success defined before execution begins. If intent wasn't designed at entry, nothing you build at exit recovers it. The most common failure isn't at the output — it's here, where ambiguous input gets interpreted as if it were precise. The agent acts confidently on the wrong thing. The user never knew the gap existed.
Progress 02
Not a spinner. A user who watches work build in real time trusts the result more than one who waited at a blank screen. Confidence forms through visibility, not through time passing. When the work is invisible, users fill the silence with doubt. When it's visible — steps completing, sources resolving, reasoning accumulating — they build a working model of what the system is doing. That model is the foundation trust stands on.
Review 03
The user is silently measuring this output against their model of what the system reliably does. The design either helps them calibrate or leaves them guessing. The tool here is evidence: what sources, what confidence level, what did the system not find? An output without provenance asks the user to trust a result they can't evaluate. That's not trust design. That's a leap of faith dressed as a product.
Gate 04
Not "approve?" A designed surface with enough context to make a real call without redoing the work. Automate what you can verify. Keep a human where the outcome is irreversible. At Thermo Fisher, five agents ran continuously — monitoring batch status, exception flags, partner delivery performance across nine manufacturing sites. One human gate was non-negotiable: QA regulatory release before any batch shipped. Not because automation couldn't surface the issue. Because the downstream cost of a wrong automated release was irreversible and had a name on it.
Feedback 05
Output of one loop becomes input for the next. A slight misclassification in run one shapes run two, informs run three. By run five, the error isn't a wrong answer — it's a wrong strategy. This compounds fastest in systems where nobody designed the feedback path. The model drifts. The outputs feel subtly off. Users can't name what changed. What changed was the loop — a small error at the handoff, amplified by iteration.
hover each card to reveal · click to hold
Design all five. The ones you skip are the ones that compound.
✦ ✦ ✦
§ 04
Nobody delegates on day one.
Design the path, not just the destination.
Every person who has ever trusted a complex system started by not trusting it.
They watched first — ran the agent, compared its output to what they would have done, looked for the gaps. Then they verified — spot-checked a sample, built a mental model of where the system was reliable and where it wasn't. Then, slowly, they delegated — trusted the output enough to act without review, because they'd accumulated enough evidence to know what "reliable" looked like for this system.
This is the trust ladder. Every user is on some rung. Most products design for the top rung and wonder why people don't climb.
The users who stayed got enough early runs to build a working model of the system's behavior. They'd seen it be right about things they could verify. They'd seen it admit uncertainty instead of performing confidence. They could stand on the ladder because the design put rungs there.
The corrective is counterintuitive. Reduce friction, says conventional wisdom — simpler prompt, cleaner UI. I tried it. In one system, removing structure and simplifying the first interaction collapsed first-task success. Users needed structure, not simplicity: examples of what a good run looked like, a first task designed to succeed rather than discover limits. The template isn't constraint. It's the rung at the bottom of the ladder.
Confidence signals, evidence trails, traceable provenance, a designed gate at every loop boundary — these aren't independent features. They're the architecture that makes the ladder climbable. Remove one and users stall on the rung below, or fall.
✦ ✦ ✦
Conclusion
The actor changed.
The job didn't.
Fifteen years. Four contexts. One problem.
A buyer committing six months of inventory spend on the strength of a screen. A pharma logistics system with $20 million of margin and a regulatory clock. A disaster-relief platform where the wrong caseworker decision was not a UX error. Now: an agent doing real work in a place the human can't watch, producing outputs people stake their reputation on.
Each time, the same four design jobs came back.
The upstream job
Design the intent layer. Structured intake, reflected interpretation, confirmed scope before the work starts.
The consistency job
Make reliability visible. Confidence at the decision point. Evidence beside synthesis. Traceability when things change beneath you.
The handoff job
Design all five transitions. Treat every loop boundary as a gate. The ones you don't design are the ones that compound.
The trust ladder job
Map where your users are, not where you want them to be. Build rungs for the rung they're on.
The engineering is accelerating. The human end of the contract — how we communicate intent, measure consistency, design handoffs, earn trust incrementally — hasn't followed. Not because it's harder. Because nobody has treated it as design work yet.
It is.
And the opening is still there.