Most AI agents that break after launch don't break because the model isn't smart enough. They break because one model was given the whole job: reading the inputs, applying the rules, checking its own work and deciding where the result goes. The fix is almost always to give the model less to do.
Why do agents work in the demo and fail in production?
A demo runs on clean, hand-picked inputs, so a model that is right most of the time looks right all of the time. Production brings messy inputs, edge cases and volume. Around week four the output starts to be sometimes wrong, and nobody can tell which answers those are.
That last part is the real problem. An agent that fails loudly is easy to fix. An agent that returns a confident, well-formatted, wrong answer one time in twenty is much worse, because the wrong answers look exactly like the right ones. People stop trusting all of the output, and the team goes back to checking everything by hand.
The instinct at this point is to add more: a bigger model, a longer prompt, a retrieval layer, another round of examples. Sometimes that buys a few weeks. Usually it moves the error somewhere else, because the underlying design hasn't changed. The model is still being asked to do things it does probabilistically that the business needs done deterministically.
What is the anatomy of a task?
Every agentic workflow has four parts: a source (where the inputs come from), logic (what has to happen to them), an eval (how you know the output is right) and a destination (where the result goes and who acts on it). Writing those four parts down is the first step of every build we do.
Our forward-deployed engineers write this out before anyone opens an editor. A portfolio report that arrives by email, lands in SharePoint and gets enriched with market data is already a workflow; most teams just haven't named its parts. Once the parts have names, you can decide who owns each one.
In the five levels we use to describe AI maturity, this is where Level 3, Workflow Agents, begins: the steps of a task are mapped and moved out of a person's hands. It's also where most teams first hit trouble, because mapping the steps forces the question of which steps the model should be trusted with.
Where does the model belong, and where does code belong?
The model belongs at the source end, where inputs are messy and understanding context matters. Logic that has to come out the same every time belongs in code: rules, calculations, matching, validation. Probabilistic where it helps, deterministic where it counts.
Models are very good at reading. They parse a scanned PDF, find the right field in a layout they've never seen and understand a vendor's note in the margin. A rules system would choke on all of that. But models are a poor place to put a calculation that must be identical on every run, or a decision that has to match a policy exactly.
So we layer them. Traditional code sits inside the agentic workflow and does the deterministic work, and the model hands it clean, structured inputs. On one monthly reporting workflow, adjusted EBITDA isn't computed by the model at all. A rules engine computes it, inside the same workflow, from figures the model extracted. The model never decides what the number is.
This is what we mean when we say hallucinations disappear by design. It isn't a claim about the model getting better. It's a claim about scope. A model with a narrow job and no room to invent an answer doesn't invent one. Models exaggerate when they're given too broad a brief and asked to solve the whole problem at once.
What this looks like on an invoice-matching build
The agent reads each invoice and extracts the vendor, purchase order number and line items into a fixed schema. A rules engine decides whether each line matches the purchase order. Matches post to the ERP; exceptions go to a person with the reason stated.
Accounts payable is a good example because it's boring, high volume and unforgiving. Invoices arrive in a dozen layouts. Some are scans. Vendors abbreviate things differently every month. That variety is exactly what the model is good at, so the model owns extraction.
Matching is different. Whether a line is within tolerance of the purchase order is a rule, and the finance team needs the same answer on Tuesday as on Friday. So matching is code. So are the checks after it: totals reconcile, required fields are present, and every record carries a logged confidence. Anything that fails a check goes to the AP team's queue with the reason attached. The model is never asked to guess a match, so it never does.
The result is a workflow where the expensive failure, a plausible wrong match posted to the ledger, can't come from the model. Errors still happen, but they happen where people can see them.
Why more model, more prompt and more retrieval don't fix it
Adding capability to a model that has too much to do makes it wrong less often, but no less silently. The failure that hurts is a plausible wrong answer. Only a smaller job, with checks in code, removes it.
We've watched teams build impressive agents in house and then hit a gate. The first 80 percent of the value came quickly. The last 20 percent, the edge cases and the consistency, needed a different kind of design: breaking the task into layers, giving each layer one job, putting evals on the results and re-aggregating. That's the work of a forward-deployed engineer, not a better prompt.
It's also why "which model should we use?" is usually the wrong first question. The system around the model, meaning how the task is broken up and checked, decides most of the reliability. A strong model in a weak system still produces confident errors.
How to scope your next agent
Write the workflow out as source, logic, eval and destination. Give the model only the steps where reading and context matter. Move anything that must be consistent into code, and put a check in front of the destination. Then build.
- Name the four parts of the task in plain language, with the systems involved.
- Mark each step probabilistic (reading, judgment) or deterministic (rules, math, policy).
- Give the model the probabilistic steps and nothing else. Have it return structured output.
- Put deterministic steps in code that sits inside the workflow, and test it like code.
- Define the eval before launch: what does a wrong output look like, and what catches it?
- Route every failed check to a person, with the reason, rather than letting the agent retry until it passes.
If you want a faster version of this, we use a four-line test that fits on one page: The Four-Line Test. And if your team is already reaching for retrieval and vector databases, read Do You Need RAG? first.