Here's a question I ask every operating team we work with: if you switched AI models tomorrow, what would you lose?
If the honest answer is "most of it," you haven't built your AI. You've rented it. The prompts, the definitions and the rules for what counts as revenue all live inside one tool, and they aren't written down anywhere else.
That question got a lot more practical in the last two weeks. The price of a narrow AI decision fell to almost nothing. Anthropic removed a setting that many teams relied on for consistent answers. And Anthropic's own Applied AI team told companies to build fewer agents. Each of those is a change to the model layer. None of them should have to touch your business logic. This post is about how to make sure it doesn't.
What changes, and what shouldn't?#
The model, its price, its settings and the vendor's API all change every quarter now. What counts as revenue, your approval thresholds, your match rules and your metric definitions shouldn't change when they do. The work is keeping those two lists apart.
Look at what moved in October alone. OpenAI opened its Decisions API to all developers, a model built to pick from a fixed set of answers instead of writing text. In a public test by OpenRouter, TypeSafe's Jev sorted 3,080 banking support messages at about 1/22 of what Claude Opus cost. Anthropic's new API version removed temperature, the dial teams turned down to get the same answer more often, and said plainly that no other setting replaces it.
If your business logic lives in prompts tuned to one model at one setting, every one of those changes is a migration project. If it lives somewhere you own, each one is a configuration change.
What is a context layer?#
A context layer is where your business logic lives in a form an AI agent can call: the definitions, rules, calculations and thresholds the agent uses instead of working them out itself. It's usually an MCP server or an API, and any model can call it.
MCP, the Model Context Protocol, is the standard plug that lets an AI model use another system as a tool. An API does the same job in a more traditional way. How it connects doesn't matter much. What matters is that the logic is written down, tested and owned by your team, in one place, outside the model.
The agent still does the part models are good at. It reads messy documents, pulls the right figures, understands what a vendor meant in a free-text field. When it needs to apply a rule, it calls the context layer and gets back an answer that comes out the same every time.
What does that look like in a build?#
On one portfolio reporting workflow we built, the model never decides adjusted EBITDA. A rules engine computes it, and the definition sits in a context layer the agent calls like any other tool. Point a different model at it and you get the same number.
Adjusted EBITDA is a good example because every company defines it a little differently. Which one-time costs get added back, how owner compensation is normalized, what counts as a run-rate adjustment. Those are decisions the finance team made, often years ago. They're business logic, not language work.
In the first version a lot of teams build, those rules end up in a prompt: "When calculating adjusted EBITDA, add back the following..." It works in the demo. Then someone edits the prompt to fix an unrelated issue, or the model gets upgraded, and the number moves by a rounding error nobody catches for a quarter.
In ours, the agent gathers the inputs from the source documents. It hands them to a calculation the finance team can read, test and sign off on. The model can change underneath and the number doesn't move. When the definition does change, the finance team changes it in one place, and every agent that calls it gets the new rule.
Isn't this just a data warehouse?#
It's the closest comparison. A warehouse's data model is an opinionated view of your business, and that opinion is the asset, not the database under it. The comparison breaks on cost: a context layer covers one use case at a time and usually pays for itself in the time it saves.
The companies that got the most out of their data warehouses weren't the ones with the best database. They were the ones that agreed on what a customer is, what counts as a booking and which number goes in the board deck, and wrote it down. The context layer is the same idea, pointed at agents instead of dashboards.
The difference is that you don't build it all at once. You build the slice one workflow needs, usually a handful of definitions and a few rules, and you add to it as the next workflow needs more. It isn't a two-year migration.
Why does this matter more now than a year ago?#
Because prompts are becoming the new spreadsheet. AI, like Excel, is one of the most useful tools a company has. Used badly, both become the place where information goes to die: logic one person wrote, in a file nobody else opens, that the quarter quietly depends on.
Every finance leader has a story about the spreadsheet that ran the close, built by someone who left. The formulas worked. Nobody could tell you why, and nobody wanted to touch them. Prompts are heading the same way. They hold real decisions, they change without review, and they're tied to one model at one setting.
That's not an argument for slowing down on AI. It's an argument for putting the logic somewhere you'd be comfortable showing an auditor, and letting the model do what it's good at.
Does this mean fewer agents?#
Often, yes. Anthropic's Applied AI team now recommends folding most subagents into one main loop with tools and skills. The scoping doesn't go away. It moves out of the org chart and into the tools, and a rules engine in the context layer is one of those tools.
In their October 22 guidance, Anthropic says a single agent loop, loading skills and tools only when it needs them, usually beats a stack of specialist agents on quality, cost and speed. One customer's single loop ran about twice as fast as its multi-agent setup. Every handoff between agents is a place where context gets dropped.
A lot of first architectures draw a box for every function and put an agent in every box: one for finance, one for compliance, one for reporting. Often the "finance agent" exists to hold the finance rules. Move those rules into the context layer, and the main agent can call them directly. Keep a separate agent only when you can name what it needs that the main loop can't have, like work that has to run in parallel or a context window that would overflow.
The same guidance has a warning worth repeating. When you fold an agent in, run each test task more than once and look at which tasks changed, not just the average score. Some breaks only show up every other run.
How do you start?#
Pick one AI workflow and list every rule it applies. Mark where each rule lives today. Move the one that would hurt most if it changed silently into code the owning team can read, and have the agent call it. Then run the workflow on a second model and compare.
That last step is the test. If the outputs that matter come back the same on a different model, you own that workflow. If they don't, you've found the next rule to move.
You don't have to do it everywhere at once. Most companies we work with start with the one number that goes to the board, or the one approval that moves money. The model will keep changing. Your business logic shouldn't have to.
For the cost side of this shift, read What an AI Decision Costs Now. For why consistent answers aren't the same as correct ones, read Repeatable Isn't Right.