Here is what I keep seeing in our portfolio conversations: the AI discussion at the IC level is almost entirely about capabilities. What can the model do? What workflows can we automate? What does the product roadmap look like? These are the right questions at the pilot stage. They are the wrong questions once you are in production.
The real question, the one that determines whether AI creates value or consumes it, is whether the operator can see, meter, and route inference spend by return.
Most cannot. And that gap is where AI economics fall apart.
The pilot bill does not predict the production bill.
When a company runs an AI pilot, the cost structure looks clean. You have a model license or an API cost that scales with test volume. The numbers in the IC memo are usually per-query estimates, and they are modest enough to pass without scrutiny. The pilot moves into production. Then the bill changes.
A chatbot call costs a few cents in tokens. An agentic workflow, the kind that actually does the work a portco hired AI to do, costs many times that. When a workflow has to reason across documents, take multi-step actions, call tools, verify outputs, and loop until complete, each task can consume 10 to 50 times the tokens of the original pilot benchmark, though the range varies significantly by workflow type and document complexity. That is not a rounding error. That is a different business.
The gap between pilot and production token consumption is one of the most consistent patterns we see across engagements. The pilot was built to demonstrate a capability on a clean, curated input. The production workflow has to handle edge cases, retries, verification steps, and coordination overhead that the demo never touched. Nobody benchmarks agentic tasks against production document variance. They should, because it is the benchmark that predicts the real bill.
There is also a structural reason the pilot misleads: the IC memo often compares AI cost to the alternative of doing nothing, or to the headcount cost of doing it manually. That framing makes the pilot look cheap. It does not capture the difference between pilot token volume and production token volume. Once you deploy a workflow that is running thousands of times a day rather than hundreds of times in a demo, the inference line item grows in ways the pilot model does not anticipate.
Inference becomes the majority line item.
Model licensing feels like a cost. Inference feels like a utility bill. That framing matters more than it should.
When a company negotiates a model license or a platform fee, there is a contract, a negotiation, a benchmark. The number is visible and contested. When inference runs in production, it accumulates in the background. It shows up aggregated in the cloud bill. If nobody is tracking it at the task level, nobody knows which workflows are profitable and which are not.
At meaningful agentic scale, inference often accounts for the majority of AI run cost, ahead of licensing, tooling, and headcount combined. That shift happens faster than most operators expect, because the growth curve for agentic task volume is steeper than the growth curve for the pilot. You do not go from one chatbot to two. You go from one chatbot to a dozen workflows running thousands of times a day, each with a token footprint many times larger than the single-turn interaction the pilot modeled.
This is the moment where the economic frame has to change. You are no longer buying a capability. You are running a metered service, and the unit of cost is the task. The question is no longer what can this system do. The question is what does it cost to run this task class at this volume, and does that cost pencil against the return.
The metering gap is where margin disappears.
Most of the companies we work with have some form of AI dashboard. They can tell you how many users are active, how many queries ran, maybe what the error rate is. What they cannot tell you is what it cost to complete any given task class, and what that task class returned.
This is the metering gap. And it is expensive.
Without per-task cost visibility, there is no way to know whether a given workflow has positive unit economics. A document-review agent might be generating value on complex contracts and burning money on simple forms. A data-extraction workflow might be priced at $X per run on the assumption that the model does most of the work in one pass, when in practice the retry rate on certain document types makes the actual cost three to five times higher.
The operator who cannot see this is making pricing and scaling decisions in the dark. They will scale the workflows that look busy, not the ones that are profitable. And the CFO who asks for an AI ROI number will get an answer built on volume, not margin. The AI program looks productive on the dashboard and confusing on the P&L. That confusion compounds as volume grows.
There is a simpler version of this problem that shows up earlier. When a portco does not track per-task inference cost, it also does not catch regressions. A prompt change that improves output quality by 5 percent but doubles token consumption looks like a win in evaluation and a mystery in the bill. Without the connection between task outcome and task cost, you cannot make that tradeoff consciously. You make it by accident.
Task-complexity routing is the first fix, and it pays immediately.
The correction here is not complicated in principle, though it requires some architectural discipline to implement. Not every task needs the most capable, most expensive model. Many tasks do not need a frontier model at all.
In almost every engagement where we have mapped task complexity against model selection, we find the same pattern: a large share of tasks are routine, low-ambiguity calls that a smaller, cheaper model handles just as well as the frontier model. The expensive model is being used everywhere because nobody built the routing logic to use it selectively. The default was the most capable option available, which is also the most expensive option available.
Routing by task complexity is the fastest lever for improving AI margin without degrading output quality. You keep the frontier model where it earns its cost: complex reasoning, high-stakes outputs, ambiguous inputs that require genuine judgment. You route the rest to a model that is an order of magnitude cheaper and fast enough that users do not notice the difference.
We worked with a financial services company running agentic workflows on a single model tier across all task types. After mapping task complexity against cost and output quality across their production workload, we implemented a two-tier routing approach: a smaller model handling classification, extraction, and structured data tasks; the frontier model reserved for analysis, recommendation, and anything touching regulated output. Inference costs on the routed workload fell substantially. Error rates did not move.
That is not a small improvement at the margin. At the volume they were running, it was the difference between a workflow that paid for itself and one that was a budget line item the CFO was quietly circling.
The routing logic itself is not the hard part. The hard part is having the per-task cost data to know where the routing boundary should be. That requires the metering infrastructure described above, which is the same infrastructure that surfaces the ROI picture in the first place. Metering and routing are the same problem viewed from two sides.
What the IC memo should be asking.
The question at the investment stage should not be: what does this model do? It should be: when this company scales this workflow to ten times the current volume, what does the inference bill look like, and who owns it?
If the answer is "we do not know yet" or "we will optimize later," that is the risk. Later does not arrive on a predictable schedule. The workflows will scale because they work, and by the time the bill is large enough to get board attention, the cost structure is embedded and the architectural changes required are expensive. Fixing metering post-scale is harder than building it pre-scale because you are now working against live systems with real users.
Operating partners and portfolio CFOs who have gotten this right treated inference cost as an operational metric from day one. They named an owner: not the CTO’s team as a side project, but a function with a target and a reporting line. They built cost visibility before they built scale. They set routing rules before the bill got large enough to require crisis management.
That is a different discipline from the capability conversation, and it is not a harder one. It is just a different frame: not what can we build, but what does it cost to run, and which runs are worth it. The same data set also changes provider negotiations: when you have per-task cost data, you know exactly where spend is concentrated and where you can push back. That is a different conversation than arriving with a pilot invoice and a roadmap.
The operators who will win here are the ones who meter first.
AI value creation in the next phase is not going to be determined by which company has access to the best models. The operators who build metering and routing discipline into their AI programs before scale will have the unit economics to show for it.
The operators who build this discipline are the ones who can tell you, at any point, which AI workflows are profitable and which are not. They can negotiate from data, grow what works, and cut what does not pencil. The ones who skip this step will find the AI budget growing and the ROI story getting harder to defend, not because the technology failed, but because the cost structure was never managed.
The inference bill is already your AI margin, and it is accumulating regardless of whether you are watching it.