An agent demonstration usually shows the finished result. Production economics live between request and result. The agent searches, reads, calls tools, writes intermediate outputs, retries, perhaps escalates, and then hands work to a person. Each step can add cost, latency and another chance for error. Counting the first model call alone systematically understates the workflow.
Five lines in the operating budget
Inference: all model input and output, including intermediate agent calls. Google's pricing notes explicitly include inference during managed agent loops. Google API pricing (opens in a new tab).
Data and tools: search, OCR, retrieval storage, paid grounding, external APIs and cache storage. These may be charged per call, page or hour rather than per text token.
Reliability: retries, validation, observation, recovery from tool failure and tests after changes.
People: review, exception handling, support and training. Minutes nominally saved do not automatically become cash savings if the process and staffing stay the same.
Risk and control: permissions, audit trails, re-evaluation, stop rules, a manual path and the ability to undo or contain an action. These are part of a safely operating system.
Put limits on agent behaviour
Consider an agent for incoming customer requests. First it classifies the topic and retrieves the current policy. Next it drafts a reply with a verifiable source. Then it can propose a standard reply for employee approval. Permission to change an order or promise a refund comes only after separate tests of that action, with amount limits and a recovery path. At every stage, compare acceptance, employee time, exceptions and complete cost. The economic case grows with proven authority, not with a single claim that “the agent pays for itself.”
Set a maximum number of steps, retries, input size and spend per case and per user. Define conditions that stop the agent and return control to a person. Report average and tail cost and latency, not just a smooth monthly average. If a strong model handles only unusual cases, test whether the router actually identifies them rather than quietly sending difficult work to a cheaper model.
METR's expenditure-horizon proposal (opens in a new tab) compares human and agent performance as a function of total expenditure, including compute and labor in its studied setting. Its specific estimates do not transfer to customer service, but the measurement principle does.
Financial cost and energy use are also different measures. The Microsoft Research analysis (opens in a new tab) shows how per-query energy estimates depend on system assumptions and test-time computation. An unsupported universal “energy per prompt” figure is not a sound procurement metric. For self-hosting, measure energy per accepted task under the real workload; for APIs, ask suppliers for the measurement boundary.
A weekly owner dashboard can show case volume, accepted-without-edit rate, escalations, serious errors, time to acceptance, model/tool/human spending and cost per accepted case. Model updates, price changes and new data sources trigger a new comparison. The process owner needs authority to stop automated action when evidence changes.
An agent has different economics from one chat response
A single chat call usually has a visible input and output. An agent creates a sequence in which the result of each step changes the next request. It may repeatedly read history, pass oversized tool results and revise its plan after failure. Costs and elapsed time are therefore variable and connected: a poor search can trigger more searches; an excessive tool result inflates later context; a timeout starts a retry. Average price per model call cannot explain the cost of a completed case.
Divide the run into budgeted stages: understand the request, find evidence, prepare a decision, validate it, act and confirm the resulting state. Each stage receives a limit on time, calls, spending and authority. At the limit, the system should hand the case to a person with a compact history and the sources found so far, rather than “trying harder” indefinitely. That is both a financial limit and a quality safeguard.
Authority should follow evidence
Level one is observation and classification. Level two is a proposed draft. Level three is a reversible action prepared for human approval. Level four is a bounded reversible action in a proven, familiar case. Error cost and requirements for tests, audit and recovery rise at each step. A single metric such as “tasks automated” obscures this: a drafted reply and an executed payment do not carry the same authority.
The 2026 AI Index technical chapter (opens in a new tab) documents agent improvement on structured benchmarks alongside a substantial failure share. Public tests show a direction of progress, while authority in a company must be based on its own tasks and errors. Static questions do not test action sequences; AgentBench (opens in a new tab) was among the studies designed for interactive environments.
Time, queues and bottlenecks
An agent can reduce employee effort while worsening customer time if it spends too long trying tools. The reverse is also possible: a quick draft waits in an approval queue. Measure active human time, queue time, execution time and total time to case closure separately. If human review remains mandatory, higher volume can turn review into the new bottleneck. The model's savings may simply move the queue.
Batch and lower-cost service tiers may suit background classification or periodic reports. A customer conversation may justify a faster, more predictable tier. Decide from deadline, the value of time and repeat-contact rate. Do not transplant a tariff optimization from one mode of work into another without testing service quality.
What to buy, build and keep deterministic
A model API can often be a replaceable component. Unique data, permissions, acceptance criteria and action logs belong to the organization's system. A rule such as “do not promise a refund without an order status and amount limit” is better enforced deterministically. The model can interpret the message and prepare an explanation; the final action condition should be reproducible.
When considering an agent platform, ask for step-level logs, actual tokens and charges, step limits, model versions, tool-disable controls and audit export. A platform showing only “cost per task” without a breakdown is hard to improve or govern. Building in-house offers control but requires a team, maintenance and evaluation; those fixed costs belong in the business case too.
A staged pilot
Choose a narrow, frequent workflow with checkable outputs and known exceptions. Start on historical cases with no live actions. Continue in shadow mode on current work. Then allow proposals for human approval. Only after separate testing grant bounded reversible actions. Each transition needs evidence on quality, cost, time and serious failures. If the threshold is missed, improve the route or return to a lower authority level. A successful pilot works on an ordinary day and during an exception, not just in a compelling demonstration.
Running example: locating the return on investment
Return to the retailer. The first agent project answers routine delivery questions using the order system and an approved policy. If employees accept most drafts quickly, active preparation time falls. That is not yet a cash saving. Did the queue shrink? Can rising volume be served without another shift? Did repeat questions fall? If not, the value may still be service quality and resilience rather than direct labor reduction.
Address changes form a different project. The agent could save more employee time, but a mistake can send a parcel to the wrong location. Identity verification, carrier status, a time limit and customer confirmation become part of the workflow and its cost. If manual address changes are rare and quick, full automation may not pay back; assembling the facts and a proposed action for an employee may be the better point. Disputed returns may justify a strong model for document analysis while keeping the final decision with a person.
This creates a portfolio of tasks, not one “agent ROI” number. On one operation the agent completes the case; on another it retrieves information; on a third it assembles evidence. All can create value through different economic mechanisms. Report by category so success on easy high-volume requests does not hide costly mistakes on rare actions.
Common ways an agent leaks budget
It can loop on a failing tool, repeat nearly identical searches, reread a large document instead of using a verified result, ask a strong model to rewrite an already accepted draft, accumulate logs in context, or retry a paid action after a timeout without checking whether the first attempt succeeded. These patterns create both charges and the possibility of duplicate action. They call for idempotent tools, safe result reuse, step limits, state checks before retry and logged escalation reasons.
Some costs cannot be fixed by selecting a model. Poor knowledge-base quality produces repeated search. Unclear authority creates human approvals. An unstable API forces the agent through a fragile interface. An economic audit should distinguish model, information and organizational causes of loss. Improving data or process can produce more value than a cheaper token.
The operating contract
Before release, write a concise contract between process owner and agent: authorized tasks, authoritative sources, available tools, actions requiring approval, budgets, how uncertainty is reported and who owns incidents. Version it. When policy or API changes, update both contract and evaluation set. Governance becomes a daily engineering and operational practice, rather than an annual document detached from the system.
If the organization cannot name the owner of exceptions and the person allowed to stop automatic action, it is too early to expand autonomy. A successful agent is not one that never hands work to people. It hands it over in time, with sufficient context and clear responsibility.
One agent or a team of agents?
Splitting work into a “researcher,” “writer” and “reviewer” can clarify roles, but every additional agent adds calls, context handoffs and another failure path. Two models checking each other are not automatically independent; they can share the same wrong premise. Justify a multi-component scheme with a specific test: it must improve accepted outcomes or reduce risk enough to cover coordination cost and delay. A simple job often needs one model call, deterministic validation and a person for exceptions.
A route can also combine components other than language models. The right tool for order status is the order system's API; for an amount, code with a testable formula; for a policy, a versioned knowledge base; for a clear customer explanation, a language model. This composition reduces what the model must infer and makes the outcome more reproducible. If the financial report calls all of it “agent tokens,” the team cannot see which component needs improvement.
From pilot to investment decision
Scaling requires two separate checks. Technical: does the system pass quality and control standards on normal and difficult cases? Economic: does it create realizable value after all variable and fixed costs? Passing the first does not guarantee the second; a safe useful agent may still be too expensive for a high-volume flow. Failing technical review cannot be offset by projected savings.
For the financial case, choose a time horizon and expected volume. Include integration, monthly maintenance, knowledge-base updates and an incident reserve. Then separate benefits: labor, extra capacity, quality and customer waiting time. Avoid counting one benefit twice. Show a break-even boundary by volume and acceptance rate, alongside conditions that would stop automation. Even without precise future model prices, this reveals which variables drive the decision.
Expand by category. First add more familiar cases, then new channels, then new actions. Every expansion changes data, exceptions and authority, so earlier quality evidence carries over only in part. A responsible owner must be able to return to a previous authority level without interrupting customer service.
The series' governing idea is simple: tokens meter model use, context is a managed resource, models are components, and efficiency exists at the level of the complete workflow.
Start the series: Is the token the new currency of AI?.