Tokens have become the familiar unit on AI invoices. Teams estimate prompt size, compare prices per million tokens and celebrate cache discounts. It is tempting to call the token a new currency. That metaphor captures an important change: AI work has a metered marginal cost. It also obscures the question a business actually needs to answer: what does it cost to produce one result we can use?
A token is a unit of model input or output. It is not a word, a fact, a minute of expert work or a completed task. Models can split the same text into different numbers of tokens. Anthropic notes that a newer tokenizer in some Claude models may produce roughly 30% more tokens for the same text than its previous one, depending on the workload. A price per million tokens therefore does not guarantee an equal bill for the same document across providers. Anthropic pricing notes (opens in a new tab).
Read the invoice as a process
The basic model charge is input tokens times the input rate plus output tokens times the output rate. In practice, a workflow may also pay for cache writes and reads, search, OCR, tools, longer contexts, a faster service tier, regional processing and failed attempts. OpenAI (opens in a new tab), Google (opens in a new tab) and Anthropic (opens in a new tab) each publish different conditions. Output is often more expensive than input. A shorter prompt that produces a sprawling answer can cost more than a clear instruction that produces a short, valid record.
Falling model prices matter. The Stanford AI Index 2025 (opens in a new tab) estimated that the price of querying a model at GPT-3.5 level on MMLU fell more than 280-fold between November 2022 and October 2024. That is a historical comparison at one capability threshold. It does not say every present-day workflow became 280 times cheaper. Organizations also ask models to process more material, use more tools and carry out longer tasks.
Budget for accepted cases
Suppose a cheap extraction call rarely fails but occasionally needs an employee to check or repair the output. The model can be a small part of the true variable cost. The useful numerator is model calls plus tools, retries, review, corrections and allocated operating costs. The denominator is the number of outputs accepted against a written rubric. Alongside that figure, track elapsed time, serious errors and exceptions. A low price for an unusable answer is not an efficiency gain.
Start with one defined workflow. Record real input and output sizes, cache hits, retries, human review time and an acceptance rule. Compare candidates on the same cases, including the difficult tail. A median request may be cheap while unusual cases dominate the budget.
Why the billing unit cannot be the value unit
The price of a token resembles the price of a cloud-server minute or an OCR page: it describes the use of one component. A company buys the ability to complete work. Between component and outcome lies a chain: acquire data, prepare context, call the model, verify the result, take an action and observe its consequences. The weakest link determines the economics. If a worker must still reread every source document because the answer is opaque, even a correct-looking output may save little time. If a rare failure is costly, average model charges conceal the risk.
Separate the marginal cost of another case from the full cost of ownership. Marginal cost includes variable API charges and case-specific human work. Full cost adds design, integration, evaluation data, maintenance, access control, prompt revision and staff training. Fixed investment must be assigned to a realistic volume and time horizon. A pilot that ignores it can look profitable; a large deployment can become expensive when the exception rate rises.
Compare with a real alternative
AI creates value relative to how the work would otherwise be done: a person, a deterministic rule, a form template, a search tool or the removal of an unnecessary step. The baseline must use the same acceptance rule and include the same difficult cases. Otherwise an apparently excellent AI result may simply reflect easier input.
In Generative AI at Work (opens in a new tab), researchers studying customer-support workers found productivity improvements in the organization they observed, with a much larger effect for novice workers than experienced ones. This is not a promised return for another company. It shows that the economic effect depends on the workforce, task mix and the way assistance transfers effective practice.
The “jagged technological frontier” experiment (opens in a new tab) adds an important caution: AI capability is uneven across neighboring professional tasks. A team should not budget from a single claim that “AI makes analysts faster.” It should map the operations on which AI is reliable, the operations where it prepares a useful draft, and those where it currently increases risk.
A practical unit-economics worksheet
For each case category, record frequency, baseline human time, automatically accepted share, review share, correction effort, tool charges, model charges and the severity of failure. Evaluate normal volume, a surge and a shift in case mix. A plan that works only with perfect cache hits and no exceptions is too fragile for budgeting.
Be explicit about the type of benefit. Is it lower paid labor, more capacity without extra hiring, shorter customer wait, fewer errors or more expert time for difficult work? These benefits should not automatically be added together. If the employee spends the “saved” time checking an AI result, time saved and quality gained may be claims on the same resource.
Also value a correct refusal. A system that says “the evidence is insufficient; send this to a specialist” has consumed tokens but may have avoided an expensive mistake. An acceptance metric that rewards only full automation quietly pushes the system to answer when it should stop.
Running example: incoming requests at a growing retailer
Imagine a retailer handling delivery questions, returns and order changes. Today an employee reads the message, looks up the order, checks policy, replies and sometimes changes a record. “Use AI for replies” is too broad to cost. Split the flow. A delivery question with a known order status is regular and checkable. A disputed return requires interpretation and approval. Changing an address after handoff to a carrier may require several systems and might not be reversible.
Define the outcome for each category. For delivery: “the customer receives the correct current status and applicable policy, without a promise the retailer cannot keep or a repeat contact caused by the answer.” For a disputed return, success may instead be “a recommendation with the contested facts, reviewed by an authorized employee.” Full automation is not required for the system to create value.
Keep a case ledger in the pilot: category, sources, model and version, tool calls, actual usage, human review, corrections, closure time and customer outcome. Group cost by cause. If the most expensive part is locating order status, changing the language model will not solve it; better data access might. If verbose drafts dominate output cost, change the answer format. If disputed returns consume reviewer time, improve escalation and the evidence packet.
The ledger supports a sensitivity analysis. What happens if volume rises, the share of difficult cases changes, a provider increases its rate, cache hits decline or reviewers correct more outputs? The aim is not to forecast every variable precisely. It is to identify which one drives the business case and where a limit is needed. A robust workflow should survive plausible departures from an ideal pilot.
Distinguish realized value from potential
Report separately the observed reduction in active employee time, capacity actually released, improvement in customer outcome and financial effect. They are connected but not equivalent. Faster drafting may reduce a queue or support growth without another shift; it is not automatically a reduction in payroll. Better answers may have commercial value, but estimating that value requires a separate link to retention or repeat contacts.
Do not erase employees’ knowledge from the calculation. Experts maintain the policies on which the system relies, handle exceptions and teach new staff. Success may move their work from routine replies to knowledge maintenance and quality control. That can be a strong result, provided the new work appears in the operating budget.
The “new currency” becomes a more accurate metaphor here: an organization exchanges compute, data and human attention for reliable completed work. A token is one meter of that exchange, not its final price.
A budget that survives a model change
Separate work volume from processing price. Volume means the expected number of cases by type, how often tools are needed and the share of difficult exceptions. Processing price means actual token use, published rates, review time and correction cost. When a provider changes, update rates and tokenization profiles without rewriting the business case. When demand changes, update case mix rather than pretending every message is identical.
Account for processing mode. Batch may suit overnight document sorting but not a live customer conversation. Caching may reduce repeated instruction cost, but its value depends on stable input. Long context and specialist tools may have separate tariffs. A worker's subscription to a chat product is not equivalent to an API integration with access controls, logs and limits. Both can be useful at different stages, but they are not one interchangeable “AI cost.”
Use three conditions in the forecast: floor, base and stress. The floor is ideal execution with minimum calls and no corrections. The base uses observed pilot behavior. Stress introduces more unusual cases, fewer cache hits and more human checks. The purpose is not to guess every future model price. It is to see whether the workflow remains viable under plausible deterioration and where an expensive chain should be stopped.
Describe the limits of the evidence beside the financial estimate. An English-only pilot cannot establish the same result for Portuguese or Russian. Immediate expert review in a trial does not reproduce a production approval queue. Historical cases without policy changes do not measure maintenance after launch. Honest boundaries make a decision more reliable than a precise-looking but opaque return-on-investment figure.
Finally, error cost is often nonlinear. Ten awkward greetings and one unauthorized refund are not eleven equivalent defects. Set risk limits and a human decision path for high-consequence categories before calculating savings. Compare costs only among configurations that satisfy those limits. This prevents a small average saving from being purchased with a rare expensive failure.
The token is a good meter of consumption. The economic unit of AI work is an accepted outcome, delivered on time with tolerable risk.
Next: The economics of the context window.