ENTERPRISE AI

What a run actually costs

Count the cost of an accepted outcome, not the cost of a token. Batch and cache change the hosted bill. Spare capacity is not free headroom.

Figure 1: Cost is accepted work, the full bill including review, and spare capacity you must not also sell.

1. A token is not an outcome

The useful unit is the cost of an accepted outcome. An accepted outcome is a result a person or a check kept. A cheap token price on a run that was rejected, retried, and then repaired by hand can be the expensive path. Attempted tasks are not the denominator. Accepted ones are.

The full bill includes the model, the review time, the retries, the capacity you hold in reserve, and the vendor minimums. A dashboard that shows tokens and hides the analyst’s afternoon is not a cost model.

2. Hosted and private fail in different places

Batch processing and prompt caching can change a hosted bill by a lot. They are not free features you assume are on. Cache eligibility, the minimum prefix, and expiry are provider-specific, and they belong in the measurement. A design that “will be cheap once we cache” is a design you have not priced.

Private infrastructure is priced from throughput, concurrency, availability, and spare capacity. N+1 means one extra unit beyond what peak load requires. That spare unit is the reserve. It is not unused capacity you can also sell as headroom once volume grows. If you consume it, you no longer have the spare, and the availability story was fiction.

Layman rule: Compare routes on cost per accepted outcome. A low token price can still be the costly system.

3. Quality still has a price

A private model that is cheaper per token and wrong more often can lose on accepted outcomes, because the repairs dominate. The reverse is also true: a frontier model that is glorious and rarely accepted is an expensive demo. Route by measured quality, latency, data class, and the full bill. Revisit the route when any of those move.

This page does not publish a dollar figure. A number from someone else’s cluster is not your bill. Measure yours. The finance masterclass makes the same point about a close cycle: a baseline you measured beats a benchmark you quoted.

4. The value has to be named

Cost without a value is only a spend. Name the value in the unit the operator already uses: cycle time, a class of errors that stopped, capacity the team got back, a risk that is now logged. If the only value sentence is “we use AI,” you cannot tell whether the route is worth it.

Ownership of the bill is a role, not a shared feeling. Someone watches utilization, the cache hit rate, and the cost per accepted outcome, and is allowed to say the private path did not pay.

5. What to price next

Take last month’s runs. Split them into accepted and rejected. Add the review time you can actually account for. That ratio is the first real number. Do not announce it as a savings until a second month agrees.

With a cost you trust, the build choice is one of six things a company can build.