1. A token is not an outcome
The useful unit is the cost of an accepted outcome. An accepted outcome is a result a person or a check kept. A cheap token price on a run that was rejected, retried, and then repaired by hand can be the expensive path. Attempted tasks are not the denominator. Accepted ones are.
The full bill includes the model, the review time, the retries, the capacity you hold in reserve, and the vendor minimums. A dashboard that shows tokens and hides the analyst’s afternoon is not a cost model.
2. Hosted and private fail in different places
Batch processing and prompt caching can change a hosted bill by a lot. They are not free features you assume are on. Cache eligibility, the minimum prefix, and expiry are provider-specific, and they belong in the measurement. A design that “will be cheap once we cache” is a design you have not priced.
Private infrastructure is priced from throughput, concurrency, availability, and spare capacity. N+1 means one extra unit beyond what peak load requires. That spare unit is the reserve. It is not unused capacity you can also sell as headroom once volume grows. If you consume it, you no longer have the spare, and the availability story was fiction.
3. Quality still has a price
A private model that is cheaper per token and wrong more often can lose on accepted outcomes, because the repairs dominate. The reverse is also true: a frontier model that is glorious and rarely accepted is an expensive demo. Route by measured quality, latency, data class, and the full bill. Revisit the route when any of those move.
This page does not publish a dollar figure. A number from someone else’s cluster is not your bill. Measure yours. The finance masterclass makes the same point about a close cycle: a baseline you measured beats a benchmark you quoted.
4. The value has to be named
Cost without a value is only a spend. Name the value in the unit the operator already uses: cycle time, a class of errors that stopped, capacity the team got back, a risk that is now logged. If the only value sentence is “we use AI,” you cannot tell whether the route is worth it.
Ownership of the bill is a role, not a shared feeling. Someone watches utilization, the cache hit rate, and the cost per accepted outcome, and is allowed to say the private path did not pay.
5. What to price next
Take last month’s runs. Split them into accepted and rejected. Add the review time you can actually account for. That ratio is the first real number. Do not announce it as a savings until a second month agrees.
With a cost you trust, the build choice is one of six things a company can build.