Two facts that do not want to sit together. Token prices dropped by something close to 80% between early 2025 and early 2026. Over the same period enterprise spend on inference went up, and survey after survey finds most AI programmes overshooting their cost estimate by a third to a half.
The resolution is unglamorous: consumption grows faster than unit prices fall. But it leaves an operator with a question no pricing page answers — inside a contact centre that is actually running, which part of the bill is large?
We went looking in ours. Four findings, in the order they surprised us.
Almost none of it is the output
We broke a month of platform token usage into three buckets. The split was not close.
Nearly every cost-cutting instinct in the industry points at the bottom row.
Shortening answers. Capping response length. Telling the model to be concise. All of it operates on half a percent of the bill. The system prompt, the tool schemas, the retrieved context and the conversation history are the bill.
Two things follow. The highest-leverage work is confirming your provider’s prompt caching is genuinely being hit, because that 86% is only cheap while the cache holds — a change that quietly busts it does not look like an incident, it looks like a larger invoice. And the thing to review is not how the model writes. It is what you send it on every single turn.
A large share of turns are not questions
The second finding needed no instrumentation to understand. A lot of what arrives in a conversation is not a request for anything: greetings, thanks, “ok”, a thumbs-up, the acknowledgement at the end of a thread that is already resolved.
Every one of those was going through the full pipeline — retrieval across the knowledge base, the agent loop, then a frontier model composing a courteous reply. We were paying the most expensive component in the stack to recognise the word thanks.
There is now a classification step in front of it. A small open-weights model decides whether the turn is an acknowledgement or a real question. Acknowledgements return a signal instead of an answer, before retrieval or generation runs at all. Real questions pass through untouched.
Three details matter more than the idea itself:
- It fails open. Any error or timeout and the customer gets a normal answer. A cost optimisation that can break a conversation is not a cost optimisation.
- It costs latency — 350 to 500 milliseconds on the turns it classifies. That is a real trade, and worth stating plainly rather than burying.
- It was validated in both languages before it went anywhere near production. An acknowledgement classifier that only really understands English will quietly start swallowing questions in Arabic, and nothing in your dashboards will say so.
“Cheap” is not one price band
Choosing the model for that classifier took longer than building it, and produced the most portable lesson here.
Between three models that all qualify as small and cheap, the annual cost of running the same classification at our volume differed by a factor of five. Not five percent. Five times. Every one of them reads as inexpensive on a pricing page.
At low volume that gap is invisible and not worth an afternoon. At a million-plus turns a month it is a line item. The only way to see it is to multiply list price by your own measured token shape — ours averages about 431 input tokens and 2 output tokens per classification, a profile so lopsided that input pricing decides everything and output pricing is close to irrelevant.
Your shape will be different. That is precisely the point: the ranking of models by cost depends on your traffic, not on theirs.
Cheaper and better stopped being opposites
The old reflex was that cutting model cost meant accepting worse answers. That is no longer reliably true. Our most recent migration moved the platform onto a newer frontier model that took roughly 86% off the cost of a typical turn while winning every shared benchmark we compared the two on, with a context window more than twice as large.
The interesting decision was not the migration. It was the scope. We moved 492 prompts and deliberately left several categories where they were: voice and IVR, the query-rewriting step inside retrieval, media recognition, and the live chat of our largest customer — where the cost of a subtle regression is worth more than the saving. Summarisation and quality scoring for that customer moved. The conversations their customers actually see did not.
Move the parts where a regression is recoverable first. Let them run. Move the rest on evidence rather than on a benchmark table.
The failure mode nobody warns you about
The new model needed four configuration fields set together: model, provider, reasoning effort, temperature. Set three of them and the prompt does not fail. It works — and becomes quietly slower and more expensive.
That is the shape of almost every cost incident we have seen. Nothing goes down. No error rate moves. No customer complains. The bill arrives three weeks later.
So if there is one operational habit to take from this: alarm on cost per conversation, not on total spend. Total spend moves with traffic and hides everything inside it. Cost per conversation catches a misconfiguration in days instead of at the end of the quarter.
And the model is only one of the meters
Worth saying, because it surprises people in this region more than any of the above: there are two bills running, and they behave nothing alike. On WhatsApp, Meta charges per delivered message at a rate set by the recipient’s market and the message category, with graduated volume tiers — and for most of our customers that bill is the larger of the two. We built a calculator for it.
None of this is exotic. Measure your token shape before optimising anything. Don’t send expensive models work a cheap one can refuse. Price the cheap options against your own traffic, not the vendor’s example. Migrate the recoverable parts first. And put an alarm on cost per conversation, because in this domain the expensive mistakes are the quiet ones.