Cheaper tokens have not made AI cheaper to run

Most data teams are drawing up their 2027 AI budgets now, and the price of a model call is the number most likely to mislead them. Gartner expects inference on a model with a trillion parameters to cost providers more than 90% less in 2030 than it did in 2025.¹ It also expects the inference cost of each agentic workflow to rise more than fivefold through 2028.² Both forecasts can hold at once, and the second is the one that reaches your bill.

The 2026 numbers already point that way. In the FinOps Foundation’s survey of 1,192 practitioners, 98% said they now manage AI spend, against 31% two years earlier, and FinOps for AI was their top priority for the year ahead.³ McKinsey’s 2026 State of AI survey found about one in five respondents saying that operating costs, token costs included, were limiting how much their organisation used AI.⁴

Cheaper tokens, bigger tasks

Gartner calls this the inference paradox: falling model prices end up paying for more complex work. As AI products move from answering a question to carrying out several steps, each task uses more tokens, and often more expensive ones. “Each successive generation of AI capability will necessitate more, and often more expensive, tokens,” said Will Sommer, a senior director analyst at Gartner.² McKinsey sees the same pattern inside organisations: per-token costs have come down while “the number of tokens consumed and generated has increased even faster”.⁴

Costs that never appear on the model bill

For a data team the model call is often not the largest line. Retrieval means embedding documents and storing the vectors, then paying for every search. Each change to a prompt or a model should be tested against an evaluation set, which means more calls. Keeping prompts and responses for audit adds storage. Most of this is billed to cloud and data platform budgets, so the full cost of an AI feature is hard to see in one place.

Cost will decide which projects survive 2027

Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, because of escalating costs, unclear business value or inadequate risk controls.⁵ The first two are questions of money. A team that cannot say what a task costs will struggle to defend it in the next budget round.

What I would put in the 2027 budget

Two numbers do most of the work. The first is cost per completed task, from the user’s request to an accepted answer, including retries, tool calls and any human review, broken down by route so you can see where a smaller model or a cached answer would do. The second is the share of AI spend tied to a named use case with an owner. The FinOps Foundation found mature practices focusing on quantifying the value of AI, and that starts with knowing what each task costs.³

Then plan for volume rather than price. If a feature moves from an assistant to an agent next year, assume its cost per task goes up, even if your provider cuts its price list.

At BIAAS 2027 we would like to hear from teams that have put a unit cost on an AI feature and changed a decision because of it, whether that meant moving to a smaller model or switching the feature off.