The LLM inference cost curve has been one of the most consequential numbers in AI economics for three years. It is worth a focused look at where the numbers actually are in May 2026 because the curve has flattened in some places, accelerated in others, and the production AI cost model is more nuanced than the headline price drops suggest.

The high-level number first. Inference cost per million output tokens on flagship frontier models is around 10–15x cheaper than it was in May 2023 for equivalent reasoning quality. The drop on cheaper tier-1 models is dramatically larger — for the use cases where a mid-capability model is sufficient, the cost reduction is closer to 50–80x over the same period. The story below the headline is more interesting.

What has flattened.

Frontier-quality reasoning cost reduction has slowed materially through 2025 into 2026. The drop from May 2024 to May 2025 was much larger than the drop from May 2025 to May 2026 on equivalent tier-1 capability. The economics of pre-training and inference for frontier-quality reasoning models have hit a band where the major lab pricing is converging rather than dropping. The bigger lever is becoming inference-time technique — speculative decoding, mixture-of-experts routing, distillation onto smaller models — rather than further cost reduction on the largest models.

The cost of long-context reasoning at frontier quality has only modestly improved. Reasoning over 200k–1M token contexts still costs noticeably more than reasoning over 8k token contexts, and the cost gap has not closed as fast as some early forecasts suggested. The cost-per-million on long-context calls is the line item most worth modelling carefully on production AI applications.

What has accelerated.

Smaller specialised models have dropped in price dramatically. The cost of distilled domain-specific models running on hosted inference is now in a range where document classification, structured extraction, basic summarisation, and routine reasoning workloads can run at sub-cent per call at production scale. This has changed the economics of a number of production AI architectures.

Open-weight model inference has matured. Self-hosted inference of mid-capability open-weight models on cloud GPU has reached price-performance that is competitive with hosted closed-model inference for the right workloads. The right workload is roughly: high-volume, latency-tolerant, where the developer team has the engineering capability to run inference infrastructure. The wrong workload is bursty, latency-sensitive workloads where the cloud-vendor hosted closed-model service still wins on operational simplicity.

What this means for production AI economics.

Production AI architectures in May 2026 are converging on a tiered pattern. A high-volume, well-understood workload runs on a cheap, fast, smaller model — frequently a distilled or quantised model on dedicated infrastructure. A complex reasoning workload routes to a frontier model. A small percentage of edge cases route to the most expensive frontier-with-reasoning capability. The combined cost per call is much lower than running everything on the frontier model, and the latency for the high-volume path is much better.

Caching has become a serious cost lever. Prompt caching, response caching for repeatable queries, and partial-result caching are saving meaningful budget on production AI applications. The teams that have done the engineering work on caching are running production AI workloads at materially lower cost than the teams that have not.

Evaluation cost is the surprise line item. The unit economics conversation in 2026 increasingly includes the evaluation and monitoring cost of production AI systems — running test cases regularly against production prompts, monitoring drift on output quality, evaluating new model versions against the existing workload. The evaluation infrastructure cost can easily run 5–15% of the production inference cost at scale and is rising as a share.

A note on procurement. The procurement conversation between enterprise buyers and the major inference providers has matured. Enterprise customers are signing committed-use contracts with discount tiers, multi-region pricing, and SLA commitments that did not exist in 2023. The list-price-per-million number is the worst guide to what enterprise customers are actually paying.

Looking forward to the end of 2026, the curve looks like it will continue to flatten on frontier reasoning quality and continue to drop on smaller and specialised models. The architectural decision that pays back fastest in this environment is to design production AI workloads for cost-aware model routing from day one, rather than treating all calls as if they cost the same. Production AI cost engineering is becoming a discipline of its own.