The “we will distill our way to cheaper inference” story was loud through 2025. In mid-2026 the pattern of where distillation actually pays back is much clearer than the marketing implied, and the answer is more nuanced than either the bull or the bear case suggested.

The places distillation is working in production in 2026:

Single-task, single-domain workloads with stable input distributions. A customer support classifier, a contract clause tagger, an invoice line-extractor. The teams that took a frontier model, used it to generate a labelled dataset, and trained a 1-3B parameter student on it are running those students at a fraction of the frontier cost with accuracy that holds inside the original distribution.

Latency-sensitive paths in a multi-step agent. The teams running production agents are not running every step through a frontier model. They are routing the easy steps — intent classification, simple extraction, lightweight tool selection — through distilled students, and reserving the frontier model for the steps where the reasoning load justifies the cost.

On-device and edge workloads. The Australian regulated-data teams that need to keep inference local — primary care providers, defence-adjacent vendors, some financial services — are running distilled models on local hardware where a frontier API call would not be acceptable.

The places distillation is not working:

Open-ended reasoning. A student model distilled on a frontier teacher does not match the teacher on out-of-distribution prompts. The brittle behaviour shows up the first time a real user asks something the training set did not anticipate.

Agent self-correction. The reasoning loops where the model needs to recover from its own error tend to fall over in the student. The teacher’s slack in these moments is hard to compress.

Anything where the input distribution will shift. The student is locked to the world it was distilled in. Six months later, when the product expanded, the student is stale.

The practical 2026 read for engineering teams:

A two-tier inference stack — a small distilled student for the hot path, a frontier model for the cold path with the student as a router — is the architecture pattern that is settling out. Teams running this pattern are reporting 60-80 percent cost reductions on real production workloads without a measurable quality drop on the user-facing metric.

A pure-distilled stack is rare in production and the teams attempting it are mostly reverting.

For 2026 procurement, the question to ask a vendor is not “do you distill” but “what is your two-tier strategy, and what is the eval framework for the routing decision.” The vendors with a clean answer are the ones whose unit economics will work at scale. The vendors who do not have an answer are the ones whose pricing will rebase upward over the next two years.

Distillation in 2026 is not the silver bullet of 2025. It is a serious technique with a clear place inside a serious inference architecture. The teams who got that distinction right earlier are the teams whose AI products are now profitable.