Two years ago, if a CTO proposed running open-source AI models in production, the board would’ve asked uncomfortable questions about risk. Today, that same CTO would face uncomfortable questions if they weren’t evaluating open-source options.

The shift has been dramatic. But the conversation tends to swing between breathless enthusiasm and reflexive skepticism. Neither is useful. So let’s look at where things actually stand.

The current landscape

The ecosystem has consolidated around a handful of major model families.

Meta’s Llama 4 dropped in February and it’s the most significant open-weight release to date. The 70B version matches GPT-4o on most benchmarks, and the 405B version competes with the best proprietary models on reasoning. Meta’s permissive licence has made it the default starting point for enterprise deployments.

Mistral continues to punch above its weight. Their Mixtral mixture-of-experts architecture delivers strong performance at lower compute costs. Particularly popular with European enterprises wanting a non-US AI option — which matters more than you’d think in data sovereignty conversations.

DeepSeek remains the wildcard. Their R1 reasoning model competes with OpenAI’s o3 on mathematical and scientific tasks. The fact that it’s open-source and was trained for a fraction of the expected cost is still slightly unbelievable.

Alibaba’s Qwen 2.5 series handles multilingual tasks better than any other open model, making it the obvious choice for companies operating across language boundaries.

Then there’s the long tail — hundreds of specialised models fine-tuned for medicine, law, finance, and other domains. Hugging Face now hosts over 800,000 models.

What’s changed for enterprise

Three developments have shifted the calculus significantly.

Infrastructure got easier. Running a 70B model used to require deep ML ops expertise. Now, platforms like Together AI and Fireworks AI offer one-click deployment with enterprise SLAs. You get the benefits of open-source without building infrastructure yourself. For self-hosting, vLLM and TensorRT-LLM have made efficient serving almost plug-and-play.

Fine-tuning became accessible. This is the killer advantage. Take Llama 4 70B, fine-tune it on your company’s data, and you’ll get a model that dramatically outperforms any general-purpose API on your specific use case. Tools like Axolotl and Unsloth have reduced fine-tuning from weeks to days.

The cost math became undeniable. A company making 10 million API calls monthly to GPT-4o spends around $50,000 on inference alone. Running an equivalent open-source model on self-hosted infrastructure costs $8,000-12,000 after amortising hardware. Through an inference provider, maybe $15,000-20,000. Those savings scale linearly with usage.

The honest problems

Open-source AI isn’t a free lunch.

Safety and alignment vary wildly. Proprietary models from OpenAI, Anthropic, and Google have extensive safety testing. Open-source models have… some of that. Fine-tuned variants often have safety measures deliberately removed. Customer-facing deployments need your own safety evaluation.

No one to call when things break. When your OpenAI API returns garbage, you file a support ticket. When your self-hosted Llama deployment produces unexpected outputs, you’re on your own.

Keeping up with releases. A new state-of-the-art open model drops practically every month. Do you upgrade? How do you evaluate whether it’s actually better for your use case? That model management overhead is real and often underestimated.

Compliance burden. For regulated industries, the documentation requirements fall on you — which model version, what training data, how you evaluated for bias, how you monitor outputs. Doable, but it adds cost.

The hybrid approach most enterprises are landing on

In practice, most sophisticated enterprises aren’t going all-in on either option. They’re running both.

Proprietary APIs for customer-facing, high-stakes applications where safety, reliability, and support matter most. Think customer service chatbots, medical advice, financial advisory tools.

Open-source models for internal applications, high-volume workloads, and domain-specific tasks where fine-tuning provides a clear advantage. Think document processing, internal search, data extraction, code assistance.

This hybrid approach gives you the safety net of proprietary APIs where it matters most and the cost savings of open-source where you can manage the overhead.

What to watch next

The more important trend isn’t any individual model release — it’s the maturing ecosystem. Better evaluation frameworks. Better safety tools. Better deployment infrastructure. Better fine-tuning pipelines.

Two years ago, choosing open-source AI was a technical bet with significant risk. Today, it’s a mainstream option with a clear cost-benefit analysis. The question isn’t whether to use open-source AI. It’s which models, for which applications, with what governance framework.

That’s a much more productive conversation to be having.