Why you should arbitrage your AI models like your vendors do

Enterprise software vendors are arbitraging open weight models, developing their own options and using the frontier AI labs when needed. Take notes since you'll likely be doing the same LLM portfolio management as you wrangle your AI budget.

The latest round of enterprise software earnings featured a bevy of nuggets on mixing and matching AI models. Since most companies are consuming their AI through their applications it's worth watching the vendors who are taking on the token cost risks and managing margins with a portfolio approach to models.

In the US, the storyline for AI is proprietary vs open models (via China), but there's a lot of nuances. Nvidia is championing both open and closed models. After all, Nvidia is selling you the AI infrastructure. CEO Jensen Huang recently set the scene on the company's earnings call:

"The world will need both closed models and open models. And both closed models and open models are skyrocketing in use. Open models are doing incredibly well. Closed models, we know are doing incredibly well," said Huang. "Every major company, every country and every startup needs to build their domain specific, their proprietary AI, their proprietary alpha. And the open models reaching frontier levels has made it possible, has enabled them to all do that."

Nebius' Roman Chernin, Chief Business Officer, added that proprietary data is the only moat and open models that are valuable because they can be tuned.

• Nvidia’s $12.93 billion Hugging Face acquisition validates open weight models
• Will enterprises learn to love Meta AI models again?

Ultimately, you'll be leveraging open models to offset frontier AI costs. As previously noted, the only things that'll matter in AI are price, performance and outcomes. Z.ai, the company behind one of the lead open models GLM-5.3 said as much in its earnings missive.

Z.ai said: "Industry evaluation criteria are also shifting from single-turn response quality and token consumption volume to long-horizon task success rate, unit intelligence cost, result verifiability, and the actual business outcomes delivered to customers."

The company added that it is focused on making better models and not chasing ancillary business models as a diversion.

With that backdrop in mind, here's a look at what software vendors are saying about managing their own models.

Oracle CEO Mike Sicilia was most explicit about open models and their role in controlling costs. Speaking at a recent investment conference, Sicilia said: "We arbitrage open source models. We oversource OpenAI inside of our own applications. I think there's a place for open models. In fact, some of those Fusion examples that I gave earlier with HCM, we're arbitraging both, in this case, OpenAI and open models."

If the results are good enough with an open model, follow the value. "If there's a good enough result from the model that results in a high-quality answer for customers, we're by no means opposed to leveraging that open model and keeping our costs low as a result," said Sicilia, who said their will be multiple model winners and Oracle won't be locked into any one LLM.

• Open-Weight Models Are Gaining Ground in Enterprise AI
• Get ready for US open LLMs and just in time
• Rightsizing open models may cut your AI inference spend
• The most important LLM benchmark today: Price

Salesforce CEO Marc Benioff hit a similar theme. He said customers don't care about the model underneath the application. "There will be frontier models. There are going to be open models and there's going to be room for all of the above," said Benioff, who talked up Nvidia's Nemotron models.

Benioff added Salesforce has its own custom models too. The Benioff riff was a touch awkward given the earnings call started with a Claudeforce announcement with Anthropic.

Workday's Gerrit Kazmaier, President of Product & Technology, also said the company is excited about the open-weight model developments. "They open up new opportunities for us for having our own reinforcement learning, building our own adapters over them. They give us a much stronger optionality when we think about international and sovereignty," said Kazmaier.

"We are focused on driving the right ROI and economics for our customers," said Kazmaier. "We deploy a large set of models from multiple vendors. We have small models, open-weight models and large frontier models we use to build our AI systems and agents with."

Workday has a lab that is "actively exploring opportunities for us to not only use open-weight models, but truly specialize them to our purposes and see what lift we get," said Kazmaier.

This mixing and matching of models is going to raise the profile of harnesses for AI agents. Box CEO Aaron Levie said the company is using its model neutral harness to "direct the workload to whatever is effectively the cheapest model at the accuracy level that the customer is looking for."

"In some cases, it'll be an open weight model and there are models on the horizon we're quite excited about we assume are coming in the second half," said Levie.

HPE CFO Marie Myers said the company can continue to improve margins and is expanding its AI-enabled process optimization effort called Catalyst. She said: "We have expanded both our AI and operational simplification efforts across the enterprise. HPE is now deploying an internal agent AI platform built on our own private cloud AI, open source, and open weight models, leveraging intelligent routing that sends each workload request to the most cost-effective AI model."

HPE's internal projections indicate the company can reduce its token costs on its private cloud relative to public cloud by up to 60%. "Routine tasks stay on-premise while frontier models are reserved for the most complex work," said Myers.

Add it up and you'll need to act a bit more like your enterprise vendors when it comes to LLMs. These enterprise software players are ahead of the LLM arbitrage game since it's part of their business. Here are some things to think about:

• The harness you use has to be model agnostic and preferably vendor agnostic.
• Any use case that doesn't require a frontier model should go with the cheapest option.
• Grow expertise in tuning open weight models with your own data preferably running on your own infrastructure.
• Choose what functions you want to outsource model selection to your vendor. The reality is most companies will consume AI through their software vendors.
• Focus on outcomes where proprietary models can drive competitive edge.
• It's likely that an open model strategy will include more Nvidia, which delivers its Nemotron models that are used by software vendors as well as consulting firms. For instance, Deloitte just started an open model practice based on Nemotron.

Snowflake CEO Sridhar Ramaswamy indicated customers are moving more toward arbitrage and optimization with open weight models. He said:

“We are absolutely seeing a lot of interest in being able to switch between different models and also to optimize cost. And this is also where open source models come in. There's obviously been several generations of these open source models, and we support many of them within Snowflake. And yes, we have pretty different economics when it comes to open source models since we run the inference ourselves. So that offers a lot of potential for future optimization.”