Rightsizing open models may cut your AI inference spend

Salesforce said it has cut its AI inference bills by right sizing models. The general idea: Tune specific open source models to complete tasks instead of relying on pricey models.

In a blog post that was a bit lost in the usual barrage of Salesforce articles about Slack, Agentforce and products, the company slipped by a bit of technical knowhow that'll matter to enterprises.

Salesforce said 18 months ago Agentforce ran on one rented model and its token bill rose with traffic. The choice for the software vendor was to pass on the cots to customers or rebuild its approach. The effort proved to be prescient given the blowback from rising AI costs.

According to Salesforce, precision models are growing share in its stack. A frontier model handles the multi-step processing, but the reasoning is governed by a harness where models can be swapped freely. As open source models improve and deliver better price performance, they'll only gain more share.

• SpaceXAI, Meta puts pricing squeeze on Anthropic, OpenAI
• DeepSeek's real impact happening now
• Here’s what we learned about AI projects from enterprise buyers so far
• Where’s tokenomics for the rest of us?
• AI inference costs are going to be a big concern: What's the fix?
• Why enterprise AI leaders need to bank on open-source LLMs

Salesforce knew what jobs it wanted to optimize and knew the steps and processes behind them. As a result, Salesforce didn't need to rely on a frontier model to reason over every step repeatedly. Jayesh Govindarajan, EVP, Software Engineering, Salesforce AI, said:

"We pulled each job out and tuned an open-source model to do it — targeted models, each handling the single job it was built for. We weren’t training from scratch. We were tuning on the patterns of millions of jobs the harness had already processed."

There are numerous examples cited by Salesforce. For instance, HyperClassifier is a model fine-tuned on GPT-OSS-20B, OpenAI's open-source architecture model. HyperClassifier reads requests, detects intent and routes to the tool best for the job.

TextEval is another model tuned from GPT-OSS-20B to evaluate answers and judge them. Here's a full look at the models outlined by Salesforce.

The takeaways here highlight how enterprises can leverage models designed for a task. You don't need to reason over everything and there are plenty of good enough models.

What this means for the pricing power of Anthropic and OpenAI remain to be seen, but simply put it's not your problem. Your problem is delivering returns.