OpenAI, Broadcom unveil first AI inference chip

OpenAI and Broadcom launched their first AI accelerator called Jalapeno with plans to deploy by the end of the year. OpenAI's custom silicon effort is part of a plan to deliver a vertically integrated stack for LLMs.

In a statement, OpenAI said early testing indicates Jalapeno will deliver better performance per watt relative to the current state-of-the-art chips.

Jalapeno kicks off what's a multi-generation custom silicon plan for AI. According to OpenAI, the chip was built from a blank slate and "informed by its roadmap of models, kernels, serving systems, and product needs." Broadcom and Celestica will integrate the Jalapeno with board, rack systems, networking and production systems.

• AI inference costs are going to be a big concern: What's the fix?
• Google Cloud, AWS, Microsoft Azure: The AI vertical integration race

OpenAI said its chip will work with all LLMs. The company said:

"Jalapeño is designed with flexibility to work with all LLMs guided by OpenAI’s insights into the inference needs of current and future AI models across the industry. Engineering samples of the Jalapeño chip are running ML workloads in the lab at production target frequency and power, including GPT‑5.3‑Codex‑Spark."

OpenAI didn't provide performance data, but promised a detailed technical report in the months ahead.

Key points:

• The architecture of the chip reduces data movement and balances compute, memory and networking.
• Broadcom's silicon includes the Tomahawk networking chips.
• OpenAI's bet is that it can be more efficient if it owns the stack top to bottom.
• "The goal is to combine the power and throughput of today’s leading AI accelerators with latency closer to the fastest specialized inference systems, making Jalapeño well suited for interactive LLM products at scale," according to OpenAI.
• Jalapeno was co-developed in about nine months. OpenAI noted that it used its own models to accelerate the design and optimization process.

A few observations:

• OpenAI is following a well-worn playbook of building infrastructure to support its business. The effort rhymes with what Google, Meta and Amazon have done. Should OpenAI compete with Apple in the future (the company has a heavy dose of Google and Apple envy) custom silicon could be a differentiator.
• The big question is whether custom silicon can pay off fast enough for OpenAI’s business model.
• It’s notable that OpenAI noted that its custom silicon would work with any LLM. Could OpenAI become an infrastructure provider?
• The diversification away from Nvidia continues.

Constellation Research analyst Holger Mueller said:

"LLM vendors have to bring down costs for AI and the place to partner is with the vendor (Broadcom) who has helped the leader in AI cost - Google. The pedigree is there and now OpenAI has to get the chip built. Installing Jalapeno in data centers will not be a challenge given OpenAI's market and purchasing power. This is a shot in front of the bow for Nvidia."

More on OpenAI:

• OpenAI expands Daybreak program, updates GPT-5.5-Cyber, lands partners
• OpenAI joins IPO parade
• OpenAI raises $122 billion, touts $2 billion in revenue a month