OpenAI and Broadcom have unveiled a new chip that will cut the cost of running AI in half

On Wednesday, OpenAI unveiled its first specialized AI chip, developed in collaboration with Broadcom. It was designed specifically to handle modern large language models, rather than as a general-purpose processor. The custom processor should give the creator of ChatGPT a competitive edge by tailoring the hardware to its products, Bloomberg notes.

The chip and its platform were named Jalapeño—after the spicy Mexican pepper—though OpenAI did not explain this choice.

The company has already begun testing prototypes. According to Broadcom CEO Hock Tan, at this stage, the accelerator is demonstrating cost savings of approximately 50% compared to traditional AI graphics processors.

Full-scale production versions of Jalapeño will be deployed in major data centers operated by Microsoft and other OpenAI partners as early as the second half of the year, the company announced. Partners had previously expected to deploy chips with a total computing capacity of 1.3 gigawatts next year, but Broadcom has now stated that this forecast may be exceeded. “We expect to deliver even better results, as demand is very high,” Tan said in a press release.

Although OpenAI remains heavily reliant on Nvidia’s semiconductors, the startup is seeking to diversify its supplier base amid rapidly growing demand for AI services, according to Bloomberg. In particular, the company has signed multibillion-dollar agreements with chipmakers such as AMD and Cerebras Systems.

In October, OpenAI and Broadcom first announced that they were jointly developing graphics accelerators optimized for use with AI models. OpenAI plans to spend tens of billions of dollars on Broadcom chips, further increasing its already massive infrastructure costs, the agency notes.

In March, OpenAI raised $122 billion in funding to support large-scale investments in chips, data centers, and talent.

In the industry, some AI chips are designed to perform a large number of tasks simultaneously and are used for training models, while others are optimized to respond as quickly as possible to a single user query, a process known as inference. OpenAI states that its goal is to combine both approaches: the power of today’s best AI processors and the speed of specialized inference solutions.

The new chip is expected to improve efficiency by reducing the amount of data that needs to be transferred within the system. The architecture is also optimized for the most critical use cases involving computing resources, memory, and networking equipment for cutting-edge AI models, according to a Broadcom press release.

According to Tan, the companies have already developed a roadmap for the next generations of chips. The next version is scheduled for 2028, and new generations will be released annually thereafter. While Jalapeño is primarily focused on inference, future processors may be adapted for other types of computational workloads as well.

Tan believes that other developers of cutting-edge AI models will follow OpenAI's example by creating their own optimized, custom AI accelerators and the network infrastructure to support them.

This article was AI-translated and verified by a human editor