OpenAI announced that its in-house Jalapeño chip outperformed Nvidia processors in tests
OpenAI plans to begin using its Jalapeño chips this year / Photo: OpenAI
AI giant OpenAI announced on August 25 that its new Jalapeño chips, unveiled in June, outperformed Nvidia’s current line of processors in testing. This underscores the company’s progress in developing its own AI processors, Bloomberg notes.
Jalapeño, developed by OpenAI in collaboration with Broadcom, led the tests in two metrics: AI compute per unit of power consumption and response speed. The chip was compared to the Nvidia GB300, which was the top performer in the public benchmarking system used for the test, explained Richard Ho, head of the company’s AI division, to Bloomberg.
Jalapeño was not compared in tests with the new generation of Nvidia Vera Rubin processors, which have only just begun shipping. Furthermore, it is not designed for training AI models—an area where Nvidia’s technologies are particularly strong. The chip from OpenAI is designed for the inference stage—the process of generating responses to user queries using a pre-trained neural network.
Typically, some chips are better at handling large volumes of AI computations, while others are better at providing fast responses; however, Jalapeño combines both advantages, Ho said. OpenAI has yet to determine which of its AI models will run on it, which will allow customers to choose between options that offer either lower costs or higher performance, the executive explained.
“Under laboratory conditions, Jalapeño demonstrates high performance both in high-throughput mode—meaning it can serve a large number of clients more cost-effectively—and in low-latency mode, which means that for clients for whom this is important, response times will be truly very short,” — said Ho to Bloomberg.
According to him, since the chip delivers strong performance with relatively low power consumption, it will allow OpenAI to save on data center operating costs, where electricity is one of the key expense items. As Broadcom CEO Hock Tan noted, at this stage, the chip offers cost savings of approximately 50% compared to traditional AI graphics processors.
Full-featured production versions of Jalapeño will be rolled out to major data centers operated by Microsoft and other OpenAI partners as early as the second half of the year, the company announced.
This article was AI-translated and verified by a human editor
