Nvidia formed an AI safety alliance following an accidental hack carried out by OpenAI

Nvidia, Microsoft, SpaceX, Palantir, and dozens of other tech companies have announced the formation of an alliance dedicated to developing and distributing tools to ensure AI and cybersecurity. The initiative came about following an accidental cyberattack by an OpenAI model on Hugging Face, an open platform for building neural networks, during which the startup was forced to use a Chinese AI model for self-defense.

37 leading companies in the fields of AI, cloud computing, cybersecurity, enterprise software, and open-source software have announced the formation of an alliance to jointly develop technologies and tools for general use aimed at ensuring AI and cybersecurity. The alliance includes, among others, Nvidia, Microsoft, SpaceX, Palantir, Adobe, CrowdStrike, Dell—and dozens of other technology companies from the U.S. and Europe. The alliance has been named The Open Secure AI Alliance.

"The recent security incident involving [an open platform for building neural networks] Hugging Face served as a stark reminder: cybersecurity professionals need open, state-of-the-art AI agent systems for self-defense,” according to a joint statement by The Open Secure AI Alliance published on Nvidia’s blog.

Nvidia also stated that it is providing the alliance with open-source models, weights, data, and research in the field of AI agent infrastructure (agent harness), including the new open-source project Nvidia Labs Object-Oriented Agent, which is available on GitHub. This framework will help control systems more effectively manage the behavior of AI agents by simplifying the process of testing, tracking, verifying, and regulating their actions, the company added.

The alliance was formed following a breach at Hugging Face, which was inadvertently carried out by an OpenAI AI product. The startup was unable to use leading U.S. state-of-the-art AI models to protect its infrastructure because the built-in security mechanisms could not distinguish between the attacker and the defender. As a result, the company deployed the open-source Chinese model GLM 5.2, which was not subject to similar restrictions.

The initiative also came at a time when discussions in Washington about possible restrictions on Chinese AI models are intensifying. Last week, U.S. Treasury Secretary Scott Bessent threatened sanctions against Chinese companies that use distillation—a method of extracting knowledge from more advanced models developed by American companies.

There is a “real possibility” that the U.S. will impose restrictions on Chinese models: these restrictions could affect the purchase of tokens via APIs, as well as the deployment of such models by U.S. cloud providers, according to Chris McGuire, a senior fellow at the Council on Foreign Relations, as reported by CNBC. At the same time, the expert emphasized that the issue in Washington is not a clash between open and closed models, but rather a fight against the alleged theft of intellectual property by Chinese companies.

At the same time, representatives of the technology industry warn that excessive restrictions could harm the development of open AI. On July 24, Nvidia, Microsoft, Meta, Palantir, and more than 20 other companies sent a letter to U.S. authorities urging them to refrain from “premature restrictions” on models with open weights, as they believe this could weaken competition and lead to innovation shifting outside the U.S.

This article was AI-translated and verified by a human editor