"We Could Lose Control of the Future": Altman Outlines Two Dangerous Scenarios for AI

OpenAI CEO Sam Altman outlined “two scenarios in which AI development could go very wrong and that must be avoided.” In posts on social media platform X, he warned of the need to slow the development of AI capabilities and ensure the safety of this technology—a call that Anthropic co-founder Dario Amodei made last week and that other industry leaders have supported.

First, Altman writes, “we could lose control of the future by handing it over to AI.” We need ways to ensure that methods for ensuring AI aligns with humanity’s goals and safety will evolve faster than the capabilities of the models themselves, explains the head of OpenAI.

Second, he continues, “we may find ourselves in a world where too much power is concentrated in the hands of a few.” If an extremely powerful AI were to fall into the hands of a single person, company, or country and were used to impose their worldview on everyone else, the consequences could be extremely dystopian, the post states.

Altman also noted that “no competitive pressure on the U.S. should justify recklessness.” Last night, President Donald Trump called for not holding back the development of AI and for maintaining the country’s lead over China in the AI race.

To avoid these two threats, it is necessary to strike a delicate balance, Altman wrote. He advocated for the introduction of uniform federal safety requirements for advanced AI systems, but emphasized that companies can begin taking action on their own without waiting for new legislation. In particular, the head of OpenAI supported the development of common risk management guidelines and independent audits—Amodei had previously voiced the idea of industry standards and access to external experts.

According to Altman, previous safety mechanisms focused primarily on models that were already complete, whereas now greater oversight is needed during the development process. At OpenAI, for example, safety justifications are now prepared in advance of training phases if the capabilities of state-of-the-art models are expected to be significantly expanded. Altman called on other companies to share their approaches and hopes that collaboration among AI labs will lead to the development of unified standards for monitoring and preventing AI from diverging from human goals.

However, when the OpenAI CEO refers to “slowing the pace,” he does not mean “stopping”: progress will continue, but it must proceed more slowly than it would have without these additional measures. Altman acknowledged that these measures require significant costs, but in his view, the expenses are justified.

Bloomberg believes that AI giants will have to balance their calls for a slowdown against opposing forces in the tech industry, financial markets, and the Trump administration.

The market reaction was already evident on Monday: shares of Samsung Electronics and SK Hynix—key players in the AI infrastructure sector—fell by more than 4% in Seoul, while shares of OpenAI investor SoftBank Group dropped by 13%.

In Silicon Valley, talk of a slowdown could hit chipmakers, which are benefiting from rising spending on processors for training and running new models, according to Bloomberg. Nvidia’s pre-market shares fell 2.9%, Intel’s stock plummeted 5.8%, and Marvell Technology’s shares dropped 7.2% right off the bat. Hyperscalers such as Amazon and Microsoft, which provide cloud computing power to AI developers, may also feel the impact.

The plans of OpenAI and Anthropic “are having a fairly significant impact on the entire technology sector,” explains Evercore ISI analyst Mark Mahaney. “If these two companies sharply cut back on research and development expenses, hiring, and capital expenditures, it could have a significant impact on financial markets,” he said.

U.S. index futures were trading lower two hours before the start of the main trading session. In addition to concerns about a slowdown in AI development, they are under pressure from a sharp rise in oil prices. Contracts on the tech-heavy Nasdaq 100 fell more sharply than others, plummeting 1.5% right off the bat.

Shares of European companies involved in AI also fell sharply. Shares of chip-making equipment manufacturer ASML dropped 5%, Nokia’s fell nearly 10%, and Infineon’s fell nearly 8.5%.

This article was AI-translated and verified by a human editor