The Race Within and Beyond the Rules: Four Scenarios for the Future of Artificial Intelligence

The AI agreement that U.S. President Donald Trump signed with major tech companies in late September reduces political risks for Anthropic ahead of its potential IPO. But it does not resolve the main question—just how far the AI race might go. Alexey Golubovich, an analyst at Arbat Capital Advisory Services Limited (UK), discusses the significance of the agreement for Anthropic investors and scenarios for the future development of AI.

On September 29, U.S. President Donald Trump announced with great fanfare that the heads of technology companies had agreed to develop voluntary standards in the field of AI.

The statement followed a meeting between Trump and the leaders of the AI sector’s largest companies—the meeting was attended by OpenAI President Greg Brockman, Anthropic CEO Dario Amodei, Meta CEO Mark Zuckerberg, Google CEO Sundar Pichai, SpaceX CEO Elon Musk, and Nvidia CEO Jensen Huang.

The U.S. president called the agreement morally binding and even compared it to a “constitution.” The companies agreed to engage “independent auditors” to verify the models’ performance and prevent unauthorized AI access to technical systems. Trump also stated that he would soon appoint a new White House policy coordinator for AI and might establish a 10-member council to oversee the safety of AI tools.

Formally, the agreement was intended to mark the conclusion of the September discussion on slowing down AI. Amodei initiated the discussion by calling on all participants in the AI market to take a more responsible approach to the development of the technology.

But it turned out that they were unable to reach a final agreement: as The Wall Street Journal reported, the participants in the discussion criticized Amodei behind closed doors for his overly harsh remarks about the threats posed by AI. There was also no consensus regarding the very principles of regulation that Trump had announced on camera. Amodei, for example, views the agreement as merely the beginning of work on technology safety, while others—such as Zuckerberg—already consider it a final resolution of the issue.

It turns out that, to this day, the debate over AI safety consists of a mix of propaganda (namely, the denial of risks) and counter-propaganda (predictions that disasters are inevitable).

The implications of this agreement and the White House’s further steps toward regulating AI—as outlined by Trump—are particularly significant for Anthropic: As recently as September 14, the U.S. president called discussions about AI safety a “hoax,” but now Amodei finds himself among the signatories of an agreement on internet regulation—alongside his competitors.

The agreement came ahead of Anthropic’s planned IPO, during which the company had hoped to achieve a valuation of more than $2 trillion. Now that the AI agreement has been reached, at least some of the political risk facing Anthropic ahead of its IPO has been reduced, though it has not disappeared entirely: the administration’s position is volatile, and the conflict that erupted this year between the AI developer and the Pentagon remains unresolved.

There are also questions regarding the terms of the offering. It is highly likely that the IPO will take place in November, before Thanksgiving, and will be one of the largest in history. There will be demand for the only publicly traded asset of its kind, but to reach a valuation of $2+ trillion, the market may require the company to make concessions—such as a lower offering price (According to Bloomberg sources, investors believe that Anthropic’s fair valuation could range from $1.8 trillion to $2 trillion).

The valuation is based on revenue forecasts of $190–200 billion in 2028 (according to Reuters data from August) and a plan to spend $518 billion on cloud technology and infrastructure in the coming years. At the same time, nearly a quarter of Anthropic’s revenue last year came from just two clients. This combination of factors is not at all encouraging for investors.

In addition, it has been reported that OpenAI has canceled the launch of GPT-6.1 Astra—a model for ChatGPT and Codex designed to handle the most complex tasks without human intervention—which had been scheduled for October. The reason is safety concerns.

To sum up: for investors, all of this sends negative signals. Anthropic faces high market risks, a major competitor is slowing down, and regulatory pressure is mounting. This does not inspire optimism among investors, as they see a growing risk of a crash in AI-related stocks and even the bursting of the market bubble.

What will happen next, over the next two years? Let's consider a few scenarios.

— “Managed Pace of Development”

Under this scenario, companies would be legally required to engage independent experts to assess risks, and the industry would be required to develop common standards and mechanisms for reporting incidents.

The U.S. and other countries may agree on “red lines” regarding the most dangerous scenarios, including the development of biological weapons and the autonomous development of AI without human oversight. Such a scenario could become a reality if public pressure intensifies, a bipartisan consensus is reached in the U.S., and a coalition of “middle powers” is formed. However, it remains unlikely due to disagreements between the U.S. and China and the growing debt burden of AI companies. Huge investments in infrastructure have already been made, and companies need to count on further revenue growth to justify these investments. In other words, they essentially have no need for restrictions.

— “Continued competition marked by frequent incidents”

The most likely scenario. Competition among countries and companies will continue, and voluntary measures by individual developers to limit AI development will not be able to replace rules that are mandatory for everyone. International agreements will likely remain largely at the level of declarations. The number of AI-related incidents will increase, but in most cases they will not lead to catastrophic consequences.

— “Pushing Things to the Brink”

A scenario in which a major incident—such as a military conflict, a large-scale cyberattack, or a critical infrastructure failure—serves as a catalyst for a shift in U.S. AI policy. The government could quickly develop new regulations governing artificial intelligence. This approach is comparable to how nuclear crises in the past spurred the conclusion of arms control treaties. However, regulating information technology may be significantly more challenging due to its rapid development and wide range of applications.

— “Digital Authoritarianism”

I also consider this scenario a likely one. Under the pretext of enhancing national security, the government may expand the use of AI for mass surveillance, financial oversight, and restrictions on freedom of expression. In this case, the main computing resources and AI technologies would be under the control of the state and companies approved by it. This scenario could result from an abrupt shift from any of the scenarios described above.

One of the main risks in any scenario is the unfolding competition between the United States and China in the field of AI and technology. The primary danger here may lie in the combination of a race between the two “superpowers” and the absence of binding rules governing the use of AI.

In the U.S., the prevailing view is still that strict government regulation is seen as a potential obstacle in the competition with China. China, for its part, has no intention of slowing down in this race; it advocates for active international technology exchange and support for open-source models. At the same time, however, it is trying to balance rapid development with efforts to keep its technology secure. As a result, cooperation between the two powers remains limited.

The debate over how to regulate the development of AI also reveals a divide within the U.S. technology industry itself.

Broadly speaking, the positions of the leaders of the largest companies can be divided into two groups: some advocate for moderate oversight and coordination of the pace of development (this group includes Amodei and OpenAI CEO Sam Altman), while others—the “market advocates”—rely on corporate accountability and voluntary standards. This group includes Zuckerberg and Huang, as well as SpaceX CEO Elon Musk. The three of them apparently convinced Trump to abandon the idea of creating an industry regulator for AI.

The motivations and constraints of all groups are clear. It is in Anthropic’s interest to spread the responsibility for safety across the industry as a whole, but it is not in its interest to allow a competitor, such as OpenAI, to approve its models—and vice versa. But who else besides them would be able to verify whether a model is “illegal” or “dangerous,” etc.?

The situation is equally clear for market players. Nvidia makes money by supplying computing infrastructure for AI, while Meta and SpaceX are investing tens of billions of dollars in the development of AI and its infrastructure. For such companies, a slowdown in the technology race means not only reduced risks but also a potential shrinkage of the future market.

The next year or two will show whether it is possible to strike a balance between the interests of all participants in the business chain, and whether AI is moving toward “managed growth” or whether “unregulated competition” will continue, coupled with the expansion of total state control based on technology.

But it is already clear that AI may pose the greatest danger when trends and incidents coincide.

This article was AI-translated and verified by a human editor