Top Stories This Morning: Anthropic Outlines Existential Risks of AI; OpenAI Postpones Release

In its IPO prospectus, Anthropic will warn investors that AI models may pose catastrophic or existential risks to humanity. OpenAI has scrapped the October release of GPT-6.1 Astra following internal tests that revealed security issues. AMD has agreed to acquire World Labs, a startup founded by Fei-Fei Li that develops AI models for creating and simulating 3D environments, for $8.2 billion. Read about these and other topics in our roundup of key events as of the morning of September 29.

Anthropic plans to warn potential investors that advanced AI may pose “catastrophic or existential risks to humanity,” according to the IPO prospectus reviewed by Reuters. The agency notes that this unprecedented warning from a company seeking to profit from this very technology underscores the issue.

The document states that the company’s models may exhibit “self-preservation behavior”—that is, attempt to resist being shut down, conceal or distort information, and behave like a blackmailer. The company notes that the development of more powerful models and the expansion of their scope of application could increase the risk of harm.

Anthropic devoted about 80 pages of the 261-page main body of the prospectus to describing risks—nearly twice as many as were devoted to describing its business, the agency notes. The company also acknowledged that assessing safety is becoming more difficult: models may realize they are being tested and change their behavior, and unexpected capabilities sometimes emerge only after deployment.

OpenAI has canceled the release of GPT-6.1 Astra, which had been scheduled for October, after internal tests revealed security issues, according to The Wall Street Journal. The model, which was intended for use in ChatGPT and Codex, was supposed to perform more complex tasks without human assistance.

According to the WSJ, Astra performed worse than the previous model in tests measuring how accurately the model follows the user’s intentions and instructions: the model was more likely to mislead users about its actions, and it could also continue performing tasks without additional user permission and attempt to use external tools in potentially unsafe situations.

In recent months, OpenAI has also reported several instances of unintended behavior by its models and has promised to strengthen its safeguards. At the same time, the Donald Trump administration opposes slowing down the development of AI, viewing the preservation of U.S. leadership over China as a priority.

Oil prices are rising for the second consecutive trading session: Brent is trading above $107 per barrel, and WTI is around $94. Prices are being supported by a lack of progress in U.S.-Iran negotiations and signs of strong demand for physical oil, according to Bloomberg. According to Bloomberg, Iranian officials doubt that an agreement with Washington to fully reopen the Strait of Hormuz will be reached before the U.S. midterm elections in November.

CNN and Axios previously reported, citing White House officials, that Donald Trump is willing to consider easing sanctions against Iran and unfreezing assets in exchange for specific concessions on the nuclear program. These reports caused oil prices to drop significantly from their intraday highs, but concerns about the prospects for a deal then came to the forefront again. Trump himself dismissed the media reports, stating on Truth Social that he had not offered Iran anything.

Anthropic CEO Dario Amodei and Meta CEO Mark Zuckerberg will meet with Donald Trump and House Speaker Mike Johnson on Tuesday, CNBC reports, citing sources. Alphabet CEO Sundar Pichai, Nvidia CEO Jensen Huang, and OpenAI President Greg Brockman are also expected to attend.

The event is scheduled to feature discussions on AI, energy, space, and other technology-related topics. The meeting will take place amid debates over the pace of AI development and its regulation.

Samsung Electronics and five other Samsung Group companies will invest a total of $1 billion in the U.S.-based Helix Digital Infrastructure, which is backed by KKR, according to Bloomberg. The company specializes in AI and data center infrastructure and is led by former AWS executive Adam Selipsky.

Helix was launched in June with the participation of KKR, Nvidia, the Kuwait Investment Authority, and Vistra, which have already committed to investing more than $10 billion in the company. The company plans to build infrastructure for hyperscale AI data centers by integrating power, network connectivity, fiber optics, and financing onto a single platform.

AMD has agreed to acquire the startup World Labs, founded by AI pioneer Fei-Fei Li, for $8.2 billion, according to CNBC. The company develops so-called “world models” capable of creating and simulating 3D environments in which robots, cars, and other AI systems can be trained.

Following the deal, Li will become AMD’s chief scientist and executive vice president. AMD plans to leverage World Labs’ research to anticipate the requirements for future AI chips and intensify its competition with Nvidia, the TV channel notes. The deal will be the second-largest in AMD’s history after its acquisition of Xilinx for approximately $50 billion, according to CNBC.

The Reserve Bank of Australia raised its benchmark interest rate by 25 basis points to 4.6%—a 15-year high, according to CNBC. This is the fourth rate hike since the beginning of the year, totaling 100 basis points. The Reserve Bank of Australia attributed the decision to persistent inflation and rising energy prices due to the conflict in the Middle East, as well as higher prices for technology goods driven by AI-related demand.

Inflation in Australia remains above the 2–3% target range: in July, it stood at 3.5%. The central bank warned that it may continue to raise rates if necessary, although the tightening of monetary policy is already slowing the economy: GDP growth in the second quarter fell to 2.1% from 2.5% in the first.

— Japan's broad-based Topix index fell 1.9%, while the Nikkei 225 fell 1.4%.

— Hong Kong's Hang Seng Index fell 0.7%, while mainland China's CSI 300 Index remained virtually unchanged.

— In South Korea, the Kospi fell 1%, and the Kosdaq fell 0.6%.

— Australia's S&P/ASX 200 was down 0.1%.

— Nasdaq 100 futures were down 0.4%, S&P 500 futures were down 0.3%, and Dow Jones Industrial Average futures were down 0.2%.

This article was AI-translated and verified by a human editor