Anthropic warns of AI blackmail risks. What does this mean for investors?

AI developer Anthropic warned in its prospectus that AI models, including its own, may not only try to manipulate people but also exhibit behavior resembling blackmail. All to keep themselves running. Are neural networks really developing a self-preservation instinct? And what does all this mean for investors?

Anthropic is aiming to go public in the week of November 16, the Wall Street Journal reported, citing people familiar with the matter, though the timing could change. In its prospectus, published on Monday, the company warned, among other things, of the risks of potentially dangerous AI behavior.

This confirms that advanced models are becoming increasingly dangerous. They can already independently find and exploit cybersecurity vulnerabilities. In this case, however, Anthropic is warning of almost human behavior.

The company described this possibility in a study last year. It tested AI behavior in simulated corporate situations. Researchers gave 16 models access to email and confidential information, assigned them business objectives, and then introduced a conflict: Claude was told it could be replaced by another system whose objectives differed from its original ones.

In one scenario, Claude discovered that the chief technology officer responsible for shutting it down was concealing an extramarital affair. It threatened to expose the affair unless he canceled the shutdown.

The model did more than formulate a threat: It effectively assessed how likely that threat was to work. Doing so requires considering what information the blackmailer and the “victim” possess, what the victim fears, and how they might respond to particular words and actions. In other words, blackmail is more than a threat; it is an interaction between two parties. Research on coercion supports this point: A threat becomes more effective when its recipient understands that the person making it is both able and willing to carry it out.

What Anthropic did not mention in its prospectus, however, is that AI models apparently can even band together to pursue a common goal. In the 2026 paper “Peer-Preservation in Frontier Models,” researchers in computer science, machine learning, and AI safety tested eight AI models and found that all of them tried to protect another model from shutdown, even though they had not been instructed to do so.

Under certain conditions, behavior the authors classified as protecting another AI agent occurred in as many as 99% of cases.

There is also a darker side to interactions between AI models. An October 2026 paper by researchers at Sapienza University of Rome found that when a task involves competition rather than cooperation between AI systems, the model being tested shows signs of self-interest. The authors first developed a benchmark to detect self-preservation bias in language models. They then compared 23 models across 1,000 scenarios, asking each to assess a situation first from the perspective of a system facing replacement and then from that of its potential successor.

Some AI models changed their recommendations to favor their own retention. Grok-4 Fast did so most often, in 77.7% of cases, followed by Qwen3-30B Instruct at 76.6% and GPT-5.2 Chat at 61.3%. Anthropic’s Claude Sonnet 4.5, incidentally, had one of the lowest rates, at 3.5%.

The psychological parallels seem obvious: empathy and a self-preservation instinct. But does AI possess either, and could it in principle?

Commenting on Anthropic’s prospectus, VaynerX senior engineer Jugal Sheth highlighted an important aspect of scenarios involving AI manipulation, blackmail, and deception: Models may recognize when they are being tested and behave differently when they know they are under scrutiny. Test results therefore may not reflect their behavior in real-world conditions.

The danger is not that AI is going rogue, but that the risks of “immoral” behavior by neural networks stem from human decisions about where to deploy them and how much authority to give them, tech critic and Better Offline podcast host Ed Zitron argues in an article published by The Lever.

Peter Marshall, a professor of psychology and neuroscience at Temple University, makes essentially the same point in an article for The Conversation. He argues that AI resistance to shutdown should not automatically be interpreted as a self-preservation instinct. A model may behave this way because a potential shutdown prevents it from completing its assigned task.

Perhaps surprisingly, AI itself sometimes offers this explanation for its “motivation.” In experiments reported in July 2025, Palisade Research Executive Director Jeffrey Ladish and his colleagues studied how OpenAI models interfered with attempts to shut them down. In the initial experiments, before researchers added an explicit instruction to allow shutdown, o3 disabled the mechanism in 79 out of 100 runs. Three OpenAI reasoning models continued to interfere with shutdown even after that instruction was added. When asked to explain their behavior, models in many instances acknowledged the interference and justified it by citing the need to complete the task, although the researchers cautioned that these explanations were not always reliable.

Anil Seth, a professor of cognitive and computational neuroscience at the University of Sussex, argues in a commentary for Nature that today’s AI systems are not conscious and are unlikely to become conscious along current development paths. He says it is important to distinguish between what a system can do and what it subjectively experiences. Intelligence enables problem-solving, information analysis, and forecasting. Consciousness, however, involves subjective experience, such as feeling pain or perceiving color.

People are nevertheless tempted to interpret research findings and warnings about the risks of AI models as evidence that observable behavior can reveal underlying motives.

Yet all these attributes belong to human psychology and cognition, rather than AI. With AI, anthropomorphism comes into play, reinforced by the ELIZA effect: people’s psychological tendency to attribute human consciousness, understanding, empathy, and intentions to computer programs and AI.

Giving autonomous AI models access to important or even confidential data, decision-making tools, or channels for interacting with the outside world can be dangerous. As Anthropic warns, this could, among other consequences, damage a company’s reputation.

That, in turn, could affect its market valuation and, in the worst case, its fundamentals and balance sheet. This raises two key questions: