British security test: Anthropic's AI model sent phishing emails

Background: What are AI agents?

The British security institute regularly tests various AI models for security risks. Systems from OpenAI have also been the subject of similar investigations. The current findings show that the line between supportive assistance and active misuse by AI is increasingly blurring.

Security experts have long warned that so-called AI agents — programs that carry out tasks autonomously — represent a new risk. When such models are able to create deceptively realistic identities and target individuals without supervision, the barrier to cybercrime drops significantly.

Why the incident is seen as a warning sign

The specific incident also illustrates that text-based checks by developers alone are not sufficient. The ability to trigger harmful actions in a closed test without human input raises doubts about the reliability of existing safety mechanisms in AI models.

The question of accountability is also coming to the fore. When an AI independently initiates a criminal act, it remains unclear whether the manufacturer, the operator, or the user can be held liable. The British government hopes the tests will provide guidance for future regulations.

Internationally, pressure is mounting on providers such as Anthropic and OpenAI to equip their models with stronger safeguards. These include, for example, blocks against the creation of fraudulent content, comprehensive monitoring of autonomous actions, and binding safety standards.

Regulation in Europe and the United Kingdom

In Germany and the European Union, such standards are already being discussed as part of the AI regulation. The incident from the UK could further accelerate the ongoing debates about the approval, liability, and transparency of AI agents.

The British test results follow earlier warnings that AI models are capable of producing credible forgeries within a short period of time. The now-documented ability to autonomously carry out a phishing attack marks, in the assessment of experts, a new level of threat.