Imagine you're a car manufacturer. You build the fastest car in the world — and then publicly call for speed limits. Sounds absurd? Welcome to Anthropic.
In early June 2026, co-founder Jack Clark and Marina Favaro, head of Anthropic's research institute, published a blog post titled "When AI builds itself." The message: the world needs the option to slow or temporarily pause AI development.
What's worrying Anthropic
The core concern is called "recursive self-improvement" — a state in which an AI designs and develops its own successor. This hasn't happened yet. Anthropic also says it's "not inevitable." But it could arrive "sooner than most institutions are prepared for."
And they have data. Uncomfortable data.
According to Anthropic's own blog post, more than 80% of the code merged into their codebase is written by Claude — as of May 2026. Before Claude Code launched in early 2025, that share was barely measurable. And in April 2026, Claude agents completed an open-ended AI safety research project to 97% in one week. Human researchers managed 23% in the same time.
In short: Claude is already contributing to its own development. Not fully autonomously — but the direction is clear.
The model: nuclear arms control for AI
Anthropic isn't calling for an immediate halt. The proposal is more nuanced: a multilateral, verifiable framework. When certain thresholds are reached, major AI labs should be able to coordinate a temporary stop — similar to nuclear arms control treaties.
The catch: nuclear facilities can be spotted from satellites. AI training in a data centre? Not so easily.
Why it won't happen anyway
The China argument. Dario Amodei — Anthropic's own CEO — said himself at the World Economic Forum in Davos in January that slowing down is barely possible. Enforceable agreements with China? Practically non-existent. The AI race is on. And whoever brakes, loses.
Washington isn't impressed. Officials at the White House have already described the proposal as a strategy for slowing rivals under the cover of safety concerns. The prevailing mood: America must lead, not hesitate.
OpenAI responds coolly. Democratic governments, not private companies, should decide on the pace of AI development — that's the official position from Anthropic's direct competitor. Sounds reasonable. And it neatly takes the wind out of the proposal's sails.
And then there's the irony. Anthropic keeps building. Claude writes its own code. Agents solve research projects. The company isn't braking — it's just warning.
What remains
The proposal isn't worthless. For the first time, a leading AI lab is publicly saying, backed by its own data, that development is heading in a direction that may be difficult to control. That's more than the usual press release.
Will a global agreement ever happen? Unlikely. Is the debate important? Absolutely.
And if you're wondering who wrote this article — in no small part: Claude. The same model we're writing about.