Recursive self-improvement: Why Anthropic wants AI development slowed

As the race to build ever more powerful artificial intelligence systems accelerates, one of the industry’s leading players is urging the world to consider a possibility that until recently belonged largely to science fiction: machines improving themselves without human intervention.

Anthropic, the AI company behind Claude, said on Thursday that the ability to slow the pace of frontier AI development could prove valuable as the technology approaches capabilities that may fundamentally reshape society.

The warning came in a blog post authored by Marina Favaro, head of Anthropic’s internal research institute, and company co-founder Jack Clark.