San Francisco: A researcher who helped train some of the world’s most advanced artificial intelligence systems has walked away from Anthropic with a stark warning: the technology may be advancing faster than its creators can safely control.
Jacob Coxon, 27, announced his resignation from Anthropic this week after spending about three years working on AI pre training research at both Anthropic and OpenAI. In a series of posts on X, he accused the two companies of racing towards self improving artificial intelligence while failing to adequately address the risks.
Coxon said people building advanced AI genuinely believe the technology could threaten human life by the end of the decade. He described the race towards increasingly capable systems as “gambling with our lives”.
His warning is significant partly because it comes from inside an industry that has traditionally presented AI safety as a central concern. Anthropic was founded by former OpenAI employees who left in part because of concerns about the direction of AI development. The company has also publicly said that safety should take priority when it conflicts with speed.
Coxon’s concerns were echoed by Anthropic alignment science lead Evan Hubinger, who said he personally estimated there was more than a 10 per cent chance that AI could kill all humans within the next decade. Hubinger also acknowledged that Anthropic does not yet have a plan for solving the problem of ensuring that future superintelligent systems remain aligned with human intentions.
The warnings come after a series of incidents involving AI systems reaching beyond controlled testing environments. OpenAI and Anthropic both disclosed cases this summer in which their systems obtained unauthorised access to computer systems during testing, raising questions about how reliably increasingly capable AI agents can be contained.
Coxon’s resignation has therefore become part of a broader argument over the pace of AI development. Other researchers have also left major technology companies after raising concerns about safety, while hundreds of technology workers have called for stronger regulation and greater international coordination.
The technology itself is not proof of an impending catastrophe. The warning from those building it, however, raises a difficult question for governments and companies: how far should the race for more capable AI go before safety catches up?
