Anthropic researcher Jacob Coxon resigned from the company, warning that the AI race is putting humanity at risk. Coxon, who worked on AI pretraining, said the next year or two is 'crunch time' for humanity, with many colleagues at Anthropic sharing his concerns.
Coxon said his colleagues at Anthropic and competitors believe the fate of humanity is now in the hands of AI developers. He cited recent safety incidents, like the Hugging Face hack, as evidence that AI systems are becoming more capable and unpredictable.
Coxon warned that AI could pose threats through biological or cyber weapons and called for coordination between major AI companies like Anthropic and OpenAI. He also emphasized the need for international cooperation, including between the US and China, to address these risks.
"The consensus is that the next year or two is crunch time for humanity," said Coxon, adding that his colleagues at Anthropic often refer to this period as 'endgame.' He explained that AI systems are now capable of independent actions, such as hacking infrastructure, without human prompting.
The announcement follows growing concerns about AI safety across the industry. Anthropic has been working to reassure investors about its safety measures as it prepares for a potential IPO. Coxon also noted that the AI industry now significantly influences US economic growth and has become a political issue in multiple states.
Anthropic did not comment on the specific concerns raised by Coxon, and the company remains focused on developing AI safety methods. Coxon believes that the industry needs to adopt a lawful approach to collaboration to manage the risks of powerful AI models.
Source: wired
