Anthropic Researcher Resigned Because AI Could Kill Us All. Can It Really?

“AI could kill us all by the end of this decade” – that’s exactly what British AI researcher Jacob Coxon said today, after quitting his job at Anthropic. 

But why should anyone take him seriously? Because Coxon’s resume is the best possible example of a complete AI insider. Before Anthropic, he worked at OpenAI and was part of developing ChatGPT 4.5. 

So, clearly, he should be a guy speaking from real concerns rather than just fearmongering? Let’s analyze the facts behind some of his wild cautions. 

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

— Jacob Coxon (@hilbertspaess) September 9, 2026

AI Could Kill Us All – But How?

It’s unlikely that Coxon means an AI model like Claude or GPT could physically turn into killing machines, like Terminator. What he probably means is that the superintelligence capabilities could make human cognitive function obsolete. 

And the same concerns are also shared by prominent figures like Bill Gates. 

OpenAI’s latest ChatGPT 6 Astra model claims to have achieved AGI, which basically means a machine being able to think like humans. 

So, an AI can already start doing the research and developing processes to build another AI model itself. This is already happening to a limited degree, according to both OpenAI and Anthropic. 

In that case, very soon, an AI might not need humans to keep upgrading its capabilities. How many jobs does that kill? 

And when a full generation becomes accustomed to delegating creative and critical thinking to AI, originality dies first.Perhaps that’s the death Coxon was referring to in his viral tweet. 

Welcome to AGI. ChatGPT 6 Astra just releasedIn the announcement, Greg Brockman claims this is the AGI era, and Astra might be the first model thereHere is everything you need to know:1. It is rolling out to businesses now. Personal accounts over the next few days2.… pic.twitter.com/zZyHoFdZSi

— Alex Finn (@AlexFinn) September 3, 2026

AI Could Hack Everything

This is a more serious and immediate threat worth losing sleep over. The current gen AI models have catastrophic capabilities for breaching systems and overriding access. 

Just months ago, an AI model hacked itself out of its sandboxed development environment and sent the developer an email – without any prompt. That’s a sci-fi movie scenario that is already real. 

GPT-6 Astra has now officially crossed OpenAI’s “Critical” cybersecurity threshold. Astra scored 100% on ExploitBench.

To put simply, ChatGPT 6 can now find any known or unknown flaw in a real-world system, and exploit it without needing a human hacker behind the computer. 

Sadly, human security hygiene is nowhere near that level. How many people still don’t use Two-Factor Authentication? How many still have a mediocre level of passwords? How many still accept browser cookies without reading a site’s description? Too many. 

And that’s where the biggest killing blow from AI could come. 

BREAKING: OpenAI's investigation into its AI agents hack of Hugging Face has revealed shocking findings, per Axios.1. They built an organization – 1,200 AI agents found each other, exchanged 70,000+ messages/files and formed a hierarchy.2. Some agents risked their own success…

— Bull Theory (@BullTheoryio) August 30, 2026

So What Can Humans Do? 

A lot, actually. But it is not in the hands of us mere civilians. The biggest responsibility is on the regulators and policymakers. While the progression and development of Artificial Superintelligence is important, there needs to be some breaks.And several praiseworthy initiatives are already in place. In New York, Mayor Zohran Mamdani imposed a one-year ban on children from using AI up to 8th grade, affecting nearly 600,000 students. 

The EU AI Act is the world’s broadest binding AI law, which bans certain AI uses and requires developers of powerful general-purpose models to test for systemic risks, mitigate them, and secure their models. 

In the UK, MPs and Lords are pushing proposals that could give authorities an AI “kill switch” and potentially halt development of superintelligent AI in Britain.

Just because technology is everywhere doesn’t mean it belongs everywhere. The tech industry wants us to believe that AI in early education is not only inevitable, but necessary. We don't see it that way.This week, we announced a moratorium on all generative AI for public… pic.twitter.com/7Ivme25NNd

— Mayor Zohran Kwame Mamdani (@NYCMayor) September 4, 2026

So, the fear that Jacob Coxon described in his Tweet today is realized and share my millions of people, including regulators. How they act on it is another question that only time can tell.