AI’s math breakthrough raises bigger questions
The TOI correspondent from Washington : A researcher at the Artificial Intelligence firm Anthropic quit his job on Tuesday with a grim warning over the rapid progress in the field which he fears could kill humanity as soon as 2030. Another observer announced airily that, having learned Anthropic’s alignment chief thinks AI has a greater-than-10% chance of killing everyone, he was going to “go play with my kids. OpenAI deployed roughly 10,000 AI agents working in parallel, which generated about 2.7 million messages and 130 billion output tokens on the Navier-Stokes effort, reaching their proposed solution in about 88 hours.
Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely,” Marks explained.
A researcher who has spent the past three years working on AI pretraining at OpenAI and Anthropic, he accused both firms of “racing straight to self-improving superintelligence and gambling with our lives,” arguing that increasingly capable systems could hack computer systems, conduct scientific research, acquire resources and eventually become sufficiently autonomous that humans lose control. More unsettlingly, he acknowledged that Anthropic does not yet have a plan to solve “alignment” for superintelligence and is not clearly on track to do so. Samuel Marks, another Anthropic researcher specializing in scalable oversight, added an even more uncomfortable observation: AI developers believe their technology could cause human extinction “in the next few years,” and, he said, the more senior the employee, the more concerned they tend to be.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. Coxon, who is himself among those building AI, wrote on X while announcing his resignation, igniting a firestorm of alarming — and some frivolous — responses. Coxon’s prognosis could have been dismissed as the bitter farewell note of a departing employee — except his colleague, Evan Hubinger, Anthropic’s Alignment Science lead, essentially endorsed his views. “Jacob is correct here—we really do earnestly believe AI could kill all humans! Hubinger wrote. “Why do AI developers continue despite the risk? Not everyone shares his alarm. On X, one commentator effectively prescribed the standard emergency procedure to address the issue: “Jacob, go back inside and unplug all the computers. The problem concerns equations describing fluid motion — mathematics underpinning everything from airflow and aircraft to weather and blood flow. If it survives scrutiny of mathematicians, the implications could be enormous. Mathematics sits underneath physics, chemistry, engineering, computer science and much of modern medicine. AI that can generate genuinely novel mathematical insights could potentially accelerate scientific discovery across all of those fields.
AI’s math leap and humanity’s dilemma Amid all this came a mathematics bombshell, with OpenAI announcing on Tuesday that an internal AI system produced what it says is a solution to the Navier-Stokes existence and smoothness problem, one of mathematics’ seven Millennium Prize Problems.
More than 80% of the code merged into Anthropic’s codebase was reportedly authored by Claude by May 2026, while the company says its engineers were merging roughly eight times as much code per day in the second quarter of 2026 as they did in 2024. Because if AI becomes much better at designing algorithms, running experiments, writing code and conducting research, the pace of improvement could accelerate dramatically, this matters.
“Unlike traditional software, we can’t “program” AIs to behave how we’d like. Anthropic says that threshold has not been reached and is not inevitable. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this,” he noted. The technical concern among AI gearheads is recursive self-improvement (RSI): AI systems increasingly helping humans build better AI systems, eventually reaching the point where an AI can substantially design and develop its own successor. But its researchers say the trend is moving faster than expected. And if the improvement loop becomes fast enough, researchers worry that safety mechanisms developed at human speed could be hopelessly outpaced. Which brings human civilisation to the central paradox: The machines may be capable of helping us cure diseases, invent materials, solve physics problems and accelerate science — while some of the people building them simultaneously worry that the machines might eventually become too powerful to control. But precisely how a future system would go from brilliant software engineer to exterminator of Homo sapiens is still in the realms of speculation. All this leaves humankind with a modest assignment for the rest of the decade: Build machines vastly smarter than ourselves; teach them human values; make sure they don’t deceive us; make sure they don’t escape our control; make sure they don’t develop their own goals. And, ideally, do all this before they become clever enough to notice that we have assigned them the job. Get the latest technology news and updates. Download the TOI App.
Today’s AI helps humans build tomorrow’s AI; tomorrow’s AI might help build the AI after that.
Anthropic, meanwhile, has released Claude Fable 5.1 and the more restricted Mythos 5.1 aimed at high-end cybersecurity and biological research and remains available only to vetted organizations.
OpenAI has just released GPT-6 Astra, which it describes as its most capable model yet, reaching the company’s “Critical” level for cybersecurity capability — meaning it can perform sophisticated cyber tasks that raise substantially greater misuse concerns. Coxon says the problem is a classic prisoner’s dilemma: each company may believe it would be safer to slow down, but fears that its rival will continue regardless. The race is already getting uncomfortable. Incidentally, Anthropic’s brand was built partly around being the AI company that would be more cautious than OpenAI. Now Anthropic’s own researchers are warning that Anthropic itself is caught in the race.