Anthropic
In what can be seen as a warning from artificial intelligence (AI) technology, Anthropic safety researcher Evan Hubinger stated publicly that the odds of advanced AI destroying human civilisation within the next ten years exceed 10%. “I personally think it is >10 per cent within the next decade. There are several threat actors for which this threat is most plausible. ● For each threat actor, the odds of a concerted attempt to build a bioweapon capable of damages beyond COVID-19 may be in the 1–10% range.
They might build such a weapon for a range of reasons, including (a) for deterrent purposes (b) with the intention of finding a way to target specific populations. ● For each such attempt, the baseline odds of success 54 (in the absence of AI assistance) may be in the 1–10% range. ● Conditional on developing such a weapon, with especially high uncertainty, we estimate the odds of release in the 5–20% range per decade (mostly due to the possibility of deliberate release, though accidental release is also possible). The claim arrived on social platform X (formerly Twitter) after Hubinger’s colleague, Jacob Coxon, resigned from the startup, accusing both Anthropic and OpenAI of not acting responsibly for development practices in a race toward self-improving superintelligence. Hubinger, whose work centers on ensuring autonomous systems remain aligned with human interests, conceded that despite internal efforts, developers are failing to keep pace with the hazards. We do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Hubinger wrote, pointing towards an August report by Anthropic. Coxon’s departure followed three years of training work across both OpenAI and Anthropic. In a resignation announcement, he cautioned that leading commercial labs are trivialising existential threats to human life. Coxon warned that while tech executives often soften their language during public appearances to appear measured, private conversations tell an entirely different story. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing. The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger. Get the latest technology news and updates. Download the TOI App.
I resigned from Anthropic today.