He also explained why he no longer prefers expressing AI risks through a single probability.
Because when he left, he said, I think Anthropic is the most responsible player.”, it’s an interesting resignation.
Anthropic CEO Dario Amodei has said it is more useful to assess the risks posed by artificial intelligence by examining different possible paths for its development rather than assigning a single percentage to the chances of AI causing catastrophic harm. Addressing warnings from former Anthropic researcher Jacob Coxon and others about the rapid development of increasingly capable AI systems, Cooper asked Amodei: “Do you believe, earnestly believe that AI could kill all humans? What percent chance do you think it is? In response, Amodei stated, “It’s funny, I agree with Jacob much more than I disagree with him.
Instead of saying, it’s a 10% chance, that sounds like it’s a roll of the dice.
Amodei said Coxon’s concerns were not specifically directed at Anthropic but reflected broader concerns about the pace of the AI industry. He wasn’t calling out us, he was calling out the dynamic of the industry as a whole moving too fast,” he said. Breaking down the possible outcomes provides a clearer picture of the risks, according to Amodei. Joe Benton, who said he left Anthropic’s safety team two weeks ago, has warned that AI companies are racing to develop systems capable of recursively improving themselves.
“I think Anthropic is the most aware of these issues. “Again, I think it’s more illuminating to think in terms of how do you decompose those probabilities? What are the paths in which things go well, and what are the paths in which things go poorly? I think it’s much more illuminating to say, well, that forks into different paths. If we take the right paths, then the chance of something going wrong is very low. The comments come amid growing concerns from AI safety researchers. His concerns followed Coxon’s resignation from Anthropic, where he had worked on AI pretraining research. Coxon had warned that leading AI companies were moving towards self-improving superintelligence without adequate safeguards.