The dispute points to a much bigger question surrounding the rapid development of AI.
Olah had accepted an invitation to speak at the event on May 25, but reportedly considered pulling Anthropic out after disagreeing with the Pope’s position on AI consciousness.
The disagreement surfaced after Anthropic co-founder Christopher Olah reviewed the Pope’s first encyclical, “Magnifica Humanitas”, ahead of its launch at the Vatican. Olah has encouraged religious thinkers to consider the possibility that Anthropic’s Claude models could have something resembling moral status, according to The New York Times. The company has held private discussions with scholars from around the world and required participants to sign nondisclosure agreements, according to the Times. Anthropic’s in-house philosopher Amanda Askell said she wants the company’s models to be “the best of us”. Olah, meanwhile, has argued for “some shared notion of goodness that cuts across society in some very broad way,” according to the Times. I don’t know what that means, but I think it warrants ongoing discernment,” Olah said, according to the Times.
Anthropic has spent months consulting religious scholars and other experts about how AI systems should behave and what values they should follow. “I will be honest: We keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.