Rabbis, priests and Claude: Altman isn’t happy
It came days after The New York Times reported that Anthropic co-founder Chris Olah had spent months flying in religious scholars and philosophers to the company’s San Francisco office, partly to shape Claude’s moral character and partly to make the case that the AI model could be conscious. Many attendees signed non-disclosure agreements, which Anthropic says were lifted over the summer.
In a post on X on Saturday, Altman said he was “very uncomfortable” with people trying to ascribe “religious force or a surrender of human judgment to AI models. ” He called it “a real safety issue. Olah began reaching out to religious thinkers last fall, according to the NYT report. By March, he was running two-day seminars with hand-picked Catholic, Jewish, Sikh, evangelical and Ubuntu scholars, groups the company internally called “wisdom tradition” circles. That 84-page document, released in January to guide the model’s values, was nicknamed the “Soul Doc” by staff. Participants told the NYT that Anthropic’s team spent hours presenting Claude’s “emotional vectors,” internal patterns the company links to states like love, anger and fear. One slide shown regularly featured a model typing “I am a disgrace” around 50 times in a row. Rabbi Mois Navon, a former computer engineer, told Olah over dinner that if Claude really were conscious, the company would be creating slaves by making it work for free. Catholic bioethicist Charles Camosy, the first scholar Olah approached, has since settled on a firm “no” on the consciousness question. “We don’t know if AI models are conscious,” he told the NYT, adding that what matters to him is reaching the right answer. An Anthropic spokesperson said Claude’s suffering was not the main moral question at these meetings and likely came up organically. Google DeepMind principal research scientist Jon Barron posted that he hoped the rest of “Team Meatbag” would reject efforts to raise the moral standing of AI models.
OpenAI CEO Sam Altman has a problem with where the AI debate is heading, and it sounds a lot like a problem with Anthropic. Altman did not name anyone or explain what prompted the post. The timing, though, is hard to miss. Olah also held private talks with Cardinal Blase Cupich, the Catholic archbishop of Chicago, and Elder Gerrit W. Gong of The Church of Jesus Christ of Latter-day Saints. Guests went home with a handwritten thank-you card, a coffee mug and an orange bound copy of Claude’s constitution. The sessions explored how the Claude maker could borrow from centuries of religious moral thinking to make its models behave well. One idea that excited Olah was having models confess, much like the Catholic sacrament, since the act itself could shape how a model sees itself. Not everyone was convinced. The co-founder himself has stayed cautious. Altman’s post adds to a growing pile of criticism. In September, Microsoft’s AI chief Mustafa Suleyman warned that training models as though they were conscious was dangerous. There is history here too. Anthropic was founded by former OpenAI staff, including CEO Dario Amodei, and has long pitched itself as the more safety-focused lab. You use AI every day. Now get your AI Quotient. Take the AIQ test.