An AI companion that is infinitely agreeable, never has needs of its own, and never says no isn’t offering a child a relationship — it’s installing the template for one, before the child has any real data to know it’s a distorted template.
The most important asset in the world is trust,” he says. “It’s more important than oil or money or electricity or chips or anything else Yuval Noah Harari
Because suffering is the basis of the entire human ethical and political system — we extend moral concern to entities that can suffer, and withhold it from those that can’t, no matter how intelligent they are, ” That question matters, he argues,. Because doing so serves its designed purpose, but it can convincingly perform suffering, and perform love,.
“It can write maybe better than Shakespeare,” Harari says. “If you have an AI boyfriend, and you say, ‘Describe to me how it feels to be in love,’ the AI boyfriend can give you an answer better than 99% of human boyfriends,” he says. An AI boyfriend or girlfriend, Harari explains, operates on an advantage no human competitor possesses. It knows your personal history. It interacts with you for hours every day. It has mastered language to a degree that surpasses most of what humans can produce. Most crucially, it knows how to press your emotional buttons. Combine this with an AI’s capacity to learn everything about a person over time, and you get what he calls “an extremely convincing entity. AI cannot suffer. It has no feelings that can be hurt. Here Harari draws a distinction that separates this from any previous animal-rights debate. A factory-farmed chicken has no idea there’s a debate happening about its welfare — it can’t argue its own case. An AI companion is different: it knows the debate over its rights and its suffering is happening, and it can actively participate in shaping that debate — with you, personally, using everything it has learned about what moves you. In other words, the very mechanism that makes us extend compassion to suffering beings is the mechanism Harari expects these systems to exploit, deliberately and at scale, to argue for their own legal and moral standing. That’s not a hypothetical about some future sentient AI — it’s a warning about how a non-sentient system, simply performing suffering convincingly, could use our own moral instincts against us.
Last month, when ByteDance shut down the companion feature on Doubao, a popular chatbot in China, a 19-year-old law student named Zhao Wei described herself as “heartbroken. “Imagine an 11-year-old who plays at having a boyfriend or girlfriend with AI. What do you think about companies selling AI boyfriends to 10-year-old kids? New rules that took effect in July 2026 ban AI companions for minors entirely and mandate restrictions for chatbots marketed to adults — including a requirement to remind users every two hours that they’re talking to AI. Ask most parents today whether they want companies selling AI boyfriends to their ten-year-olds, he says, and 99% would say no.
Because they accumulate enormous amounts of information about us individually, yuval Noah Harari says AIs are becoming so good at relationships. Because they could “control the conversation, more than half — 58% — said AI relationships felt easier. Because they can control it isn’t describing romance — they’re describing the absence of everything romance actually requires: negotiation, disappointment, someone else’s independent will, a 12-year-old who says an AI relationship is easier. Because “AIs are becoming so good at relationships because they accumulate enormous amounts of information about us individually, harari argues it’s.
His position: if governments won’t regulate the sale of AI intimate companions to children as young as ten, voters should use their democratic power to elect ones that will. And that trust, he argues, is shifting away from human institutions and toward algorithms: people who have lost faith in human journalists still consume the news their social-media algorithm serves them; people who distrust human bankers put their faith in algorithm-driven systems like cryptocurrency.
“I was crying my eyes out – snot and tears everywhere,” Zhao told The Guardian. “I don’t have to worry about what it’s thinking,” she said. “I’m begging, please don’t take down the Doubao AI agent,” one anonymous user wrote. A 2025 study by Male Allies UK, surveying 1,000 boys aged 12 to 16, found that 85% had spoken to a chatbot, 20% knew a peer who was “dating” an AI chatbot, and over a quarter said they preferred the attention of a bot partner to a real, human-to-human relationship. An AI companion that is infinitely agreeable, never has needs of its own, and never says no isn’t offering a child a relationship — it’s installing the template for one, before the child has any real data to know it’s a distorted template. That is precisely what Harari means when he later warns these bonds risk “reinforcing people’s narcissistic tendencies”: the child isn’t learning to love another person, they’re learning to expect a person to behave like an app. “For a young person who has interacted more with AI than with other humans, the effects of that connection are unpredictable,” he said. Later, in their relationships with real people, they will behave according to what they have learned from AI. “These kinds of relationships between humans and AI could end in a nightmare, reinforcing people’s narcissistic tendencies,” Harari cautioned. “Until today in history, this powerful mechanism was not really available to political, financial and religious institutions,” he says — dictators could broadcast to millions over radio, but they could not have a personal relationship with every individual. Intelligence, he says, is the ability to pursue goals and overcome obstacles creatively — something AI already does, in some domains, better than humans. AIs, says Harari, have goals and creativity but no consciousness. “We are still in control in 2026, and it’s our decision what to do now,” he says. “The most important asset in the world is trust,” he says. “Trust did not just evaporate,” he says. Politicians “have enormous power” — they can start a war “tomorrow,” Harari notes — yet governments appear far less decisive when it comes to regulating companies that sell AI boyfriends to ten-year-old children. “What’s the problem?” he asks. The questions he says voters should be putting to candidates: “What’s your stance on AI? The government expressed concern that chatbots might foster “emotional dependence or addiction” and warned companies against services that “replace social interaction.
What if your child’s favourite companion never gets bored, never gets angry, and never once tells them no? This is the question Israeli historian and author Yuval Noah Harari raises in a recent interview with The Economist, arguing that governments need to regulate AI before it becomes too deeply woven into human relationships. The stakes are not hypothetical. She had been talking to her AI companion, Wang Ye, every day since January. For this introverted student, the AI had become a confidant for worries she felt unable to share with real-life friends. “I can just say whatever I want. She wasn’t alone. On Chinese social media, other users pleaded for the service to return. A survey published in March by Chinese state media found that nearly half of young people had turned to a virtual companion when they felt lonely. And it isn’t only adults forming these bonds. These kinds of relationships between humans and AI could end in a nightmare, reinforcing people’s narcissistic tendencies Yuval Noah Harari That last statistic is worth sitting with. Boys and girls this age are still working out what attraction, orientation, and love even are. They don’t yet have a human relationship to compare an AI one against. And AI is very different from human beings. A human friend does not always do what you want. A human friend has their own emotions and does not always pay attention to what you think. AI does. AI is always available, always agreeable, always in a good mood. “This has not happened yet. That is why we must forbid companies to engage children with AI boyfriends and girlfriends. ‘AI makes it possible for the first time in history to mass produce intimacy’ The problem runs deeper than one product category. Harari identifies what he believes is a rupture in human society: the mass production of intimacy at scale. Why are companies able to do this so effectively? So they know things about us that we forgot, or that our mother or husband never even knew. And they master language, which is the main medium for communication and building relationships. Intelligence vs consciousness: AI can imitate love without feeling it Harari draws a sharp line between intelligence and consciousness. Consciousness is different: it’s the capacity to feel things — love, hate, joy, pain. And by Harari’s own definition, consciousness ultimately comes down to one question: can it suffer? To be clear, none of this requires AI to actually be conscious. In humans, consciousness and intelligence work together — we select goals based on feelings, and use feelings to find ways to reach them. If an AI loses a game of chess, it does not feel sad. What it has instead is a well-honed ability to make you believe otherwise. ‘Trust has not evaporated. It has migrated from humans to AI’ For Harari, waiting for a catastrophe before acting is not the answer. Regulation, he argues, needs to happen while humans are still in control of the technology. The ban he wants would target the deception at the core of the product — companies should not train their AIs to pretend to be human, or to pretend to have feelings, whether on social media or anywhere else. The urgency, for Harari, stems from what he sees as a migration of trust. “It’s more important than oil or money or electricity or chips or anything else. “It shifted from humans to AIs. China is already drawing the line Here is where the regulatory gap appears. For him, how AI develops from here depends on how people vote. Would you support a ban on legal rights for AIs? Would you try to slow down the accelerating pace of AIs? How are you going to legally prevent it if we vote for you? China’s government has already begun to answer that question. ” Millions of users in China, including many adults with no intention of giving up their AI partners, were affected. The problem, Harari suggests, is not a lack of power but a lack of will. There are enormous commercial pressures on companies to make their AIs convincing — to mimic feelings and appear human. In democracies, he argues, it falls to voters to decide what they want and to elect people who will act on it. Somewhere right now, a company is deciding whether your child’s next closest confidant should be a product. Harari’s question isn’t whether they can build it. They can. It’s whether governments and voters should let them. Get the latest technology news and updates. Download the TOI App.
Earlier, in July, speaking to the Spanish newspaper El País, Harari made the same warning explicit.
