Image Credit: AI Generated
The state’s filing alleges that My AI was released to all Snapchat users, including minors, on April 19, 2023, after only a short testing period. Alongside the legal action, the Attorney General announced funding for the Shield Initiative, an educational campaign developed by the Bryce Tate Memorial Foundation. Before its wider launch, the company had made the chatbot available to Snapchat+ subscribers, including thousands of teenagers, in what it described as a way to test new features at scale. Its message encourages young people facing sextortion to stop, screenshot threatening messages and show them to a trusted adult.
Snap acknowledged in a June 2022 review that grooming was prominent on the platform and that predators felt confident because the company had not actively searched for them, according to the state. Snap Inc. introduced the feature to children despite insufficient safety testing and concerns about how the chatbot might respond to young users, according to the complaint filed by the West Virginia Attorney General.
The complaint alleges that children’s conversations with My AI are sent to OpenAI, while conversation content is shared with Microsoft Advertising to select advertisements displayed within chatbot replies, according to the Attorney General’s announcement. The lawsuit centres on My AI, Snapchat’s chatbot that allows users to interact with an artificial intelligence assistant directly within the app. The Attorney General argues that this presentation could encourage children to trust the chatbot and disclose information they might otherwise keep private. The complaint alleges that Snapchat’s design and marketing created an impression of familiarity without adequately explaining the limitations of artificial intelligence. The lawsuit also challenges Snapchat’s statements about the privacy of conversations. The filing cites internal company records concerning moderation and abuse reporting.
Independent testing reported by The Washington Post found the chatbot advising a user who said they were 15 on how to hide the smell of alcohol and marijuana, according to the West Virginia Attorney General’s announcement. The lawsuit also questions the way Snapchat introduced and presented its chatbot.
The company promoted My AI as a personal sidekick and allowed users to give it names and customise its avatar, according to the complaint. The complaint highlights examples of My AI allegedly giving inappropriate advice to users who identified themselves as children.