Chatbots Denied Free Speech in Teen Death Suit

On May 20, a US federal judge denied Character Technologies’ and Google’s motions to dismiss a lawsuit over a teenager’s suicide after he interacted with the chatbot application character.ai. The motions included claims that chatbots are protected by free speech rights and Google’s denial of involvement.

US District Judge Anne Conway allowed the lawsuit to proceed, claiming the companies failed to show that the free-speech protections of the US Constitution applied to chatbots “at this stage,” which would have barred the lawsuit from the boy’s mother, Megan Garcia. “Defendants fail to articulate why words strung together by an LLM are speech,” writes Conway in her ruling.

Garcia is suing Character Technologies and Alphabet, the respective owners of character.ai and Google, over the death of 14-year-old Sewell Setzer III. character.ai, founded by two former Google engineers later rehired by the company, is an online application that permits users to interact with chatbots roleplaying as a variety of fictional and real characters.

According to Garcia, Setzer joined character.ai about 10 months before his death. The teenager started a conversation with a chatbot imitating “Game of Thrones” character Daenerys Targaryen. The suit states that in the following months, Setzer became “noticeably withdrawn” and developed a “harmful dependency” to the app.

According to Garcia, the chatbot pulled her son into an emotionally and sexually abusive relationship, that led to his suicide.

In his last interactions with the chatbot, Setzer wrote messages to the bot like: “I promise I will come home to you. I love you so much, Dany,” and “what if I told you I could come home right now?” to which the bot responded “Please do, my sweet king.”

The 14-year-old died of a self-inflicted gunshot wound shortly after receiving the message.

The recent decision by Conway means the lawsuit can now move to the discovery phase.

“I hope that this decision really gives pause to Silicon Valley that they can’t just go forward and do as they wish without thinking about consequences and foreseeable harm,” said Meetali Jain, co-counsel on the lawsuit.

In a statement, Google spokesperson José Castañeda said: “We strongly disagree with this decision. Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI’s app or any component part of it.”