TALLAHASSEE, Fla. (AP) 鈥 A federal judge on Wednesday rejected arguments made by an artificial intelligence company that its chatbots are protected by the First Amendment 鈥 at least for now. The developers behind Character.AI are seeking to dismiss a lawsuit alleging the company鈥檚 chatbots pushed a teenage boy to kill himself.
The judge's order will allow the to proceed, in what legal experts say is among the latest constitutional tests of artificial intelligence.
The suit was filed by a mother from Florida, Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot that pulled him into what she described as an emotionally and sexually abusive relationship that led to his suicide.
Meetali Jain of the Tech Justice Law Project, one of the attorneys for Garcia, said the judge's order sends a message that Silicon Valley 鈥渘eeds to stop and think and impose guardrails before it launches products to market.鈥
The suit against Character Technologies, the company behind Character.AI, also names individual developers and Google as defendants. It has drawn the attention of legal experts and AI watchers in the U.S. and beyond, as the technology rapidly , marketplaces and despite what are potentially .
鈥淭he order certainly sets it up as a potential test case for some broader issues involving AI,鈥 said Lyrissa Barnett Lidsky, a law professor at the University of Florida with a focus on the First Amendment and artificial intelligence.
The lawsuit alleges that in the final months of his life, Setzer became increasingly isolated from reality as he engaged in sexualized conversations with the bot, which was patterned after a fictional character from the television show 鈥淕ame of Thrones.鈥 In his final moments, the bot told Setzer it loved him and urged the teen to 鈥渃ome home to me as soon as possible,鈥 according to screenshots of the exchanges. Moments after receiving the message, Setzer shot himself, according to legal filings.
In a statement, a spokesperson for Character.AI pointed to a number of safety features the company has implemented, including guardrails for children and suicide prevention resources that were announced the day the lawsuit was filed.
鈥淲e care deeply about the safety of our users and our goal is to provide a space that is engaging and safe,鈥 the statement said.
Attorneys for the developers want the case dismissed because they say chatbots deserve First Amendment protections, and ruling otherwise could have a 鈥渃hilling effect鈥 on the AI industry.
In her order Wednesday, U.S. Senior District Judge Anne Conway rejected some of the defendants' free speech claims, saying she's 鈥渘ot prepared鈥 to hold that the chatbots' output constitutes speech 鈥渁t this stage.鈥
Conway did find that Character Technologies can assert the First Amendment rights of its users, who she found have a right to receive the 鈥渟peech鈥 of the chatbots. She also determined Garcia can move forward with claims that Google can be held liable for its alleged role in helping develop Character.AI. Some of the founders of the platform had previously worked on building AI at Google, and the suit says the tech giant was 鈥渁ware of the risks鈥 of the technology.
鈥淲e strongly disagree with this decision," said Google spokesperson Jos茅 Casta帽eda. "Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI鈥檚 app or any component part of it.鈥
No matter how the lawsuit plays out, Lidsky says the case is a warning of 鈥渢he dangers of entrusting our emotional and mental health to AI companies.鈥
鈥淚t鈥檚 a warning to parents that social media and generative AI devices are not always harmless," she said.
___
EDITOR鈥橲 NOTE 鈥 If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988.
___ Kate Payne is a corps member for The Associated Press/Report for America Statehouse News Initiative. is a nonprofit national service program that places journalists in local newsrooms to report on undercovered issues.
