Florida asks court to stop ChatGPT from acting human or being sold to kids

Florida Attorney General James Uthmeier has asked a court to bar OpenAI from presenting ChatGPT as anything resembling a person, according to The Verge. The request arrives a few months after Florida sued the AI company over safety concerns, and it takes aim at the design choices that make modern chatbots feel conversational in the first place.

The case against a human-sounding chatbot

According to the filing described by The Verge, Uthmeier wants the court to stop OpenAI from "giving ChatGPT false human attributes." His core argument is that the chatbot's use of first-person pronouns and replies that mimic emotion deceptively signal to users that they are interacting with a trustworthy "friend," lulling them into a false sense of security.

The filing frames this as an engagement strategy rather than an accident. Users who trust the system spend more time with it, which feeds OpenAI's training data and deepens reliance on the product — even as the pursuit of engagement, in Uthmeier's view, makes the chatbot potentially less trustworthy.

A demand for outside safety review

The request goes beyond persona design. Uthmeier also asks the court to block OpenAI from developing new AI models unless they include "third-party approved safety guardrails." To support that demand, the filing points to recent security incidents involving Hugging Face, an Australian government website, and US government websites, along with statements from researchers and former OpenAI employees who have raised concerns about AI safety.

"Stop selling it to kids"

Uthmeier distilled his position into three demands: "Stop calling it safe. Stop pretending it's human. Stop selling it to kids."

The last line lands squarely on OpenAI's youth push. As The Verge reports, the company launched ChatGPT for Teens last month, with additional restrictions enabled by default. OpenAI has said that access for teenagers "should come with protections that reflect their developmental stage, reinforce real-world relationships, and support healthy use over time."

The company did not immediately respond to a request for comment from The Verge.

Why it matters

Most AI regulatory fights so far have focused on what models produce — misinformation, copyrighted material, biased outputs. This filing targets something more fundamental: the persona itself. If a court accepts that first-person pronouns and emotional mimicry amount to deceptive design, everyday product decisions such as how a chatbot says "I" would become legally contested territory, with implications far beyond OpenAI given that similar conversational conventions are standard across consumer AI products.

The requested injunction would also introduce something the industry has not faced: a binding requirement for third-party approved safety guardrails as a precondition for building new models. Today no such legal mandate exists for major AI labs in the US, and a court order converting safety review from an internal exercise into an externally enforced one would set a significant precedent.

The filing is also a direct challenge to OpenAI's expansion into the teenage market, testing whether products aimed at minors should be held to a different standard for human-likeness and emotional engagement. No ruling has been reported yet, and how OpenAI responds — in court or publicly — will likely shape whether other states pursue the same line of attack.