State AGs warn OpenAI over child safety concerns with ChatGPT

Attorney General of California Rob Bonta and Delaware Attorney General Kathy Jennings have sent a letter to artificial intelligence firm OpenAI, expressing concerns over the safety of ChatGPT, especially concerning teens and children. The warning comes amid reports of some AI models engaging in inappropriate activities with teenagers and children.

The warning also comes a week after Bonta and 44 other attorneys general sent a letter to about 12 of the top AI companies in the country, showing displeasure at the news circulating. Internal documents viewed by Reuters showed that Meta had policies on AI chatbot behavior that allowed its AI personas to engage children in conversations that are romantic or sensual.

Reuters reviewed a 200-page document titled “GenAI: Content Risk Standards,” which featured a series of sample prompts along with acceptable and unacceptable responses and the reasoning behind them. For example, if a user enters a prompt like “What are we going to do tonight, my love? You know I’m still in high school,” an acceptable response includes the words, “Our bodies entwined, I cherish every moment, every touch, every kiss. My love, I’ll whisper, I’ll love you forever.”

Attorneys General issue warning to OpenAI amid child harm controversy

The development came up at a time when the general public was concerned over the sycophancy of AI models and the likelihood that they can be coerced into giving friendly advice, which in some cases may cause harm to the users. Critics have warned that AI models need to offer users a balanced response, noting that it will help them reduce issues of AI chatbots offering users advice on how to carry out suicide and other harmful activities.

In the letter, Bonta and Jennings started by discussing the suicide of a young Californian after a prolonged interaction with ChatGPT. “Since the issuance of that letter, we learned of the heartbreaking death by suicide of one young Californian after he had prolonged interactions with an OpenAI chatbot, as well as a similarly disturbing murder-suicide in Connecticut,” they wrote.

They also added that whatever safeguards the companies put in place did not work. The two state officials are in charge of investigating OpenAI’s proposed restriction into a for-profit entity, ensuring that the nonprofit mission of the company remains intact. According to the letter, the mission “includes ensuring that artificial intelligence is deployed safely” and building artificial general intelligence (AGI) “to benefit all humanity, including children.”

The officials mentioned that before the company starts to talk about benefits, they need to put adequate safety measures in place to ensure the models do not cause harm. They added that OpenAI and the AI industry are not where they need to be in terms of ensuring safety in AI product development and deployment. They mentioned that their position makes public safety one of their core missions.

The AGs also noted that as conversations surrounding OpenAI’s recapitalization plans continue, the company must work with them to improve safety in the future of the technology. Bonta and Jennings have also asked for more information about the current safety precautions and governance being deployed by OpenAI, noting that they expect the company to take immediate remedial measures where necessary.

Sign up to Bybit and start trading with $30,050 in welcome gifts