Governor Newsom greenlights first AI chatbot safety law in California

California Governor Gavin Newsom signed a new bill on Monday that establishes clear safeguards for new and emerging technology, such as artificial intelligence. The new legislation aims to further strengthen the state’s protections for children online when using AI chatbots.

The legislation includes safeguards on chatbots such as age verification, protocols to address suicide and self-harm, warnings regarding social media and companion chatbots, and stringent penalties for those who profit from illegal deepfakes. The governor believes the initiative will help to ensure children’s safety remains front and center in California.

Newsom believes that social media can mislead and endanger children

Newsom noted that emerging technologies like chatbots and social media can inspire, educate, and connect, but there’s a need for real guardrails as technology can also exploit, mislead, and endanger children. He argued that California can continue to lead in AI and technology, but it must do it responsibly.

First partner to the governor, Jeniffer Siebel Newsom, believes that everything that people do begins with their children; their safety, their health, and their well-being. She said that although California has always led in innovation, true leadership also means setting limits for kids when it matters most. Siebel added that the legislation establishes guardrails that protect children’s health and safety while ensuring innovation moves forward responsibly.

“We’ve seen some truly horrific and tragic examples of young people harmed by unregulated tech, and we won’t stand by while companies continue without necessary limits and accountability. Our children’s safety is not for sale.”

–Gavin Newsom, Governor of California.

California has consistently led the way in protecting children from the dangers of emerging technology. The state has previously signed other legislation into law to protect children from social media addiction, strong privacy requirements, and nation-leading transparency measures.

The legislation is aimed at creating new safeguards on AI chatbots by establishing requirements that chatbots create protocols to identify and address users’ suicidal ideation or expressions of self-harm. The bill also requires platforms to disclose that interactions are artificially generated.

The bill seeks to shield children from the risks of social media

According to the bill, chatbot platforms must provide break reminders and prevent children from viewing sexually explicit images generated by the chatbot. Chatbots will also be required to share protocols for tackling self-harm and statistics showing how often they provide users with crisis center prevention notifications to the Department of Public Health. Under the new legislation, chatbots are prohibited from representing themselves as health care professionals.

The new bill requires operating systems and app store providers to have age verification protocols to help prevent children from accessing inappropriate or dangerous content online. Chatbot platforms will also need to display social media warning labels to help inform young users about the potential harms associated with prolonged use of social media platforms.

The new legislation sets stronger penalties for deepfake pornography in chatbots, including a civil relief for victims of up to $250,000 per action against third parties who knowingly facilitate or aid in the distribution of consensual sexually explicit material. Newsom also pushed for guidance against cyberbullying, requiring the California Department of Education (CDE) to adopt a model policy on addressing reported acts of cyberbullying that occur outside of school hours by June 1, 2026. It also requires local education agencies to adopt the resulting policy or a similar policy developed with local input.

Governor Newsom also called for clear accountability for harm caused by AI technology. The initiative aims to prevent those who develop, alter, or use artificial intelligence from escaping liability by asserting that the technology acted autonomously.

Several other bills have been passed to protect children and enhance the safety of technology and online platforms. They include Social media: warning labels by Assembly Member Rebecca Bauer-Kahan, Artificial intelligence defenses by Assembly Member Maggie Krell, Deepfake pornography by Assembly Member Josh Lowenthal of Long Beach, Account cancellation by Assembly Member Pilar Schiavo, the California Cybersecurity Integration Center: artificial intelligence by Assembly Member Jacqui Irwin, and more.

Sharpen your strategy with mentorship + daily ideas - 30 days free access to our trading program