Why It Matters
California Governor Gavin Newsom has moved to establish state-level oversight for artificial intelligence development, filling a regulatory vacuum he attributes to federal inaction. The executive order creates a dedicated panel tasked with recommending safety protocols for AI companies operating within the state.
What Happened
On Friday, Newsom issued an executive order creating a new panel to recommend safety regulations for artificial intelligence firms. The group is charged with developing rules that could include a mandatory “kill switch” for AI programs that malfunction or operate outside intended parameters.
This directive marks a shift from the governor’s previous stance. Two years ago, Newsom vetoed legislation that would have required AI developers to implement such shutdown mechanisms. At the time, he argued the measure risked stifling innovation and noted that smaller, exempt models could also pose dangers.
Newsom stated he felt compelled to act because President Trump and the Republican-led Congress have not passed comprehensive federal law requiring AI companies to report dangerous incidents. The governor’s move follows a similar plea from Pennsylvania Governor Josh Shapiro, who called for AI safeguards, including third-party oversight and greater transparency, the day before Newsom’s order.
State Sen. Christopher Cabaldon (D-West Sacramento), who chairs the state committee overseeing technology issues, will hold hearings on artificial intelligence this fall. Cabaldon emphasized the disparity in regulatory scrutiny between traditional businesses and high-risk technology firms.
“We regulate your barber or your sandwich shop more than we regulate [a company] telling us that they may be ending humanity,” Cabaldon said.
By the Numbers
Friday — Date Newsom issued the executive order creating the AI safety panel.
Two years ago — When Newsom vetoed a bill requiring AI kill switches.
June — Month Reps. Ted W. Lieu and Nathaniel Moran introduced federal legislation for AI shutdown capabilities.
2024 — Year Geoffrey Hinton won the Nobel Prize in machine learning; year Sen. Scott Wiener proposed his AI shutdown law.
Zoom Out
The push for state-level AI regulation occurs amid divided opinions on federal oversight and industry self-regulation. In June, Reps. Ted W. Lieu (D-Torrance) and Nathaniel Moran (R-Texas) introduced a bill requiring developers to have the capability to shut down powerful AI systems, granting federal officials intervention authority in cases of potential harm.
Industry leaders remain split on the necessity of new rules. Anthropic Chief Executive Dario Amodei called for an immediate slowdown in AI development pace, while Nvidia Chief Executive Jensen Huang stated publicly that no new regulation is needed. Geoffrey Hinton, a 2024 Nobel Prize winner in machine learning, expressed doubt on CNN that a kill switch would be effective.
President Trump has dismissed apocalyptic warnings about AI as a “hoax,” expressing concern that American companies will fall behind China if subjected to heavy regulation. Conversely, Jacob Coxon, a researcher for Anthropic who also worked for OpenAI, left his firm over concerns that AI companies are “gambling with our lives.” Connor Leahy of ControlAI noted the urgency, stating, “It’s becoming extremely clear that action is desperately needed.”
TechNet executive director Robert Boykin said the organization remains committed to working with Newsom on the new regulatory framework.
What’s Next
The newly formed panel will begin drafting recommendations for AI safety standards, including potential emergency shutdown protocols. Sen. Cabaldon’s committee is scheduled to hold hearings on artificial intelligence this fall, providing a legislative venue to discuss the governor’s executive order and broader regulatory strategies.