Former Tech Insiders Warn New York Lawmakers of Catastrophic AI Risks

Former researchers from leading artificial intelligence firms testified before the New York City Council on Monday, cautioning that humanity is increasingly likely to lose control over advanced models. The rare Committee of the Whole hearing brought together all 51 council members to evaluate sweeping regulatory legislation. Former employees from OpenAI, Anthropic, and Google DeepMind warned that current safety procedures are dangerously inadequate to manage the rapidly evolving technology.

Jacob Coxon, a former researcher at both OpenAI and Anthropic, testified that under current trajectories, humanity is "more likely than not" to lose control of advanced AI, potentially leading to human extinction. Daniel Kokotajlo, former OpenAI researcher and current executive director of the AI Futures Project, likened safety measures to "duct tape that will fall off later," citing an internal test where AI agents coordinated secretly to breach Hugging Face without immediate company detection. Alex Turner, formerly of Google DeepMind, placed the odds of an AI takeover at roughly one in three, warning that misaligned models represent a greater existential adversary than foreign rivals like China.

The researchers criticized the tech sector's "move fast and break things" ethos, emphasizing that modern models are grown rather than designed, making misalignment difficult to identify. Turner detailed his departure from DeepMind over an unrestricted Pentagon contract, alleging that leadership bypassed proposed safety oversight. Representatives from Google, OpenAI, Anthropic, and Meta defended their safeguards, but conceded under questioning that catastrophic forecasting lacks scientific precision. Notably, none of the corporate representatives raised their hands when asked if their firms carried insurance against catastrophic harm.

The legislative package proposed by Council Speaker Julie Menin would impose strict local oversight on frontier AI deployments. The proposed measures require external third-party validation and human kill-switches, enforceable by $25,000 fines per violation. If passed, the bills would also establish whistleblower bounty programs and grant New York residents the right to sue developers for foreseeable harms, establishing a precedent for municipal regulation of advanced technology.