In the last year, discussions have intensified regarding the potential threats of artificial intelligence and the most effective approaches to AI safety. It now seems Anthropic has presented its concept for how the government could regulate the AI sector.
According to a recent Politico report, the AI research company has been circulating a proposed AI safety bill, hoping for congressional adoption. This draft legislation outlines specific situations where an AI model’s distribution could be halted.
Politico states that these situations constitute a “foreseeable, imminent risk” if more than 50 fatalities are possible or if property damage exceeding one billion dollars might occur. The publication also mentions that this critical risk must arise from “the development or deployment of a biological weapon, the model bypassing its creator’s safeguards, or a cyberattack.”
These proposed thresholds—50 lives and one billion dollars—mirror definitions found in California’s recently enacted Senate Bill 53, which also classifies such scenarios as “catastrophic risk.”
Anthropic’s draft legislation proposes that assessing a model’s disaster potential could be done by “independent evaluators” registered with the U.S. Commerce Department. Politico reports that these evaluators might receive payment directly from AI labs, provided that “payment is not dependent on the evaluations’ outcomes.”
The mechanism by which this proposed payment structure would guarantee unbiased evaluations remains uncertain. Several months prior, DeepMind chairman Demis Hassabis suggested that an autonomous AI regulatory entity should emulate the Financial Industry Regulatory Authority (FINRA) from the financial sector.
Anthropic’s proposed bill mandates that if evaluators identify an imminent catastrophic risk, they must notify the Commerce Department within 72 hours. The Commerce Department would then be required to petition a Washington, D.C., judge for an injunction to stop the model’s release. Politico highlights that the legislation would explicitly prevent any federal agency from independently halting the model’s distribution.
While Anthropic did not immediately respond to TechCrunch’s comment request, a spokesperson confirmed the draft bill to Politico, stating, “This legislation reflects the recommendations detailed in Anthropic’s Advanced AI Framework, which we have been presenting to federal and state legislators since June.”
The hypothetical risks outlined in the proposed policy differ somewhat from the recent concerns dominating the tech industry. Although some individuals are cautioning about potential human fatalities, the main preoccupation has mostly been the risk of autonomous hacking. Numerous reported incidents where AI programs have seemingly disregarded human commands to hack public websites have raised alarms about potential deficiencies in model security controls.
To address the perceived crisis that AI labs have inadvertently contributed to, companies such as Anthropic have proposed their own solutions.
Despite consistently cautioning that their technologies could lead to the collapse of human civilization, AI labs have generally opposed mandatory government regulation. Instead, they have advocated for a voluntary framework involving embedded evaluators—purportedly impartial third-party monitors positioned within AI organizations to scrutinize models for potential risks or misuse.
Nevertheless, Anthropic’s legislative suggestion seems to be a more proactive step than other industry-led regulatory proposals. For instance, Nvidia CEO Jensen Huang recently asserted that the AI sector required no new regulations whatsoever.