William Roper: “So, now you give the Devil the benefit of law!”
Sir Thomas More: “Yes! What would you do? Cut a great road through the law to get after the Devil?”
William Roper: “Yes, I'd cut down every law in England to do that!”
Sir Thomas More: “Oh? And when the last law was down, and the Devil turned 'round on you, where would you hide, Roper, the laws all being flat? This country is planted thick with laws, from coast to coast, Man's laws, not God's! And if you cut them down, and you're just the man to do it, do you really think you could stand upright in the winds that would blow then? Yes, I'd give the Devil benefit of law, for my own safety's sake!”
The recently-proposed model legislation from the "Center for AI Policy" (CAIP) creates a federal administration with arbitrary power to ban or regulate any machine-learning training process -- i.e., power to regulate any ML training of any size.
That is, the sometimes-mentioned limit for training runs of 10^24 FLOPs, beneath which the new administration is sometimes described as having no new authority, is just incorrect -- it doesn't consider the deliberately built-in capacity of the new administration to modify its own standards, with no substantial limits placed on this modification.
I will walk through the legislation and explain why this is so, before concluding with some reflections the AI safety movement's predilection for despotism and Platonic top-down solutions.
The "Responsible Advanced Artificial Intelligence Act" from the Center for AI Policy proposes a new "Frontier Artificial Intelligence Systems Administration" (FAISA). The purpose of the administration is to "oversee and regulate advanced general-purpose artificial intelligence systems."
Concretely, it (pg. 4) proposes four tiers of models, with different levels of regulation applied to each. The initial definitions of these tiers and the broad regulations imposed on them are as follows:
Low concern:
Medium concern:
High concern:
Open-weights AI projects would score 0 on a large number of these desiderata -- i.e., you cannot monitor them; you cannot prevent finetuning; you cannot recall the deployment -- so I take "high concern" to be the level that pretty much forbids open weights, although the document is rather reluctant to say this outright.
In general, the administration would have powers to ban the training, possession, or use of models at this level if they did not meet various requirements.
Extremely high concern:
I could spend more time on the vagueness of the fourth tier, but it's not really necessary to make my point.
The vital point is that these thresholds are the initial thresholds, rather than final thresholds.
And these tiers can be modified arbitrarily by the Administrator of FAISA. The text (pg. 4) states clearly: "The Administrator may modify any or all of these thresholds as set forth in Sections 4 through 6."
Section 6 speaks to this directly, and establishes that although the Administrator may make any these thresholds more constraining without any particular standard of evidence at all, he may not make them less constraining in the same way:
Except as otherwise modified by this section, the Administrator shall have full power to promulgate rules to carry out this Act in accordance with section 553 of title 5, United States Code. This includes the power to update or modify any of the technical definitions in this Act (including the definition of “frontier AI” and the definitions of “tiers of concern”) to ensure that these definitions will continue to adequately protect against major security risks despite changes in the technical landscape such as improvements in algorithmic efficiency. However, neither these definitions nor any rule promulgated under this Act may be altered by the Administrator so as to be more permissive of frontier AI development unless the Administrator first makes findings supported by clear and convincing evidence that such alterations will not significantly increase major security risks.
You'll note that this specifically anticipates tightening thresholds due, for instance, to improvements in algorithmic efficiency.
Unless I'm misunderstanding the law, after FAISA "updated a threshold," Congress could issue a disapproval of the rule altering the threshold, and prevent it from going into effect, beneath 5 U.S.C. § 802, during a 60-day window after the submission of the rule.
Congress has apparently issued such a disapproval only a total 20 times since 1996, when 5 U.S.C. § 802 came into existence, so it disapproves rules at the rate of about 3/4th of a rule per year. Literally 19 of these disapprovals were Democratic / Republican administrations disapproving rules that were passed during the last 60 days of of the prior Republican / Democratic administration. So -- it's not a particularly likely restraint on a rule-making body by any means at all.
However, even this great unlikelihood is too heavy a limit upon the Administrator's power for the drafters of this law.
Normally, regulatory agencies cannot simply re-pass a nearly identical rule after Congress disapproves if it is in "substantially the same form." This obviously makes sense -- given how unlikely Congress is to specifically disapprove a rule even once, if you could just re-submit a rephrased version of the rule then Congress would be even more nullified.
But the model legislation establishing FAISA has a special provision that specifically allows it to pass similar rules to those previously disallowed by Congress!
Because of the rapidly changing and highly sensitive technical landscape, a rule that appears superficially similar to a rule that has been disapproved by Congress may nevertheless be a substantially different rule. Therefore, a rule issued under this section that varies at least one material threshold or material consequence by at least 20% from a previously disapproved rule is not “substantially the same” under 5 U.S.C. § 802(b)(2).
I want to note that even without this provision, the freedom of regulatory power granted to FAISA seems absurdly broad. This provision brings it to actual parody.
Again the document states that the Administrator has a power and obligation to tighten thresholds -- but fails to mention a corresponding obligation to loosen them, should they be excessive.
No later than September 1 st of each year, the Administrator shall review each of the thresholds in section 8(a), together with the relevant definitions in section 3, and determine whether each threshold and each definition remains adequate to defend against major security risks. If any threshold or definition has become inadequate, then the Administrator shall promptly promulgate rules to appropriately strengthen or tighten the threshold or definition.
Imagine the following scenario:
The new Administration could lower the threshold for "low concern AI" from 10^24 FLOPs to 10^21 FLOPs, and of high-concern AI from 10^26 to 10^23. This would effectively outlaw making an open-weights model like the original Llama model from Meta -- not even Llama 2 or 3, but Llama 1.
(This is, notably, the kind of thing that highly-concerned AI-safetyists have specifically said they want to do. The StopAI people, for instance, want to outright ban AI training runs of more than 10^23 and require official government permission for AI runs to an literally unspecified degree below that.)
Congress might, in an uncharacteristic spasm of concern for limited government, decide this was a little too much and nullify the rule.
And then FAISA could change the threshold for low-concern AI from 10^21 FLOPs to 1.2 * 10^21, which is a 20% change, and reinstate the rule anyhow.
This isn't an accidental ability built into the act; this is extremely deliberate.
I've seen a number of people who care about AI safety say that this act seems like a good starting point. I've seen people praise it as a serious and worthy attempt. I've seen others even claim that the bill is too weak.
Look.
When the archetypical man-in-the-state-of-nature finds that something has gone wrong -- "Someone is stealing our sheep!" -- he gathers with his fellows, and institutes government, by giving Some Dude the Authority to stop the wrong thing -- "You, find who is stealing our sheep and figure out how to stop it."
Shortly afterwards, in this fake mythological origination of government, a second discovery is made -- "Fuck, we should have set forth clear rules for what Some Dude could not do without talking to us, rather than just giving him authority to do whatever he wants." Thus the origin of due process, democratic institutions, and proceduralism in general.
And shortly afterwards, the third discovery gets made -- "Huh, maybe there should be limits to what Some Dude can do -- even if he does talk do us?" Thus the origin of rights, specifically limited government, freedom of association, freedom of the press, and liberalism in general.
If you think AI is just going to kill everyone... I mean, I get it. You might just not think limited powers should apply to anything to do with machine learning at all. You think that we need arbitrary power to guard ourselves from the Devil.
In general, people like to destroy the features of limited government whenever they think they're dealing with something sufficiently bad. The Catholic fundamentalist says "Lo, freedom of speech is good, but heresy sends souls to hell so we must establish an Index of Forbidden Books." The NSA agent says "Lo, freedom of association is good, but non-visible-to-me online discussions allow terrorists to talk about things with each other, so we must ban civilian cryptography." And the AI safety guy from MIRI says, "Lo, limited government power is nice, but a superintelligent AI could sprout forth in mere days from any computer at all, so someone ideologically identical to me should have unlimited power over all ML training."
But -- in general -- civilization depends on taking a look at people who think that they've found something Sufficiently Bad that we should destroy these features of government and politely saying, "No. We will not."
I want to be clear -- I think even without the self-modifications allowing arbitrary powers, this would be a really awful act.
Something like 10^24 is just too low. I think implementing this act would prematurely cripple the mechanisms by which much human knowledge is generated, in the name of safety. (Seriously, take a look at the number of groups that have tried to ban Llama 2, and then look on ArXiV for the number of safety and interpretability papers that have depended on it. It's increasingly likely Meta has assisted safety research than most and potentially any AI safety org.) And I think that the likely corruptions of this kind of act could make it bad for AI safety like the National Environmental Protection Act is bad even just for the environment. But it is the provisions to let the new Administration extend its power however it wants to that make me think, "Yeah, I'm done. Whoever wrote this should just... not be writing laws, period, full stop."
If people who advocate for extreme AI regulation do want to make a Tsar of AI because they think the situation is just that desperate -- I hope they can say so more clearly than the discussion about this law says it. Like that would be a big step, let's at least be clear that is the step we would be taking. If we want to say, "Nah, government should have arbitrary authority over all ML training everywhere," I'd at least hope we could say that to ourselves before we take that step, rather than try to sneak it in like an intelligence agency sneaking in provisions for universal surveillance.
But mostly, I hope that people concerned with AI safety can come to the conclusion that a Tsar of AI is neither the best way to deal with AI risk nor a good feature to add to our government in general.