Red Flags in the AI Executive Order about “Dual Use” Models

The government's "dual use" definition has some surprises for the AI industry.

Image: “Janus” by Morgan Schmorgan. Unedited image from Flickr used via CC BY-NC 2.0 DEED.

AI innovations are coupled with mounting concerns over their alignment risks. For instance, OpenAI’s technical report on GPT-4 revealed that the technology poses challenges with respect to hallucinations, harmful content, disinformation, privacy, proliferation of weapons, and more. Since it’s trained on large Web information, early versions of GPT-4 could provide users with instructions on how to manufacture bombs and where to buy illegal weapons online. These issues were discovered—and mitigated—through “red-teaming,” the act of emulating an adversary’s attack on a system. Naturally, concerns have risen about how the technology should be governed given the risks it poses.

On October 30^th, President Biden's AI Executive Order was issued. This executive order (EO) seeks to guide the advancement and governance of AI under a set of policies and priorities. A goal of the EO is to prevent AI models from being used for purposes outside of their intention that have negative implications for national security, the economy, health, and safety. These so-called “dual-use”—having both civilian and military applications—AI models are defined in the EO as follows:

(k)  The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, such as by:

             (i) substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;

             (ii) enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyber attacks; or

             (iii) permitting the evasion of human control or oversight through means of deception or obfuscation.

Models meet this definition even if they are provided to end users with technical safeguards that attempt to prevent users from taking advantage of the relevant unsafe capabilities.

Due to its lack of specificity, many companies and models could be affected by this EO. “Generally uses self-supervision” can be interpreted many different ways and doesn’t narrow down AI models on its own. Terms like “broad data” may end up covering models that have no training data of concern to national security. Nonetheless, likely covered companies and models include OpenAI (ChatGPT), Google (Bard), Meta (LLaMa), and Anthropic (Claude)—language models that are trained on a massive collection of information. Even visual generative AI like Midjourney or OpenAI’s DALL·E 2 could be deemed “dual use” under the EO since these models could generate propaganda or false imagery. Furthermore, there are many open-source language models that may fit the dual-use definition, including UL2, Bloom, StableLM-Alpha, FastChat-T5, and h2oGPT.

Security officials could even interpret the dual-use definition to include recommendation algorithms (algorithms to recommend something, like a YouTube video or Amazon product) that have billions of parameters and use self-supervision. While such algorithms are designed for a single purpose, their dual uses are conceivable. Web search engines query a breadth of information and may have alignment risks, such as suggesting a search involving sensitive security subjects. 

In addition, there are many drug discovery companies that use AI to generate new molecules. The EO may cover such companies as the technology could potentially be repurposed to generate harmful substances or weapons. AI models are very plastic. Speaking in generalities, any model can be “easily modified” to perform any task provided appropriate and sufficient data. What a model is capable of is independent of its weights or parameters. For instance, drug discovery technology seeks to generate new molecules, but is trained on large drug-oriented chemical datasets. A model trained on this type of data will ideally be good at generating new drugs. However, the same model could be trained to synthesize harmful chemicals provided the right data to do so. Does every company who uses such a model need to report their activities? If the EO is interpreted too broadly, companies in such accelerating industries could begin to slow progress depending on the expectations of these contingencies. Further specificity is needed to clarify how “easily modified” should be understood.

Finally, companies that perhaps should be affected may not be. Models that could be dual-use could be compressed such that they have fewer than “tens of billions” of parameters. It is common practice to make models smaller so that they are faster and cheaper to run. Studies have shown that 10% or less of the parameters may be necessary to achieve good performance. Models can also be subdivided into smaller models that perform smaller tasks. This could look like breaking down a question-answering system into one model that handles math questions, one that handles history questions, etc. The model that powers ChatGPT, GPT-4, is actually not a single model but a “mixture of experts”—a collection of models in which each is elected to solve a task in which it is most apt. Typically, another model controls which model is selected to solve a task. So, is this still a single model or are they separate, and how should relevant activities be reported?

A good inclusion in the dual-purpose model definition is that “[m]odels meet this definition even if they are provided to end users with technical safeguards.” This is important as safeguards are imperfect. For example, it is common practice to use a large language model (LLM), like ChatGPT, itself via prompt engineering to safeguard what is shown to a user. If the LLM is imperfect (all are), then there will be edge cases that this safeguard will not protect from. Examples of this can be seen in the GPT-4 technical report. The reason why these models are still treated as dual-use is that they’re black boxes. If engineers had a better understanding of a model’s learned knowledge, they could remove anything that, for example, could describe how to build a bomb. White box (or glass box) models on the other hand allow for such control. However, their performance on the same tasks as black box foundation models is typically quite lower with present technology. Since a goal of the EO is to establish safe, secure, and trustworthy AI, this could be a promising direction for innovation.

This EO targets important challenges presented by AI innovations that have implications for  national security, the economy, health, and safety. Hopefully, the feedback period from academia, industry, and other agencies will yield fruitful insights. It is possible that its breadth of interpretation will settle harmoniously over time as a result of this collaboration. Nonetheless, the imprecision of the “dual use” definition could give national security officials new powers over many mainstream and open-source models.