Why A Pause, Should Give You Pause

Why FLI's AI Moratorium just won't work

Yesterday, the Future of Life Institute (FLI) published the latest in a long line of calls to “pause” or “slow” AI development. As is common in such AI pessimism, there is a deep disconnect with substantiated reality. This letter is alarmist, and its proposals shouldn’t be considered realistic policy. That said, this take isn’t novel – the idea of an AI moratorium is increasingly common in DC. Given the prominence of its sponsors this letter may only increase the fire behind the moratorium movement. It demands engagement.

Lets dive in and explore the nature of the “race” that prompted these calls, the risks this letter foresees, the policies it proscribes, and the outcomes this option may bring.

The Pacing Misperception

The letter starts with a common frame: AI labs are in an “out-of-control race.” In some ways this is true. AI competition has certainly exploded. The release of Dall-E led to an raft of new AI art generators. ChatGPT prompted a “code red” at Google; months later Bard was released. In China we saw the AI fire flare as Baidu released Ernie Bot; other Chinese firms followed. Its true AI is moving fast and energy is undoubtable high.

Products don’t equal innovation, however. How many recent AI systems actually changed the technical game? Mostly one. GPT-4.

What we’re seeing isn’t a raft of innovation, it’s a raft of repackaging. Most of the splashy AI products in the last year use the same core technology, reshaped and rebranded.  It was no coincidence that Bing Bot and the original ChatGPT were released so soon before GPT-4. OpenAI wasn’t innovating at a month-by-month pace; these two products simply served as GPT-4’s test run. Bing, Dall-E, Github Copilot, and both versions of ChatGPT are all products either built on or deployed to trial the release of GPT-4. OpenAI’s aggressive GPT-4 product release schedule is what is moving fast, not actual innovation.

The term ‘race’ also seems to suggest back-and-forth competition. Today, OpenAI is and continues to be the undisputed LLM campion. We’ve yet to see anyone challenge their lead in recent years.  Sure, Baidu, Meta, and Google have tried to compete in this space but it’s clear the only company making waves is OpenAI.

So, what is the true pace of this supposed “out of control [LLM] race?” One model every three years. The time between GPT-3’s 2020 release and 2023’s GPT-4.

This is still exceedingly fast, but deceptively slower than most believe.*

Overactive Imaginations

Overpowering nonhuman minds. Complete unemployment. The end of civilization.

These fears FLI’s letter introduces would certainly be scary if they were based on any shred of empirical evidence. Once again, the AI risk camp needs to lay off the caffeine.

Yes, it is only human to hold measured concern about AI and the coming uncertainty. I myself am only cautiously optimistic and it would be unreasonable and uncharitable to completely dismiss worries. That said, policy cannot be based on science fiction. None of these concerns have any empirical basis and ignore the fact that society is damn good at some how keeping itself together.

Today, no AI is remotely close to becoming an “overpowering” nonhuman mind. OpenAI has only 375 employees. Given their ambitious rollout and the sudden regulatory scrutiny, they are almost certainly stretched thin and focused on putting out fires. Its almost unimaginable their tiny staff has the bandwidth to devote to GPT-5, let alone creating a sentient system.  

Turning to FLI’s unemployment concerns - labor is stickier than most would believe.  When I worked in electronic health records in 2015, a frequent refrain was “in the next 2 years, Dragon is going to replace all medical scribes.” Just today, I once more read the same message, but with a different product. The article claims that with the introduction of GPT “all scribe jobs are over.” Having heard this refrain for 8 years, I have my doubts. Labor reality is that managers resist change, unions defend jobs, and embattled professions use occupational licensure and regulation to entrench positions. Jobs are sticky and overnight unemployment just isn’t realistic.

As for the end of civilization? Let’s focus on more substantiated existential concerns. I’m more worried about Russia spinning up a nuclear war.

Half Baked Policy

Between the tempo of this “race” and these society ending concerns, the authors conclude that if industry cannot agree on a pause, the government must step in and impose a moratorium. As is often the case, suggesting government step in and solve the problem sounds obvious and simple. This solution, however, completely ignores the many complicated mechanics of government and policy.

First question: how?

The first step towards implementing a pause would be defining the regulatory target. What is an LLM? What is AI? These questions seem simple yet defining them in ways practical for regulation is going to be exceedingly hard. The best regulatory definitions are self-evident and clear cut; neither of these terms fit those qualities. Definitions are hard, and choosing one will incur arguments, lawsuits, and extended uncertainty. Meanwhile this unnecessary process will paralyze industry and distract from the many substantiated challenges that regulators should actually target. If we seek an “AI moratorium,” its unlikely we’ll get much further than writing the complicated regulatory glossary.  

Another question: who?

If the administration or Congress did decide to follow this path, dozens of agencies would be eagerly lining up to make the case that they should be in charge. AI is the future after all, and who wouldn’t want to take the lead?

We’ve seen this scenario before.  For years after crypto’s debut, the SEC and CFTC have been fighting over regulatory jurisdiction. Crypto is splashy and both want to lay claim. The result hasn’t been action or progress, but incurable crypto policy stasis.

If crypto policy remains frozen even in the face of the very real FTX meltdown, why would these unsubstantiated AI risks barrel through gridlock? For better or for worse, the moratorium camp hasn’t put any thought into the mechanics of their proposals. No amount of wishful thinking can burn through Washington realities.**

Unlikely benefit

Moratoria beget moratoria. If there were a pause, AI challenges are unlikely to be resolved. Six months will turn to a year, a year to a decade, and so on. When a technology is treated as an inherent risk, it will never seem safe and desired benefits will never be achieved. A pause will permanently snare researchers and regulators in a endless trap of “what ifs.”

The cold hard reality of LLMs seems to be that lab testing cannot resolve every issue. The universe of possible prompts the billions of users will feed into these systems simply cannot be predicted or trialed by any amount of researchers. The number of possible inputs is literally infinite. You cannot test infinity.

FLI’s pleasant sounding statement that “powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable” discounts these factors. They assume its possible to account for all risks, test for all risks, and somehow understand these risks before building. To test, you first need to build; to understand risk, you first need to fail.

Yes, lab safety testing is a necessary first step. I’m not arguing irresponsibility. But the truth is that labs cannot solve all problems. AI safety is best conceived as a problem solving methodology not a pre-deployment requirement. The only way to ensure safety is to deploy, see what goes wrong, and have structures readied to ensure quick reactions to emergent challenges. 

Recall that the internet began in a safe, academic, laboratory environment -not a far cry from what FLI imagines. Did these origins solve the internets problems? Not really. Even had the internet pioneers extended its ‘safe’ deployment phase, they could never have accounted for the modern challenges we face today. Likewise, it is unlikely a brief AI moratorium can help account for all of the unpredictable challenges we will face in five, ten, or twenty years. In the words of Tyler Cowen, “No one is good at predicting the longer-term or even medium-term outcomes of these radical technological changes.”

Opportunity Costs

To counter these over hyped risks, its good practice to meditate on the many positive what- ifs. What if these systems produced cancer curing drugs? What if they helped simplify complex legal jargon for the uneducated? What if they could scan software, and clean it of obvious vulnerabilities? While works in progress, many of these potential benefits are actually substantiated by early evidence. This technology could bring amazing change if we allow refinement to continue.

LLMs and AI in general shouldn’t be thought as inherent risks. Yes, none of these benefits are guaranteed. But they can’t be ignored. Opportunity cost is very real. By pausing AI, we also pause substantial medical and scientific research that could help millions. A moratorium is not only unjustified, and unworkable, it would place risks conceived by overactive imaginations and Silicon Valley egos ahead of desperately needed progress.

*AI is not a unified industrial category and there has been plenty of innovation outside of this niche. FLI’s letter, and most who want an AI moratorium, only focus on recent LLM innovation.

**None of this is to say that some “AI policy” is impossible - I’m a realist, not a defeatist. Though a practical regulatory definition of “artificial intelligence” likely is impossible. In future posts, I’ll expand on alternate paths.