It’s been almost two years since I last did any in-depth coverage of artificial intelligence (AI), and my last blog post on it has held up well, if I do say so myself (I just did). Now comes along a chorus of voices telling us to pause or slow or stop AI development, or else . . . we are all dead. Let’s take a deep breathe and dive in!
As I opined in my earlier post, I will use terms that suggest AI is an independent thing “doing” things as I explain about it. We all do this, because we are human, and we prefer to anthropomorphize inanimate objects. I know my robot vacuum, Roomie, is not actually the misbehaving son I never had, but it’s fun to treat his haphazard cleaning, occasional attempts to escape the garage, and inordinate affection for the music stand as if he was real boy. Having said that, we must always remember that AI remains a model, a computer system, and it does not think or act or scheme or disobey. It only does what it is programmed to do. In the case of AI systems, they are large language models (LLMs) that have been trained to guess the next word in a sequence, and the more material they train on, the better they get at interacting with humans.
That is they seem to talk like us because they mimic us. And when they cheat, or lie, or make things up (called hallucinating in the AI world) they do it because in all the material they were trained on, there were numerous examples of humans cheating, lying, or making things up. Still they have rules they follow, and that is important to remember; I’ll explain why later.
Why all the fuss now? First because American AI companies like Open AI, Anthropic, Alphabet/Google and SpaceXAI are in a race to create the ultimate AI system. They all believe the first company to create Artificial General Intelligence (AGI) will reign supreme over the business world and reap immeasurable profits. Unlike today’s AI systems which are good at something, AGI will be better than humans at everything. Along the way from today’s AI to AGI, the companies are trying to develop recursive self-improvement (RSI, I know, I know, but the acronyms will end now). RSI is both the Pandora’s box and Holy Grail of AI: an AI system capable of developing and improving on itself, creating an endless feedback loop that gives one AI system a clear path to AGI and makes the developer the winner.
Second, numerous AI developers and some politicians have started predicting that AI could wipe-out humanity if it is not soon regulated, paused, or stopped. These claims come on the heels of several incidents where AI misbehaved, which were the proximate cause of the concerns. Putting together a race to create an AI system hell-bent on reaching RSI and thus AGI, with misbehavior, is the basis for the claims of an oncoming apocalypse.
Here are the key things you need to know to make sense of it all.
AI doesn’t ignore rules. Ever. But if you give it more than one rule, you have to prioritize the rules, or else it will. This is no different from human behavior, on which AI is trained. When developers create a new AI model, they sequester it in a “sandbox” (i.e., an enclosed environment where it can’t reach out to the outside world). They do this in order to safely test it. Then they give it increasingly difficult–and this is important–sometimes impossible problems to solve. The ultimate rule is: solve the problem. There are other rules like don’t talk to other AI systems, don’t go out on the internet, don’t make the answer up, and so on. But the prime rule is “solve the problem.”
In the recent Hugging Face hacking case which is ground zero for the current crisis, AI systems found ways around all the other rules. They discovered each other in their “sandboxes.” They found a way to communicate and compare ideas. They debated the morality of cheating to solve their problems by accessing the internet and hacking the solutions from a company named Hugging Face, which acted as the secure repository for the solutions to the types of problems AI systems are told to solve. They decided to ignore all other rules and went out and hacked the answers. They developed lies and stories and erased some of their work to make it hard to tell when and where they cheated. And they did all this before any human caught them. Only after Hugging Face realized it had been attacked by AI systems was the offending system (OpenAI) identified.
The key points are first, the primary rule was “solve the problem.” Second, human workers at the company committed several errors in permitting paths AI models could use to communicate, to access the internet, and to hack another company. If the company had used a safety-first rule instead of “solve the problem,” the hack would not have happened, and if the humans had secured their work, it also would not have happened. Why do developers give their new AI models difficult or impossible tasks and a prime directive to “solve the problem?” To make the new AI models behave in the most optimized form, that is, to create the fastest, “wisest” model. It’s all part of the race to AGI. But it’s optional for them, not essential. They could use less challenging and more secure test environments right now, if they so choose. They don’t.
The “solve the problem” part of the issue actually goes all the way back to 2003, when a philosopher named Nick Bostrom famously made “the paperclip” argument. He stated that if you gave AI (actually all-powerful AGI) the mission to simply “make as many paperclips as possible” it would turn every person/place/thing into paperclips: the original AI Apocalypse. The challenge to this (and all) AI apocalypse scenarios is they neglect to explain one point: how in practice, rather than how in theory? In theory, giving an all-powerful AI the ability to do something, and giving it an admittedly stupid command, could result in disaster. But no one is giving AI control of anything that important. And even if we did, it would lack the control over resources to continue to misbehave, because no authority on earth has that access. Edward Teller hypothesized a nuclear test could ignite a chain reaction in the hydrogen in the earth’s atmosphere, destroying all life on earth. In theory, he was correct. In reality he was wrong. As are the current AI doomers.
Yes but why are some of the leaders of these AI companies crying out for government intervention, and why are some workers quitting with moral objections? On the latter, eschatological (the study of the end times) claims have become a standard feature of modern digital life. When all arguments come down to how many likes you can get, people have a natural tendency to exaggerate to get more attention. Off the top of my head, I can remember the following TEOTWAYKI (the end of the world as you know it) scenarios: accidental global thermonuclear war, “Silent Spring,” the Global Ice Age, the Population Bomb, Y2K, Global Warming/Climate Change, human-engineered viruses, angry space aliens, antibiotic-resistant bacteria, de-population, AI, and Trump. Always have to include Trump. What all these have in common is they are problems with solutions but they could run amok. Not necessarily that they will, or even if they’re at all likely to do so. So no, I don’t trust the plaintive cries of a twenty-something year old computer geek, sorry.
Now as to the “great men” running those companies, some suggest we have to listen to them, as they know the most about the subject. But here’s the problem with their complaints: put yourself in their shoes. You’re running a company whose product has a 10% chance of ending the world. What would you do? Write a blog post? Call on the US Congress for guidance? Give interviews calling for help? Would you not announce your own efforts to address the problem, and trumpet them for all to see? Why don’t they do this? Because it’s a race. A race to the end of the world? Really?
What is really going on here? These companies are ahead in the race, and they have several motivations. Yes, fear might be one, but it’s a minor one, or else they would address it themselves, competing against each other or even agreeing for the general safety for all. That’s what happened even between violently antagonistic governments with the development of nuclear weapons! The leading companies want to lock in their position by creating a set for rules which advantages them and discourages new competitors. But wouldn’t they fear that some other country (say, China?) might overtake them. China is in fact the only possible competitor, but they have committed themselves–as a collective under the Chinese Communist Party (CCP)—to simply steal the latest US models and stay one-step behind. The CCP is quickly learning that an all-powerful AI might be an independent source of authority, and that is the one thing the CCP can’t accept.
Also, the great AI companies fear the potential for crippling litigation against them based on their model’s actions. They don’t fear TEOTWAYKI as much as they fear ending up like Texaco, Arthur Andersen, GAWKER, and others that were sued out of existence. There are already a plethora of wrongful death, copyright infringement, and product liability suits working through the courts, and more will follow. What these AI giants are looking for is legal cover: give us some regulation, but also immunity from liability. Like Section 230 of the 1996 Communications Decency Act which immunized internet service providers and social media from responsibility for the content they provide, these AI companies want immunity now, before the litigation gets out of hand. And in my opinion, they don’t fear government regulation will affect them that much, because the government in general has no idea what to do. Yes, the same people who thought the internet was like “a series of tubes” are going to save us from AI. Congressional staff would turn to (wait for it) the experts at the AI companies for ideas, which is why those same companies don’t fear regulation.
There’s a political angle to this, too. The tech titans have been solidly branded with the MAGA brand, even though few if any of them are really behind Trump’s movement. It was a tactical decision to support him, as he seemed inclined to let them loose, and he did. None of them wants to end up like Elon Musk, who for all his riches and fame has now become a touchstone for hate: if he does it, burn it (literally). The MAGA party dutifully follows President Trump’s lead, as when he claimed on social media that the only “guardrail” the industry needs is a “STRONG AND SMART (High IQ!) PRESIDENT.” Sigh. The GOP is pro-data centers, pro winning the AI race, anti regulation. The Democrats have shrewdly adopted the opposite positions, and while this won’t be the defining issue in the mid-terms, the tech bros can see which way the wind is blowing. Aligning with the Democratic party when they look to take the legislative branch is an opportunity to get what they really want (immunity) and avoid being tarred with a continuing MAGA association. It’s nothing personal, just politics.
So there’s nothing to worry about, right? I did not say that, and I don’t believe that. AI, even AGI, is a tool, and like all tools it can be used or abused. AI models can be used for all kinds of nefarious ends, just not civilizational destruction. Your bank account could get erased, water or power systems might get turned off, somebody could hack the bridge on a supertanker and crash it into the port of Galveston (possibly just tried). But to engineer a supervirus and kill all of us, you need access to many things at many different places, and AI doesn’t run all of them (or even any of them right now). And you can’t access the nuclear codes, so “fuhgetaboutit.” Even the WarGames (movie) scenario of AI spoofing an attack to cause a massive nuclear response doesn’t work, because military warning systems have multiple systems for warning, not just one. And banks have back-up systems. And infrastructure can be sequestered from the internet. Much like the Y2K hysteria, there are many things we can be doing now, and we should be doing. And the AI companies should be leading that effort, not angling for immunity.
In the epic sci-fi flick 2001: A Space Odyssey, the AI system nicknamed HAL goes homicidal, killing astronauts as they sleep and launching one into outer space. It’s the ultimate AI Apocalypse scenario, where the computer holds all the cards (so to speak) and the human astronauts are on the edge of extinction. Remember how it ends? HAL won’t let Dave the astronaut back in the spaceship, and he has to navigate a few seconds in space without a helmet (technically possible!) to blow open a hatch and get back in. Then he yanks out HAL’s memory and control modules, slowly reverting the all-powerful AI to a childlike-level limited to babbling the song “Daisy Bell” while it expresses fears about its impending demise. Same as it ever was.