In my midwestern town, you can't escape the cicadas. They fill the trees and synchronize their calls in a pulsing drone, and I'd find it deafening if I hadn't grown up with it. To me, it's the sound of summer.
In the mornings when we leave for school, my sons typically find a few lying motionless on our front porch, and they are, as you'd expect, both disgusted and fascinated; they crouch and stare, hands on knees, inching closer to see if the insects might move. Sometimes they do, suddenly and with indignant clatter, and the boys scatter in gleeful terror. Usually they don't. On the walk, we trade theories for why these strange bugs seem to seek out our stoop to die.
One night, long after the kids were asleep, I learned the answer. I heard small thumps against our front door, spaced a few seconds apart. Peering out the window, I saw that it was cicadas, flying at full speed into our door and front wall. Often they would ricochet and continue flying, but sometimes they'd fall to the ground, dead or at least dazed. These were the unlucky ones we found each morning.
Cicadas orient by light. They stay level by keeping their backs towards the light and they fly toward silhouettes to find landing sites. Our porch lights and white walls—multiple light sources close to the ground; a broad, illuminated surface—disorient them, and because they are heavy and awkward flyers, they cannot correct course. They crash.
I turned off the lights, and I thought about how hundreds of millions of years of evolution produced this flight system, one that served well enough to carry cicadas across the globe. A century and a half of electric light has not been nearly enough time to adjust, and so they follow their instincts into the wall.
Our public debate about Artificial Intelligence is collapsing, like all of our public debates, into a binary. The two emerging camps are, roughly:
- AI will be good, so we should go. This one has the money behind it.
- We don't know that AI is going to be good, so we should stop or at least slow down. This one is gaining public support.
(These are essentially the positions people call "zoomer" and "doomer," but I prefer to state them rather than use the labels.)
Both are logically consistent, but I don't believe either view is defensible. The first fails because we cannot know. The second fails because we cannot stop. That's a disquieting position, but I believe it. My reasons:
- The path AI will take in our world is not predictable. That doesn't simply mean we don't know what the outcome will be. It means we cannot even assign probabilities to possible outcomes.
- The compulsion to build AI is too strong to stop. For discovery, for profit, or for power, we will build it.
The path AI will take is not predictable
The idea that AI draws a curtain in front of the future has been around at least since 1993, when Vernor Vinge argued that it would create a technological singularity. When machine intelligence becomes capable enough to improve itself, we will no longer be able to model its progress and therefore cannot predict what will happen. That hasn't stopped people—notably, the leaders of AI companies—from making all kinds of predictions, even assigning specific probabilities to them.
I believe Vinge was essentially correct, but I'll add a few arguments to the case.
We are not built to understand AI
A few years ago, I noticed that a few of the books I'd read recently had each talked, in different ways, about the difficulty the human mind has imagining, describing, or explaining certain concepts. We struggle to visualize beyond three dimensions, or think probabilistically or exponentially or in very large numbers, or come to grips with consciousness. These aren't necessarily hard limits—there are certainly people who can reason intuitively with probabilities, and maybe some who can visualize four dimensions (I certainly can't). But they are beyond many people and can strain even those who think deeply about them.
Strikingly, AI implicates so many of these. Probabilistic math occurring in multi-dimensional spaces at enormous scale, exponential improvement curves, emergent consciousness. Steam power may have reshaped the world, but there's nothing about heating water so steam can move a turbine that tests the limits of human cognition. AI does that in multiple ways.
For most people, this places AI outside of comfortable comprehension, but even experts, with decades of training pushing against these barriers, do not understand their creation the way a mechanic understands a motorcycle. Even as they succeed in making rapid progress, there is still trial and error and surprise; as with most sciences, practice precedes theory, but in this case the human mind is swimming against the current.
==Perhaps it's poetic justice that incomprehension is the price of creating intelligence.==
AI misleads our intuition, by design
In contrast to its interior operation, the expression of AI often looks familiar and human, which is to be expected from a technology expressly designed to emulate and eventually replace human intelligence. The obvious example is chatbots that speak in natural language, but other kinds of AI, from image generators to task-completion agents, can produce output that looks very human.
There is a gap between machine and human, and the interface is where they meet. Early computers required humans to cross the gap and come to them: to work with a computer could mean painstakingly organizing punched cards that encoded a program, thinking in binary code, and interpreting indicator lights to know what was happening. Humans had to interact in the way that worked for the machine.
The personal computers we've used for the last several decades, from the first personal computers with graphical user interfaces to the smartphone, have met us about halfway. We still need to operate through an interface and build a mental model of how the machine works (what is an application and what can it do? how is data stored? what is the configuration of the settings menu?), but it's much friendlier now than the text terminals early generations had. That ease is why computing has been able to move from a dedicated room at the company headquarters to the pockets of billions of people.
AI promises to complete the journey, pushing the interface all the way to the human end of the gap; the machine will come to us. We speak naturally and it understands. We request and it delivers. It can see and speak and touch, and we won't need to understand anything about its internal organization.
The convenience of that shift is undeniable, but there are costs. The first is the temptation of anthropomorphism. At its mildest, we talk about AI thinking or wanting things because we don't have better language. At its most severe, we confuse AI for consciousness. In all cases, our understanding of what the machine is doing, what it can do, and what it will do in a new situation, is wrong. Whether or not AI as we know it is intelligent, it is not and does not behave like human intelligence.
The second is the tendency to treat machines as magic. To operate an early computer, you had to have some understanding of how the machine worked, or you couldn't navigate it. As interfaces have become more user-friendly, they've implicitly told us that we don't need to know how any of it works. AI promises to sweep all that complexity under the rug.
Again, experts and the general public will differ, though anecdotally, I find that experts are not immune to these risks.
The trajectory of AI moves through human social complexity
Predictions about the future of AI often proceed as though technical development occurs in a vacuum. The human context is noted, if at all, as a factor to be assumed away for simplicity.
But AI's progress has always been deeply contingent on that context. That context is the market reactions to AI, which determines how the investment flows. It's the sentiment of civil society and regulators, which shapes the policy environment—first by promoting or proscribing some things, and then by shifting where they happen. It's even the office politics within AI companies, which determines which ideas get funding and resources and under what conditions. The pace and direction of AI do not unfold inevitably according to the internal logic of the technology. At least until the day AI takes over its own development, its development is contingent on human chaos.
That means that any prediction about the future of AI must also be a prediction of the human response to it. That's why the discussion above covers how well not only AI engineers but the public understand AI, because public understanding will shape the path. Predicting the direction of human society is prohibitively difficult, even in a narrow region, and even over just a few years. To predict what the world will do over generations, and to do it for the complex of AI and human society, can be an interesting exercise but won't yield useful results.
Together, these reasons argue for a future that is bewilderingly complex. Predicting the outcome of complex systems is difficult, if not practically impossible; when we do it—say, for the weather, or a social phenomenon—it's largely because we have prior examples we can draw on. AI does not offer that. The technologies we reach for as predecessors—railroads, electricity, nuclear weapons—all offer some interesting parallels but are obviously not close enough to be useful for prediction. This hasn't happened before.
We are left with a future so uncertain that beyond not knowing what will happen, we don't know how likely each of the possibilities is.
The compulsion to build AI is too strong to stop
Given the uncertainty and the stakes, the rational choice is plain: stop. Even if we can't assign a specific probability to apocalypse (or at least dystopia), if there is a chance it's greater than zero, then all the drug discovery and customized learning and economic efficiency and all the rest of the promise isn't worth it.
Yet I'm convinced that humanity will keep building AI for the foreseeable future. A few reasons, beginning with the most obvious.
The race is real
The typical reason given for building despite the danger is the race—between the companies, and more importantly, between the United States and China. The argument is that if the pace of progress in the United States slows, the most important effect will not be a long-term slowdown of AI but a shift in dominance from the United States to China. That means that, instead of Silicon Valley and Washington setting the global standard for AI, it will be set by the Chinese Communist Party.
This debate is covered in plenty of places, and I don't want to belabor it here. It's difficult to prove either way, because the difficulties in achieving multinational agreements are open to interpretation. For example, is nuclear non-proliferation a victory (because there haven't been nuclear attacks since Hiroshima and Nagasaki) or failure (because after so many decades, the world still lives with the threat of these horrific weapons)? Is AI like nuclear weapons, or is it different?
My view is that the belief that China might willingly slow down, pause, or stop AI development is saturated with rich-world sensibility. In the United States, we have enough collective satisfaction with the status quo that we can talk about the most significant technological advance in memory as though it's optional. A nation (or, more precisely, its governing party) that is not satisfied, that hungers for continued rapid development and influence on the global stage, and that has long sought more strategic latitude will not voluntarily sit out the greatest opportunity in generations to leap ahead economically and shift the global order.
AI's unpredictability may be a threat to the Party's control, and I believe some agreement on safety and standards is possible. But there is no other opportunity like this on the horizon, and I cannot believe they will miss it.
The danger is why we build
A still darker view of the race is that the danger is not a reason to stop, but a reason to build. I wrote about this in a shorter note, but in brief: when a technology is dangerous, and when that danger can be used by some of us against others, it is rational to fear not only the technology itself but that an adversary will have it. The more dangerous the technology, the more furious the race to build it—particularly because, as we've learned over the past year, AI can not only launch attacks but find and patch vulnerabilities ahead of them. A nation with weaker capabilities can neither defer nor defend. American public sentiment may be hardening against AI, but no American president or senator wants the legacy of leaving the nation open to an existential strike.
Building is what we do
The simplest explanation is that we will build because it's what we do. This is the argument most quickly dismissed—incorrectly, because it might be the most powerful.
It's easy to see the development of AI as simply a grab for money, or for power, or as the fulfillment of adolescent science-fiction fantasies (it is, in part, all of those). Building, inventing, and discovering for their own sake are not always taken seriously. But many who would dismiss the drive to invent would acknowledge and fight for the profound drive to create art. Exploration and expression are essential to being human, even if we don't need them to eat and breathe, because a life of material security without beauty is arid and meaningless.
The drive to invent and discover is just as intrinsic to humanity; in fact, it may be the same drive. It's been with us since antiquity—there are shades of it in Eve, in Icarus, in Babel—and it is alive today. A tinkerer spends her nights bent over a workbench in the garage, just as her neighbor spends his working on his novel. In both cases, the creative drive can become big business, but that doesn't mean it isn't also human expression.
AI is clearly among the most consequential areas of inquiry today, and to many (including me) easily the single most interesting. The models we have today are like the Wright Brothers' planes: proof that the thing is possible, but nowhere near the shape it will eventually take. There are people all over the world dreaming of the jet planes, so to speak, that they could build. I can imagine law that slows AI down. Now that we know what's possible, I cannot imagine a world where someone, somewhere, is not trying to build it.
Maybe it's contradictory to argue against prediction and then say we'll keep building AI. I don't see this as a prediction of the choices we'll make, but an argument that these choices are not really in the option set. Taken together, humanity does not operate rationally to maximize our wellbeing, as the persistence of war and climate inaction attest. We are motivated by envy and desire and ambition, along with a reckless impulse to discover and create. All told, building AI is not as difficult as not building AI.
Action without expectation
The confusion and paralysis we suffer now is because we are used to prediction preceding action. It's why regulators, used to weighing costs and benefits, deliberate and do not act (or clearly choose not to act); why advocacy is coalescing around bans and boycotts; and why any prediction from people in the industry—particularly if it includes an absurdly quantified probability—is breathlessly reported in the press.
This can't continue. This era demands that we sever action from prediction, choosing a path without knowing what is coming, and adapting as circumstances change. I want to see the United States:
- Win the race. It's happening, and losing is not a tenable outcome. We should be doing what it takes to ensure American AI leads and is the global standard.
- Move aggressively and creatively on governance. Safety, oversight, and values alignment are essential, but our approaches to regulation are holdovers from the last century, if not the one before. We need new approaches that preserve competitiveness while increasing safety. To the extent these are at odds, we have to innovate in governance where possible to break the choice, and resort to tradeoffs when necessary.
- Invest in preparation. The greatest failure of both leading camps is that they are distractions from the work we could be doing now to better prepare for an AI transition. Children need to understand AI (and, while we're at it, catch up on understanding information technology in general), so let's have resources and curriculum and teacher training and more. Some occupations, like many care professions, will be resilient to automation and would be perfect to expand, so let's have investment in skill development and placement. There is so much we can do once we decide to do it.
But I am arguing here less for a specific course of action than for a cast of mind, one defined not by optimism, not by pessimism, but by work. I'd like to move past the credulous reporting and calls for bans and get to work on the conditions—American leadership, strong governance, economic preparation—that should be the goal if we want to increase the chances of a good outcome.
We may not be able to see the future, and we may be hemmed in by our nature and by generations of political dysfunction. But we have the capacity to learn and to adapt and to strain against our constraints. We aren't gods, but we aren't insects either.