حرکت معکوس

Reversion

Wouldn’t it be tempting to escape the perils of technology without succumbing to stagnant totalitarianism? Let’s explore a scenario where this was accomplished by reverting to primitive technology, inspired by the Amish. After the Omegas took over the world as in the opening of the book, a massive global propaganda campaign was launched that romanticized the simple farming life of 1,500 years ago. Earth’s population was reduced to about 100 million people by an engineered pandemic blamed on terrorists. The pandemic was secretly targeted to ensure that nobody who knew anything about science or technology survived. With the excuse of eliminating the infection hazard of large concentrations of people, Prometheus-controlled robots emptied and razed all cities. Survivors were given large tracts of (suddenly available) land and educated in sustainable farming, fishing and hunting practices using only early medieval technology. In the meantime, armies of robots systematically removed all traces of modern technology (including cities, factories, power lines and paved roads), and thwarted all human attempts to document or re-create any such technology. Once the technology was globally forgotten, robots helped dismantle other robots until there were almost none left. The very last robots were deliberately vaporized together with Prometheus itself in a large thermonuclear explosion. There was no longer any need to ban modern technology, since it was all gone. As a result, humanity bought itself over a millennium of additional time without worries about either AI or totalitarianism.

Reversion has to a lesser extent happened before: for example, some of the technologies that were in widespread use during the Roman Empire were largely forgotten for about a millennium before making a comeback during the Renaissance. Isaac Asimov’s Foundation trilogy centers around the “Seldon Plan” to shorten a reversion period from 30,000 years to 1,000 years. With clever planning, it may be possible to do the opposite and lengthen rather than shorten a reversion period, for example by erasing all knowledge of agriculture. However, unfortunately for reversion enthusiasts, it’s unlikely that this scenario can be extended indefinitely without humanity either going high-tech or going extinct. Counting on people’s resembling today’s biological humans 100 million years from now would be naive, given that we haven’t existed as a species for more than 1% of that time so far. Moreover, low-tech humanity would be a defenseless sitting duck just waiting to be exterminated by the next planet-scorching asteroid impact or other mega-calamity brought on by Mother Nature. We certainly can’t last a billion years, after which the gradually warming Sun will have cranked up Earth’s temperature enough to boil off all liquid water.

Self-Destruction

After contemplating problems that future technology might cause, it’s important to also consider problems that lack of that technology can cause. In this spirit, let us explore scenarios where superintelligence is never created because humanity eliminates itself by other means.

How might we accomplish that? The simplest strategy is “just wait.” Although we’ll see in the next chapter how we can solve such problems as asteroid impacts and boiling oceans, these solutions all require technology that we haven’t yet developed, so unless our technology advances far beyond its present level, Mother Nature will drive us extinct long before another billion years have passed. As the famous economist John Maynard Keynes said: “In the long run we are all dead.” Unfortunately, there are also ways in which we might self-destruct much sooner, through collective stupidity. Why would our species commit collective suicide, also known as omnicide, if virtually nobody wants it? With our present level of intelligence and emotional maturity, we humans have a knack for miscalculations, misunderstandings and incompetence, and as a result, our history is full of accidents, wars and other calamities that, in hindsight, essentially nobody wanted. Economists and mathematicians have developed elegant game-theory explanations for how people can be incentivized to actions that ultimately cause a catastrophic outcome for everyone.

Nuclear War: A Case Study in Human Recklessness

You might think that the greater the stakes, the more careful we’d be, but a closer examination of the greatest risk that our current technology permits, namely a global thermonuclear war, isn’t reassuring. We’ve had to rely on luck to weather an embarrassingly long list of near misses caused by all sorts of things: computer malfunction, power failure, faulty intelligence, navigation error, bomber crash, satellite explosion and so on.7 In fact, if it weren’t for heroic acts of certain individuals—for example, Vasili Arkhipov and Stanislav Petrov—we might already have had a global nuclear war. Given our track record, I think it’s highly unlikely that the annual probability of accidental nuclear war is as low as one in a thousand if we keep up our present behavior, in which case the probability that we’ll have one within 10,000 years exceeds 1− 0.99910000 ≈ 99.995%.

To fully appreciate our human recklessness, we must realize that we started the nuclear gamble even before carefully studying the risks. First, radiation risks had been underestimated, and over $2 billion in compensation has been paid out to victims of radiation exposure from uranium handling and nuclear tests in the United States alone.8

Second, it was eventually discovered that hydrogen bombs deliberately detonated hundreds of kilometers above Earth would create a powerful electromagnetic pulse (EMP) that might disable the electric grid and electronic devices over vast areas (figure 5.2), leaving infrastructure paralyzed, roads clogged with disabled vehicles and conditions for nuclear-aftermath survival less than ideal. For example, the U.S. EMP Commission reported that “the water infrastructure is a vast machine, powered partly by gravity but mostly by electricity,” and that denial of water can cause death in three to four days.

Third, the potential of nuclear winter wasn’t realized until four decades in, after we’d deployed 63,000 hydrogen bombs—oops! Regardless of whose cities burned, massive amounts of smoke reaching the upper troposphere might spread around the globe, blocking out enough sunlight to transform summers into winters, much like when an asteroid or supervolcano caused a mass extinction in the past. When the alarm was sounded by both U.S. and Soviet scientists in the 1980s, this contributed to the decision of Ronald Reagan and Mikhail Gorbachev to start slashing stockpiles.10 Unfortunately, more accurate calculations have painted an even gloomier picture: figure 5.3 shows cooling by about 20° Celsius (36° Fahrenheit) in much of the core farming regions of the United States, Europe, Russia and China (and by 35°C in some parts of Russia) for the first two summers, and about half that even a full decade later.*4 What does that mean in plain English? One doesn’t need much farming experience to conclude that near-freezing summer temperatures for years would eliminate most of our food production. It’s hard to predict exactly what would happen after thousands of Earth’s largest cities are reduced to rubble and global infrastructure collapses, but whatever small fraction of all humans don’t succumb to starvation, hypothermia or disease would need to cope with roving armed gangs desperate for food.

I’ve gone into such detail on global nuclear war to drive home the crucial point that no reasonable world leader would want it, yet it might nonetheless happen by accident. This means that we can’t trust our fellow humans never to commit omnicide: nobody wanting it isn’t necessarily enough to prevent it.