NEW YORK TIMES BESTSELLER • An urgent warning of the unprecedented risks that AI and other fast-developing technologies pose to global order, and how we might contain them while we have the chance—from a co-founder of the pioneering artificial intelligence company DeepMind“A fascinating, well-written, and important book.”—Yuval Noah Harari“Essential reading.”—Daniel Kahneman“An excellent guide for navigating unprecedented times.”—Bill GatesA Best Book of the Year: CNN, Economist, Bloomberg, Financial Times, The Guardian, CEO Magazine, Semafor • Winner of the Inc. Non-Obvious Book Award • Finalist for the Porchlight Business Book Award and the Financial Times and Schroders Business Book of the Year AwardWe are approaching a critical threshold in the history of our species. Everything is about to change. Soon you will live surrounded by AIs. They will organise your life, operate your business, and run core government services. You will live in a world of DNA printers and quantum computers, engineered pathogens and autonomous weapons, robot assistants and abundant energy. None of us are prepared. As co-founder of the pioneering AI company DeepMind, part of Google, Mustafa Suleyman has been at the centre of this revolution. The coming decade, he argues, will be defined by this wave of powerful, fast-proliferating new technologies. In The Coming Wave, Suleyman shows how these forces will create immense prosperity but also threaten the nation-state, the foundation of global order. As our fragile governments sleepwalk into disaster, we face an existential dilemma: unprecedented harms on one side, the threat of overbearing surveillance on the other. Can we forge a narrow path between catastrophe and dystopia? This groundbreaking book from the ultimate AI insider establishes “the containment problem”—the task of maintaining control over powerful technologies—as the essential challenge of our age.
High-quality synthetic media changes this equation. Not all nations currently have the funds to build huge disinformation programs, with dedicated offices and legions of trained staff, but thats less of a barrier when high-fidelity material can be generated at the click of a button. Much of the coming chaos will not be accidental. It will come as existing disinformation campaigns are turbocharged, expanded, and devolved out to a wide group of motivated actors.
id: 530d78e3a8410bba6f9f000b0db3942b - page: 191
The rise of synthetic media at scale and minimal cost amplifies both disinformation (malicious and intentionally misleading information) and misinformation (a wider and more unintentional pollution of the information space) at once. Cue an Infocalypse, the point at which society can no longer manage a torrent of sketchy material, where the information ecosystem grounding knowledge, trust, and social cohesion, the glue holding society together, falls apart. In the words of a Brookings Institution report, ubiquitous, perfect synthetic media means distorting democratic discourse; manipulating elections; eroding trust in institutions; weakening journalism; exacerbating social divisions; undermining public safety; and inflicting hard-to-repair damage on the reputation of prominent individuals, including elected officials and candidates for office.
id: 1c591824cf7b48acfd332e6df701b56a - page: 191
Not all stressors and harms come from bad actors, however. Some come from the best of intentions. Amplification of fragility is accidental as well as deliberate. LEAKY LABS AND UNINTENDED INSTABILITY In one of the worlds most secure laboratories, a group of researchers were experimenting with a deadly pathogen. No one can be sure what happened next. Even with the benefit of hindsight, detail about the research is scant. What is certain is that, in a country famed for secrecy and government control, a strange new illness began appearing. Soon it was found around the world, in the U.K., the United States, and beyond. Oddly, this didnt seem like an entirely natural strain of the disease. Certain features raised alarm in the scientific community and suggested that something at the lab had gone horribly wrong, that this wasnt a natural event. Soon the death toll started rising. That hyper-secure lab wasnt looking so secure after all.
id: 8cc5714ce61ea5eaf020e0c8d6ff6dad - page: 191
If this sounds like a familiar story, it probably isnt the one youre thinking about. This was 1977 and an influenza epidemic known as the Russian flu. First discovered in China, it was detected in the Soviet Union soon after, spreading from there and reportedly killing up to 700,000 people. What was unusual about the H1N1 flu strain was how closely it resembled one circulating in the 1950s. The disease hit young people hardest, a possible sign they had a weaker immunity than those around a few decades earlier. Theories abound over what happened. Had something escaped from the permafrost? Was it part of Russias extensive and shadowy bioweapons program? To date, though, the best explanation is a lab leak. A version of the earlier virus likely somehow escaped during lab experiments with a vaccine. The epidemic was itself caused by well-meaning research intended to prevent epidemics.
id: 0d500b63bb34928d5d5cbf183ef4d0b9 - page: 192