What to Think About Machines That Think

Today's Leading Thinkers on the Age of Machine Intelligence

John Brockman

• Publisher: Harper Perennial
• Year: 2015
• ISBN:
  9780062425669
• Topic: Psychology
• Subtopic: Cognitive Science
• Index: Psychology

Weighing in from the cutting-edge frontiers of science, today's most forward-thinking minds explore the rise of "machines that think."

Stephen Hawking recently made headlines by noting, "The development of full artificial intelligence could spell the end of the human race." Others, conversely, have trumpeted a new age of "superintelligence" in which smart devices will exponentially extend human capacities. No longer just a matter of science-fiction fantasy ( 2001, Blade Runner, The Terminator, Her, etc.), it is time to seriously consider the reality of intelligent technology, many forms of which are already being integrated into our daily lives. In that spirit, John Brockman, publisher of Edge. org ("the world's smartest website" – The Guardian ), asked the world's most influential scientists, philosophers, and artists one of today's most consequential questions: What do you think about machines that think?

SELF-AWARE AI? NOT IN 1,000 YEARS!

ROLF DOBELLI

Founder, Zurich Minds; journalist; author, The Art of Thinking Clearly

The widespread fear that AI will endanger humanity and take over the world is irrational. Here’s why.

Conceptually, autonomous or artificial intelligence systems can develop in two ways: either as an extension of human thinking or as radically new thinking. Call the first “Humanoid Thinking,” or Humanoid AI, and the second “Alien Thinking,” or Alien AI.

Almost all AI today is Humanoid Thinking. We use AI to solve problems too difficult, time-consuming, or boring for our limited brains to process: electrical-grid balancing, recommendation engines, self-driving cars, face recognition, trading algorithms, and the like. These artificial agents work in narrow domains with clear goals their human creators specify. Such AI aims to accomplish human objectives—often better, with fewer cognitive errors, distractions, outbursts of bad temper, or processing limitations. In a couple of decades, AI agents might serve as virtual insurance sellers, doctors, psychotherapists, and maybe even virtual spouses and children.

But such AI agents will be our slaves, with no self-concept of their own. They’ll happily perform the functions we set them up to do. If screwups happen, they’ll be our screwups, due to software bugs or overreliance on these agents (Dan Dennett’s point). Yes, Humanoid AIs might surprise us once in a while with novel solutions to specific optimization problems. But in most cases novel solutions are the last thing we want from AI (creativity in nuclear-missile navigation, anyone?). That said, Humanoid AI solutions will always fit a narrow domain. They’ll be understandable, either because we understand what they achieve or because we understand their inner workings. Sometimes the code will become too enormous and fumbled for one person to understand, because it’s continually patched. In these cases, we can turn it off and program a more elegant version. Humanoid AI will bring us closer to the age-old aspiration of having robots do most of the work while humans are free to be creative—or amused to death.

Alien Thinking is radically different. Alien Thinking could conceivably become a danger to Humanoid Thinking; it could take over the planet, outsmart us, outrun us, enslave us—and we might not even recognize the onslaught. What sort of thinking will Alien Thinking be? By definition, we can’t tell. It will encompass functionality we cannot remotely understand. Will it be conscious? Most likely, but it needn’t be. Will it experience emotion? Will it write bestselling novels? If so, bestselling to us or bestselling to it and its spawn? Will cognitive errors mar its thinking? Will it be social? Will it have a Theory of Mind? If so, will it make jokes, will it gossip, will it worry about its reputation, will it rally around a flag? Will it create its own version of AI (AI-AI)? We can’t say.

All we can say is that humans cannot construct truly Alien Thinking. Whatever we create will reflect our goals and values, so it won’t stray far from human thinking. You’d need real evolution, not just evolutionary algorithms, for self-aware Alien Thinking to arise. You’d need an evolutionary path radically different from the one that led to human intelligence and Humanoid AI.

So, how do you get real evolution to kick in? Replicators, variation, and selection. Once these three components are in place, evolution arises inevitably. How likely is it that Alien Thinking will evolve? Here’s a back-of-the-envelope calculation:

First, consider what getting from magnificently complex eukaryotic cells to human-level thinking involved. Achieving human thought required a large part of the Earth’s biomass (roughly 500 billion tons of eukaryotically bound carbon) during approximately 2 billion years. That’s a lot of evolutionary work! True, human-level thinking might have happened in half the time. With a lot of luck, even in 10 percent of the time, but it’s unlikely to have happened any faster. You don’t only need massive amounts of time for evolution to generate complex behavior, you also need a petri dish the size of Earth’s surface to sustain this level of experimentation.

Assume that Alien Thinking will be silicon-based, as all current AI is. A eukaryotic cell is vastly more complex than, say, Intel’s latest i7 CPU chip—both in hardware and software. Further assume that you could shrink that CPU chip to the size of a eukaryote. Leave aside the quantum effects that would stop the transistors from working reliably. Leave aside the question of the energy source. You’d have to cover the globe with 10³⁰ microscopic CPUs and let them communicate and fight for 2 billion years for true thought to emerge.

Yes, processing speed is faster in CPUs than in biological cells, because electrons are easier to shuttle around than atoms. But eukaryotes work massively parallel, whereas Intel’s i7 works only four times parallel (four cores). Eventually, at least to dominate the world, these electrons would need to move atoms to store their software and data in more and more physical places. This would slow their evolution dramatically. It’s hard to say if, overall, silicon evolution will be faster than biological. We don’t know enough about it. I don’t see why this sort of evolution would be more than two or three orders of magnitude faster than biological evolution (if at all)—which would bring the emergence of self-aware Alien AI down to roughly a million years.

What if Humanoid AI becomes so smart it could create Alien AI from the top down? That’s where Leslie Orgel’s Second Rule kicks in: “Evolution is smarter than you are.” It’s smarter than human thinking. It’s even smarter than Humanoid Thinking. And it’s much slower than you think.

Thus, the danger of AI is not inherent to AI but rests on our overreliance on it. Artificial thinking won’t evolve to self-awareness in our lifetime. In fact, it won’t happen in 1,000 years.

I might be wrong, of course. After all, this back-of-the-envelope calculation applies legacy human thinking to Alien AI—which by definition we won’t understand. But that’s all we can do at this stage.

Toward the end of the 1930s, Samuel Beckett wrote in a diary, “We feel with terrible resignation that reason is not a superhuman gift . . . that reason evolved into what it is, but that it also, however, could have evolved differently.” Replace “reason” with “AI” and you have my argument.

MACHINES DON’T THINK, BUT NEITHER DO PEOPLE

CESAR HIDALGO

Associate professor, MIT Media Lab; author, Why Information Grows: The Evolution of Order, from Atoms to Economies

Machines that think? That’s as fallacious as people who think! Thinking involves processing information, begetting new physical order from incoming streams of physical order. Thinking is a precious ability, which unfortunately is not the privilege of single units such as machines or people but a property of the systems in which these units come to “life.”

Of course I’m being provocative here, since at the individual level we do process information. We do think—sometimes—or at least we feel like we do. But “our” ability to think is not entirely “ours”—it’s borrowed, since the hardware and software we use to think weren’t begot by us. You and I did not evolve the genes that helped organize our brains or the language we use to structure our thoughts. Our ability to think is dependent on events that happened prior to our mundane existence: the past chapters of biological and cultural evolution. So we can only understand our ability to think, and the ability of machines to mimic thought, by considering how the ability of a unit to process information relates to its context.

Think of a human born in the dark solitude of empty space. She’d have nothing to think about. The same would be true of an isolated and inputless computing machine. In this context, we can call our borrowed ability to process information “little” thinking—since it’s a context-dependent ability that happens at the individual level. “Large” thinking, by contrast, is the ability to process information embodied in systems, where units like machines or us are mere pawns.

Separating the little thinking of humans from the larger thinking of systems (which involves the process that begets the hardware and software that allow units to “little think”) helps us understand the role of thinking machines in this larger context. Our ability to think isn’t only borrowed; it also hinges on the use and abuse of mediated interactions. For human/machine systems to think, humans need to eat and regurgitate one another’s mental vomit, which sometimes takes the form of words. But since words vanish in the wind, our species’ enormous ability to think hinges on more sophisticated techniques to communicate and preserve the information we generate: our ability to encode information in matter.

For 100,000 years, our species has been busy transforming our planet into a giant tape player. The planet Earth is the medium wherein we print our ideas: sometimes in symbolic form, such as text and paintings, but, more important, in objects—like hair dryers, vacuum cleaners, buildings, and cars—built from the mineral loins of planet Earth. Our society has a great collective ability to process information because our communication involves more than words: It involves the creation of objects, which transmit not something as flimsy as an idea but something as concrete as know-how and the uses of knowledge. Objects augment us; they allow us to do things without knowing how. We all get to enjoy the teeth-preserving powers of toothpaste without knowing how to synthesize sodium fluoride, or the benefits of long-distance travel without knowing how to build a plane. By the same token, we all enjoy the benefits of sending texts throughout the world in seconds through social media or of performing complex mathematical operations by pressing a few keys on a laptop computer.

But our ability to create the trinkets augmenting us has also evolved, of course, as a result of our collective willingness to eat one another’s mental vomit. This evolution is the one that brings us now to the point where we have “media” that are beginning to rival our ability to process information, or “little think.”

For most of our history, our trinkets were static objects. Even our tools were solidified chunks of order, such as stone axes, knives, and knitting needles. A few centuries ago, we developed the ability to outsource muscle and motion to machines, causing one of the greatest economic expansions in history. Now we’ve evolved our collective ability to process information by creating objects endowed with the ability to beget and recombine physical order. These are machines that can process information—engines that produce numbers, like the engines Charles Babbage dreamed about.

So we’ve evolved our ability to think collectively by first gaining dominion over matter, then over energy, and now over physical order, or information. Yet this shouldn’t fool us into believing that we think or that machines do. The large evolution of human thought requires mediated interactions, and the future of thinking machines will also happen at the interface where humans connect with humans through objects.

As we speak, nerds in the best universities of the world are mapping out the brain, building robotic limbs, and developing primitive versions of technologies that will open up the future when your great-grandchild will get high by plugging his brain directly into the Web. The augmentation these kids will get is unimaginable to us—and so bizarre by our modern ethical standards that we’re not even in a position to properly judge it; it would be like a sixteenth-century Puritan judging present-day San Francisco. Yet in the grand scheme of the universe, these new human/machine networks will be nothing other than the next natural step in the evolution of our species’ ability to beget information. Together, humans and our extensions—machines—will continue to evolve networks that are enslaved to the universe’s main glorious purpose: the creation of pockets where information does not dwindle but grows.

TANGLED UP IN THE QUESTION

JAMES J. O’DONNELL

Classical scholar; University Professor, Georgetown University; author, Augustine, The Ruin of the Roman Empire, Pagans

Thinking is a word we apply with no discipline whatsoever to a huge variety of reported behaviors. “I think I’ll go to the store,” and “I think it’s raining,” and “I think, therefore I am,” and “I think the Yankees will win the World Series,” and “I think I’m Napoleon,” and “I think he said he would be here, but I’m not sure”—all use the same word to mean entirely different things. Which of them might a machine do someday? I think that’s an important question.

Could a machine get confused? Experience cognitive dissonance? Dream? Wonder? Forget the name of that guy over there and at the same time know that it really knows the answer and if it just thinks about something else for a while, it might remember? Lose track of time? Decide to get a puppy? Have low self-esteem? Have suicidal thoughts? Get bored? Worry? Pray? I think not.

Can artificial mechanisms be constructed to play the part in gathering information and making decisions that human beings now play? Sure, they already do. The ones controlling the fuel injection in my car are a lot smarter than I am. I think I’d do a lousy job of that.

Could we create machines that go further and act without human supervision in ways that prove good or bad for human beings? I guess so. I think I’ll love them, except when they do things that make me mad—then they’re really being like people. I suppose they could run amok and create mass havoc, but I have my doubts. (Of course, if they do, nobody will care what I think.)

But nobody would ever ask a machine what it thinks about machines that think. That’s a question that makes sense only if we care about the thinker as an autonomous and interesting being, like ourselves. If somebody ever does ask a machine this question, it won’t be a machine anymore. I think I’m not going to worry about that for a while. You may think I’m in denial.

When we get tangled up in this question, we need to ask ourselves just what it is we’re really thinking about.

MISTAKING PERFORMANCE FOR COMPETENCE

RODNEY A. BROOKS

Panasonic Professor of Robotics, emeritus, MIT; founder, chair, and CTO, Rethink Robotics; author, Flesh and Machines

Think and intelligence are both what Marvin Minsky has called suitcase words—words into which we pack many meanings so we can talk about complex issues in shorthand. When we look inside these words, we find many different aspects, mechanisms, and levels of understanding. This makes answering the perennial questions of “Can machines think?” or “When will machines reach human-level intelligence?” difficult. The suitcase words are used to cover both specific performance demonstrations by machines and the more general competence that humans might have. We generalize from performance to competence and grossly overestimate the capabilities of machin...

Table of contents

• Dedication
• Contents
• Acknowledgments
• Preface: The 2015 Edge Question
• Murray Shanahan: Consciousness in Human-Level AI
• Steven Pinker: Thinking Does Not Imply Subjugating
• Martin Rees: Organic Intelligence Has No Long-Term Future
• Steve Omohundro: A Turning Point in Artificial Intelligence
• Dimitar D. Sasselov: AI Is I
• Frank Tipler: If You Can’t Beat ’em, Join ’em
• Mario Livio: Intelligent Machines on Earth and Beyond
• Antony Garrett Lisi: I, for One, Welcome Our Machine Overlords
• John Markoff: Our Masters, Slaves, or Partners?
• Paul Davies: Designed Intelligence
• Kevin P. Hand: The Superintelligent Loner
• John C. Mather: It’s Going to Be a Wild Ride
• David Christian: Is Anyone in Charge of This Thing?
• Timo Hannay: Witness to the Universe
• Max Tegmark: Let’s Get Prepared!
• Tomaso Poggio: “Turing+” Questions
• Pamela Mccorduck: An Epochal Human Event
• Marcelo Gleiser: Welcome to Your Transhuman Self
• Sean Carroll: We Are All Machines That Think
• Nicholas G. Carr: The Control Crisis
• Jon Kleinberg & Sendhil Mullainathan: We Built Them, but We Don’t Understand Them
• Jaan Tallinn: We Need to Do Our Homework
• George Church: What Do You Care What Other Machines Think?
• Arnold Trehub: Machines Cannot Think
• Roy Baumeister: No “I” and No Capacity for Malice
• Keith Devlin: Leveraging Human Intelligence
• Emanuel Derman: A Machine Is a “Matter” Thing
• Freeman Dyson: I Could Be Wrong
• David Gelernter: Why Can’t “Being” or “Happiness” Be Computed?
• Leo M. Chalupa: No Machine Thinks About the Eternal Questions
• Daniel C. Dennett: The Singularity—an Urban Legend?
• W. Tecumseh Fitch: Nano-Intentionality
• Irene Pepperberg: A Beautiful (Visionary) Mind
• Nicholas Humphrey: The Colossus Is a BFG
• Rolf Dobelli: Self-Aware AI? Not in 1,000 Years!
• Cesar Hidalgo: Machines Don’t Think, but Neither Do People
• James J. O’Donnell: Tangled Up in the Question
• Rodney A. Brooks: Mistaking Performance for Competence
• Terrence J. Sejnowski: AI Will Make You Smarter
• Seth Lloyd: Shallow Learning
• Carlo Rovelli: Natural Creatures of a Natural World
• Frank Wilczek: Three Observations on Artificial Intelligence
• John Naughton: When I Say “Bruno Latour,” I Don’t Mean “Banana Till”
• Nick Bostrom: It’s Still Early Days
• Donald D. Hoffman: Evolving AI
• Roger Schank: Machines That Think Are in the Movies
• Juan Enriquez: Head Transplants?
• Esther Dyson: AI/AL
• Tom Griffiths: Brains and Other Thinking Machines
• Mark Pagel: They’ll Do More Good Than Harm
• Robert Provine: Keeping Them on a Leash
• Susan Blackmore: The Next Replicator
• Tim O’Reilly: What If We’re the Microbiome of the Silicon AI?
• Andy Clark: You Are What You Eat
• Moshe Hoffman: AI’s System of Rights and Government
• Brian Knutson: The Robot with a Hidden Agenda
• William Poundstone: Can Submarines Swim?
• Gregory Benford: Fear Not the AI
• Lawrence M. Krauss: What, Me Worry?
• Peter Norvig: Design Machines to Deal with the World’s Complexity
• Jonathan Gottschall: The Rise of Storytelling Machines
• Michael Shermer: Think Protopia, Not Utopia or Dystopia
• Chris Dibona: The Limits of Biological Intelligence
• Joscha Bach: Every Society Gets the AI It Deserves
• Quentin Hardy: The Beasts of AI Island
• Clifford Pickover: We Will Become One
• Ernst Pöppel: An Extraterrestrial Observation on Human Hubris
• Ross Anderson: He Who Pays the AI Calls the Tune
• W. Daniel Hillis: I Think, Therefore AI
• Paul Saffo: What Will the Place of Humans Be?
• Dylan Evans: The Great AI Swindle
• Anthony Aguirre: The Odds on AI
• Eric J. Topol: A New Wisdom of the Body
• Roger Highfield: From Regular-I to AI
• Gordon Kane: We Need More Than Thought
• Scott Atran: Are We Going in the Wrong Direction?
• Stanislas Dehaene: Two Cognitive Functions Machines Still Lack
• Matt Ridley: Among the Machines, Not Within the Machines
• Stephen M. Kosslyn: Another Kind of Diversity
• Luca De Biase: Narratives and Our Civilization
• Margaret Levi: Human Responsibility
• D. A. Wallach: Amplifiers/Implementers of Human Choices
• Rory Sutherland: Make the Thing Impossible to Hate
• Bruce Sterling: Actress Machines
• Kevin Kelly: Call Them Artificial Aliens
• Martin Seligman: Do Machines Do?
• Timothy Taylor: Denkraumverlust
• George Dyson: Analog, the Revolution That Dares Not Speak Its Name
• S. Abbas Raza: The Values of Artificial Intelligence
• Bruce Parker: Artificial Selection and Our Grandchildren
• Neil Gershenfeld: Really Good Hacks
• Daniel L. Everett: The Airbus and the Eagle
• Douglas Coupland: Humanness
• Josh Bongard: Manipulators and Manipulanda
• Ziyad Marar: Are We Thinking More Like Machines?
• Brian Eno: Just a New Fractal Detail in the Big Picture
• Marti Hearst: eGaia, a Distributed Technical-Social Mental System
• Chris Anderson: The Hive Mind
• Alex (Sandy) Pentland: The Global Artificial Intelligence Is Here
• Randolph Nesse: Will Computers Become Like Thinking, Talking Dogs?
• Richard E. Nisbett: Thinking Machines and Ennui
• Samuel Arbesman: Naches from Our Machines
• Gerald Smallberg: No Shared Theory of Mind
• Eldar Shafir: Blind to the Core of Human Experience
• Christopher Chabris: An Intuitive Theory of Machine
• Ursula Martin: Thinking Saltmarshes
• Kurt Gray: Killer Thinking Machines Keep Our Conscience Clean
• Bruce Schneier: When Thinking Machines Break the Law
• Rebecca Mackinnon: Electric Brains
• Gerd Gigerenzer: Robodoctors
• Alison Gopnik: Can Machines Ever Be As Smart As Three-Year-Olds?
• Kevin Slavin: Tic-Tac-Toe Chicken
• Alun Anderson: AI Will Make Us Smart and Robots Afraid
• Mary Catherine Bateson: When Thinking Machines Are Not a Boon
• Steve Fuller: Justice for Machines in an Organicist World
• Tania Lombrozo: Don’t Be a Chauvinist About Thinking
• Virginia Heffernan: This Sounds Like Heaven
• Barbara Strauch: Machines That Work Until They Don’t
• Sheizaf Rafaeli: The Moving Goalposts
• Edward Slingerland: Directionless Intelligence
• Nicholas A. Christakis: Human Culture As the First AI
• Joichi Ito: Beyond the Uncanny Valley
• Douglas Rushkoff: The Figure or the Ground?
• Helen Fisher: Fast, Accurate, and Stupid
• Stuart Russell: Will They Make Us Better People?
• Eliezer S. Yudkowsky: The Value-Loading Problem
• Kate Jeffery: In Our Image
• Maria Popova: The Umwelt of the Unanswerable
• Jessica L. Tracy & Kristin Laurin: Will They Think About Themselves?
• June Gruber & Raul Saucedo: Organic Versus Artifactual Thinking
• Paul Dolan: Context Surely Matters
• Thomas G. Dietterich: How to Prevent an Intelligence Explosion
• Matthew D. Lieberman: Thinking from the Inside or the Outside?
• Michael Vassar: Soft Authoritarianism
• Gregory Paul: What Will AIs Think About Us?
• Andrian Kreye: A John Henry Moment
• N. J. Enfield: Machines Aren’t into Relationships
• Nina Jablonski: The Next Phase of Human Evolution
• Gary Klein: Domination Versus Domestication
• Gary Marcus: Machines Won’t Be Thinking Anytime Soon
• Sam Harris: Can We Avoid a Digital Apocalypse?
• Molly Crockett: Could Thinking Machines Bridge the Empathy Gap?
• Abigail Marsh: Caring Machines
• Alexander Wissner-Gross: Engines of Freedom
• Sarah Demers: Any Questions?
• Bart Kosko: Thinking Machines = Old Algorithms on Faster Computers
• Julia Clarke: The Disadvantages of Metaphor
• Michael Mccullough: A Universal Basis for Human Dignity
• Haim Harari: Thinking About People Who Think Like Machines
• Hans Halvorson: Metathinking
• Christine Finn: The Value of Anticipation
• Dirk Helbing: An Ecosystem of Ideas
• John Tooby: The Iron Law of Intelligence
• Maximilian Schich: Thought-Stealing Machines
• Satyajit Das: Unintended Consequences
• Robert Sapolsky: It Depends
• Athena Vouloumanos: Will Machines Do Our Thinking for Us?
• Brian Christian: Sorry to Bother You
• Benjamin K. Bergen: Moral Machines
• Laurence C. Smith: After the Plug Is Pulled
• Giulio Boccaletti: Monitoring and Managing the Planet
• Ian Bogost: Panexperientialism
• Aubrey De Grey: When Is a Minion Not a Minion?
• Michael I. Norton: Not Buggy Enough
• Thomas A. Bass: More Funk, More Soul, More Poetry and Art
• Hans Ulrich Obrist: The Future Is Blocked to Us
• Koo Jeong-A: An Immaterial Thinkable Machine
• Richard Foreman: Baffled and Obsessed
• Richard H. Thaler: Who’s Afraid of Artificial Intelligence?
• Scott Draves: I See a Symbiosis Developing
• Matthew Ritchie: Reimagining the Self in a Distributed World
• Raphael Bousso: It’s Easy to Predict the Future
• James Croak: Fear of a God, Redux
• Andrés Roemer: Tulips on My Robot’s Tomb
• Lee Smolin: Toward a Naturalistic Account of Mind
• Stuart A. Kauffman: Machines That Think? Nuts!
• Melanie Swan: The Future Possibility-Space of Intelligence
• Tor Nørretranders: Love
• Kai Krause: An Uncanny Three-Ring Test for Machina sapiens
• Georg Diez: Free from Us
• Eduardo Salcedo-Albarán: Flawless AI Seems Like Science Fiction
• Maria Spiropulu: Emergent Hybrid Human/Machine Chimeras
• Thomas Metzinger: What If They Need to Suffer?
• Beatrice Golomb: Will We Recognize It When It Happens?
• Noga Arikha: Metarepresentation
• Demis Hassabis, Shane Legg & Mustafa Suleyman: Envoi: A Short Distance Ahead—and Plenty to Be Done
• Notes
• About the Author
• Also by John Brockman
• Credits
• Back Ads
• Copyright
• About the Publisher