Cybermedia

Explorations in Science, Sound, and Vision

Carol Vernallis, Holly Rogers, Jonathan Leal, Selmin Kara, Carol Vernallis, Holly Rogers, Jonathan Leal, Selmin Kara

• Publisher: Bloomsbury Academic
• Year: 2021
• ISBN:
  9781501357053
• Edition: 1
• Topic: Media & Performing Arts
• Subtopic: Music History & Criticism

We're experiencing a time when digital technologies and advances in artificial intelligence, robotics, and big data are redefining what it means to be human. How do these advancements affect contemporary media and music? This collection traces how media, with a focus on sound and image, engages with these new technologies. It bridges the gap between science and the humanities by pairing humanists' close readings of contemporary media with scientists' discussions of the science and math that inform them. This text includes contributions by established and emerging scholars performing across-the-aisle research on new technologies, exploring topics such as facial and gait recognition; EEG and audiovisual materials; surveillance; and sound and images in relation to questions of sexual identity, race, ethnicity, disability, and class and includes examples from a range of films and TV shows including Blade Runner, Black Mirror, Mr. Robot, Morgan, Ex Machina, and Westworld. Through a variety of critical, theoretical, proprioceptive, and speculative lenses, the collection facilitates interdisciplinary thinking and collaboration and provides readers with ways of responding to these new technologies.

Part One

AI and Robotics

1

Could the AI of Our Dreams Ever Become Reality?

James L. McClelland

Ava, the humanoid robot in Alex Garland’s Ex Machina (2014), startles us with her beauty, her sexuality, her vulnerability, and her intelligence—and ultimately with her willingness to deceive and to exploit others’ weaknesses. She seems, and of course she is, all too human, even if, in a science fiction world, she has capabilities that exceed our own. We fear her because we are all too aware of our human frailties and limitations and imagine that someday, an artificially intelligent being with all of our abilities and none of our limitations will be created and, like our human conspecifics, will be all too liable to exploit our weaknesses, leaving us unable to control the outcome.

Watching Ex Machina, I was struck by how different Eva seemed to me than the artificially intelligent computer systems that we have today. It is true that in one of its matches, a contemporary artificial intelligence, DeepMind’s AlphaGo,¹ made a move that no human understood or anticipated—a move widely credited with giving it an advantage that let it go on to win its match again the Korean grandmaster Lee Sedol. We can marvel at AlphaGo and its apparent intuition and insight, and perhaps this alone is enough to spark the fears that Ava instills. Yet, AlphaGo is ultimately only a computer program, an object that runs entirely under the control of the scientists and engineers who created it—and perhaps, more importantly, has no will of its own. AlphaGo and its successor Alpha0 (“AlphaZero”) are ultimately entirely mechanical systems whose capabilities derive from the brilliance of the computational intelligence researchers who designed it and the hardware and software engineers who turned its design into a reality. This program, which takes a board position as input and produces a legal move on its output, can learn through massive experience while playing against a series of ever-improving previous versions of itself. But an instance of Alpha0 that can beat every human player in the world at chess doesn’t know anything about absolutely anything else, and the same instance of the program cannot learn to play both games at the same time. Furthermore, you can’t talk to it, it can’t explain itself, and it cannot learn except through millions of games of play experience.

For me, it is useful to contrast today’s AI systems like AlphaGo with a PhD student in the emerging field of computational intelligence, at the interface between human cognition and artificial intelligence. Comparing these programs to Ava is more difficult because some aspects of her abilities are difficult for me to separate from her overt sexuality and the mixed-up motivations of her creator. Setting these more fraught issues aside, in what follows I will focus on the purely intellectual side of human likeness, and on the emergence of advanced intelligent functions in researchers who go on to be independent contributing scientists. I have been lucky enough to have had many excellent PhD students and post-docs in my own laboratory over the years, and many of them have gone on to be professors at outstanding universities—or, more recently, computational intelligence researchers in AI companies. Surely, we would call these young scientists intelligent. What do they have that today’s AI systems lack?

To help us consider this, I’ll introduce Dana, a fictional young PhD scientist. I use as pronouns ‘e, ‘s, and ‘em to emphasize Dana’s humanness while avoiding designating a gender. I’ll start with more basic properties I think all humans possess, and then go on to consider what it is that makes Dana and others capable of succeeding in what I will consider to be the hallmark of intelligence: identifying and successfully addressing novel and previously unsolved challenges.

Mutual Simultaneous Constraint Satisfaction

Something very basic that humans possess and today’s AI systems do not is the ability to exploit multiple simultaneous sources of information to settle into an overall interpretation of a situation and its parts or aspects and/or to formulate a plan of action that addresses many such constraints simultaneously. A beautiful visual illustration of this is provided in figure 1.1. At first, we may experience this picture as an inchoate assemblage of splotches of ink, but at some point, we are likely to begin to see that the photograph depicts a dalmatian with its back toward the viewer sniffing at the ground. None of the blobs individually appear to signal the presence of a dog, but somehow, when all are considered together, the percept emerges. At the moment we see the dog, we also see the blobs differently. Some now help define the contours of its body or are seen as spots on the dog’s coat, and other blobs now become scattered leaves on the ground or parts of a tree. We can even perceive the contour of the dog’s back where no actual contour is present in the image. Thus, the perception of the whole emerges from constraints provided by many aspects of its parts, and the perception of the parts depends in turn on the perception of the whole. This is what I mean by the idea of multiple, mutual constraint satisfaction.

Figure 1.1 A Dalmatian dog emerges from an assemblage of individually uninterpretable blotches. From James, R. C. (1965), Photo of a dalmatian dog. LIFE Magazine, 58(7), 120. Copyright © 1965 Ronald C. James.

Experiences like my seeing this photograph converged with findings in the psychology of perception and language understanding, inspiring me and others to think that it might be useful to view our perceptual systems as neural networks, because of several key properties that neural networks have that seemed to make them suitable to capture this kind of experience. The goal was not simply to simulate the brain, but to draw on the properties that might make the brain especially useful to solve this kind of constraint-satisfaction problem. The brain contains hundreds of millions of neurons, each capable of receiving inputs from up to one hundred thousand other neurons. Each neuron adjusts its activation depending on the inputs it receives from others, and in turn, signals its activation to other neurons via its outgoing connections. Inspired by this idea, which we called Parallel Distributed Processing, David Rumelhart and I teamed up with others and drew on earlier work to develop neural network models that simulated this mutual constraint satisfaction process.²

A key part of the inspiration for our work was the idea that the constraints influencing the outcome of perception or understanding can come from a wide range of sources. Our brains naturally and automatically integrate input from sight, sound, touch, posture, motion, smell, and taste in interpreting the inputs we receive. Spoken and written language contribute to and participate in this process as well; the words and sounds we experience hearing depend on other sources of input that accompany them, and likewise the objects that we perceive through other senses are simultaneously constrained through language. Constraints affecting perception and thought can come from a wide range of mutually constraining sources.

Another potent source of constraint is input from memory. Consider this tiny story:

John put some beer in a cooler and went out with his friends to play volleyball. Soon after he left, someone took the beer out of the cooler. Meanwhile, the volleyball match was very intense, and it seemed that John’s team was going to lose. But after plenty of fierce competition, John’s side was able to pull out a string of victories and won the final game when John served an amazing service winner that no one on the other team could even touch.

John and his friends were thirsty after the game and went back to his place for some beers. When John opened the cooler, he discovered that the beer was ___.

In this situation, if you as a reader have been following the story, you will anticipate that the missing word is “gone” and this will influence how likely you are to perceive it from a very brief or indistinct presentation of the word itself or a misspelled version of it. But if the text had said “someone took the ice out of the cooler” you would instead be ready to perceive the word “warm.” We as humans have the ability to exploit such constraints based on information we encountered in the indefinite past, not just the immediate current context.

Finally, the considerations that may come into play are potentially unbounded and seemingly unrelated to a particular situation at hand. I believe I heard a version of the anecdote below from Jerry Fodor. Whether it really happened I don’t know, but it seems to capture something real about how we think.

Jeff, a good bridge player, has just bid six Hearts and is about to start play on the last bridge deal at the end of an evening at his bridge club. Another player, Al, from a table that has just finished its last deal, comes over, walks around the table to see the hands of all of the players, and lingers to observe the play. The player to Jeff’s left makes the opening lead. As Jeff’s partner lays down the dummy hand, Jeff surveys the situation. It looks like an easy contract. But Jeff notices that Al is still hanging around. This makes Jeff think: maybe the hand is not such an easy one after all. If it were, Al would surely have lost interest by now. He ponders: what could conceivably go wrong? Seeing only one possibility—one that would ordinarily seem remote—he devises a plan of play that would ordinarily fail but succeeds in this case, and triumphantly, he makes his contract. His opponents are outraged and complain to the director. But the director can do nothing, since Al never said or did anything that was against the rules in any way.

Here Jeff is using information from outside of the domain of the game itself to reason about what to do within the game. It was Fodor’s point, and one that I agree with, that there is no limit on the constraints that we can ultimately bring to bear when we think and reason. In other words, the constraints that can enter into our mental constraint satisfaction process are completely open-ended.

I have described here what to me are extremely bas...

Table of contents

• Cover
• Half-Title Page
• Dedication
• Series Page
• Title Page
• Contents
• Acknowledgments
• List of Editors and Contributors
• Introduction
• Part One AI and Robotics
• 1 Could the AI of Our Dreams Ever Become Reality?
• 2 Director Alex Garland Converses with Cybermedia’s Scientists and Media Scholars
• 3 (S)Ex Machina and the Cartesian Theater of the Absurd
• 4 Epiphany, Infinity and Transcendent AI
• Part Two Big Data, Sentience, and the Universe
• 5 A MASSIVE Swirl of Pixels: Radiohead’s “Go to Sleep”
• 6 The Rise of the Machine: Body-Knowing, Neural Nets, and Emergent Freedom
• 7 The Quantum Computer: Sci-Fi Narrative’s Favorite Character
• 8 Composer Ben Salisbury Discusses Scoring Science for Alex Garland
• 9 Ex Machina and the Question of Consciousness
• Part Three The Neuroscience of Affect and Event Perception
• 10 “A Solid Popularity Arc”: Affective Economies in Black Mirror’s “Nosedive”
• 11 Cognitive Boundaries, “Nosedive,” and Under the Skin: Interview with Jeffrey Zacks
• 12 Toward an AI Future of Comics Study and Creation: A Cognitive-Affective Approach
• Part Four The Digital West
• 13 The Philosophy of Westworld
• 14 New Visions of the Old West: AI, Self, and Other in Westworld
• 15 Scoring Music for Westworld Then and Now: A Cognitive Perspective
• Part Five Interface, Desire, Collectivity
• 16 Director Terence Nance Discusses Random Acts of Flyness
• 17 The Gift of Black Sonics: Interface and Ontology in Sorry to Bother You and Random Acts of Flyness
• 18 Technology, Chaos, and the Nimble Subversion of Random Acts of Flyness
• 19 Expecting the Twist: How Media Navigate the Intersections Among Multiple Sources of Prior Knowledge
• 20 Face Color
• Part Six Productive Neuropathologies
• 21 Digital Vitalism
• 22 Neuroplasticity: From Experience to Healing
• 23 Where is My Mind? Mr. Robot and the Digital Neuropolis
• 24 The Dopamine Circuits of Wanting, Liking, Habit and Goals: An Interview about Mr. Robot with Neuroscientist Talia Lerner
• 25 The Taste of Cybermedia: An Interview with Hojoon Lee, The Lee Lab at Northwestern University
• Index
• Copyright