Augmented Reality

Unboxing Tech's Next Big Thing

Mark Pesce

• Publisher: Polity
• Year: 2020
• ISBN:
  9781509540952
• Edition: 1
• Topic: Social Sciences
• Subtopic: Media Studies
• Index: Social Sciences

Slated as 'the next big thing in tech', augmented reality promises to take the screen out of our hands and wrap it around the world via 'smart spectacles'. As a pervasive, invisible interface between the world and our senses, AR offers unparalleled capacity to reveal hidden digital depths, but it also comes at a cost to our privacy, our property, and our reality. In this crucial and provocative book, Mark Pesce draws on over thirty years' experience to offer the first mainstream exploration of augmented reality. He discusses the exciting and beneficial features of AR as well as the issues and risks raised by this still-emerging technology – a technology that moulds us by shaping what we see and hear. Augmented Reality is essential reading for anyone interested in the growing influence of this impressive but deeply concerning technology. As the book reveals, reality - once augmented - will never be the same.

1
The Will to Empower

Control has always been at the center of the human relationship to machines.

The steam engine existed as a toy in Alexandrian-era Egypt. Harnessing steam for work proved difficult and dangerous until 1788, when James Watt added a “centrifugal governor,” applying the principle of “feedback” to the steam engine, allowing the engine to monitor and regulate its own output. Although the modern steam engine had been invented by Thomas Newcomen three quarters of a century before, Watt’s addition of self-regulation saw him credited as the “father” of the steam engine.

Watt’s steam engine produced a new tension: As it became possible to build machines vastly more powerful than human beings, it became ever more important to rein in their power. The catalog of the ills of the Industrial Revolution feature many entries that demonstrate the destructive power of machines: Their capacity to rip limbs from bodies, overwhelm their operators, or catastrophically fail to self-regulate – then explode.

In the mid-nineteenth century, James Clerk Maxwell – who gave the world the eponymous equations unifying the electromagnetic forces – wrote an influential paper, On Governors, framing the centrifugal governor as a key example of a control system, a system capable of self-regulation.

With the growing capacities of steam-driven machinery – embodied in the locomotive – the idea of control became synonymous with the idea of safety; with the right controls, delivered both automatically and through human oversight (in this case, a locomotive engineer), inhuman forces could be deployed for human benefit.

The word governor has roots that stretch back into Latin – gubernator – and further back into Greek – κυβερνήτης – “kubernetes,” derived from the word for “steersman,” “pilot,” or “guide,” that is, the person who responds to the boat, the current, and the wind, integrating all of that into subtle shifts and changes in direction. In the 1940s, that governing metaphor of self-control – a pilot guiding a boat through dangerous waters – would find its modern form, in weapons.

A weapon focuses power toward a destructive end, but must not destroy whoever deploys that force, and so exists in that uneasy tension between power and control. Killing power must be balanced rationally, and by design.

In the immediate post-war period, the immense technical advances made by both sides during the conflict slowed and deepened. Although the Messerschmitt jet fighter had been used to great effect by the Luftwaffe, only after the war did aeronautical engineers have the luxury to experiment, expand, and improve on those early efforts. Rapid innovation led to a jet that flew so fast it could approach the speed of sound, with fatal consequences as it encountered its shock wave; or could bank into a turn so rapidly that it slammed the pilot with intense gravitational forces, driving them to unconsciousness as blood pulled away from the brain. The jet fighter had capacities embodied in its design that could prove fatal.

In order to minimize the opportunities for disaster, these early jet fighters crowded their cockpits with an exhaustive array of instrumentation needed for flight, navigation, and combat operations. This stretched the pilot in another direction – toward a condition of “cognitive overload.” So much important data, streaming into so many displays, can easily avalanche into confusion. Many pilots tried to master these early jet fighters; only a few had the necessary capacity to safely integrate all of the information they provided, achieving “symbiosis” with the machinery, where aircraft and pilot behaved as a single unit.

That symbiosis of man and machine emerged as one of the central themes of academic discourse in the early post-war era. The “Macy Conferences,” gathering the best and brightest from disciplines as diverse as mathematics, neurology, and anthropology,¹ worked toward symbiosis using concepts garnered from both “hard” and “soft” sciences: The 300-year-old Cartesian split between res cogitans, our inner life, and res extensa, the world of things. Symbiosis, for the Macy conferees, pointed toward a new, active relationship between the inner and outer worlds, one that could be embodied in new kinds of machines.

Each participant, eminent in their own discipline, came with an open mind – and a set of proposals. None came more prepared than Norbert Wiener, a true polymath whose talents extended to mathematics, electronics, systems theory, computing – and a new discipline, which he’d both invented and named after that Greek word for “steersman”: cybernetics.

Wiener aimed for nothing less than a complete description of the theory of control in any system – living or mechanical. Laid out in elegant though nearly inaccessible mathematical detail in his 1948 volume Cybernetics,² Wiener introduced several concepts that have, in the years since, become so commonplace they seem almost obvious. First among these, feedback: The idea that the output of a process can be used to help control that process. For Wiener, obsessed with the theory of information, this meant every process generates information that can be used to control it.

Two years later, in The Human Use of Human Beings, Wiener used far more accessible language to explain his cybernetic hypotheses, writing, “Information is the name for the content exchanged with the outer world as we adjust to it, and make our adjustment felt upon it.”³ In this economy of language Wiener unites res cogitans with res extensa via the exchange of information, describing a system where messages from the world change the internal landscape, while interiority sends messages that change the world, each process informing the other in a dance of feedback.

During the war, Wiener embodied his theories of control and feedback in the design of electronic circuits controlling anti-aircraft guns, allowing them to automatically track their targets. Five months after VJ Day, ENIAC, the first true digital computer, gave mathematicians like Wiener a tool that could “process” information along lines determined by the logic of a “program.” Yet, for all its brute force as a numerical calculator, the computer had no significant connections to the outside world, beyond a few switches and relays. All mind and no body, these first computers had res cogitans without res extensa. They could think, but they could not do, nor could they sense the way their thoughts shaped the world, and respond.

To bridge that divide, the computer would have to become a cyber-physical system, a combination of mind plus senses paired with an ability to affect change in the world, then sense those changes. Taking in data continuously generated by a range of sensors, integrating it into an assessment of state, the cyber-physical device could compute decisions that would then be translated into actions. These would in turn generate new sensor data, requiring further integration and assessment, producing still further decisions and actions, generating still more data – on and on and on in a continuous flow, an exchange of content between the outer and inner worlds that seemed to perfectly embody Wiener’s conception of cybernetics.

At its essence, a cyber-physical device must be responsive in real time; sensing, deciding, and acting without delay. This real-time operation had not been considered in the design of early digital computers. In 1951, WHIRLWIND, the first computer designed for cyber-physical operations,⁴ proved that a computer program could respond in real time to a range of sensor inputs, and, although far too large, heavy, and power hungry to find its way into a jet fighter, WHIRLWIND acted as a key “tool for thinking,” a canvas upon which a new generation of computing pioneers could inscribe their own ideas for a symbiosis between a cyber-physical system and a human operator.

Research into “man–computer symbiosis” took a deadly serious turn as the Cold War commenced and development of megaton thermonuclear weapons created the specter of imminent planetary apocalypse. Military commanders needed to operate effectively and accurately within an increasingly narrow window between the launch of a surprise nuclear attack and its culmination. Real-time decision support systems could assist with command and control in these emergency circumstances, demanding the integration and display of vast amounts of often conflicting data, without producing additional cognitive overload among already-stressed decision makers.

The design of such decision support systems owed as much to psychology as to physics or mathematics. At MIT, a post-war center for both computing and sensory psychology, psychologist J. C. R. Licklider consulted on machine-to-human interfaces for SAGE, the Semi-Automatic Ground Environment, a vast continent-wide system designed to provide military decision makers data needed to make decisions about nuclear threats and attacks. SAGE’s radar sensors scanned the airspace above and beyond the United States, looking for inbound threats, eventually spanning some 78 early warning “DEW Line” radar stations, networked into a nationwide complex of supercomputers,⁵ each staffed with an entire floor of cutting-edge SAGE graphic terminals, where military operators could see, classify, and potentially elevate any perceived threats to their commanders.

Licklider consulted on the design of the SAGE graphic terminals, developing some of the first efforts toward a symbiosis of man and computer, then shared a vision for the possibilities of symbiosis in a brief but hugely influential 1960 paper, “Man–Computer Symbiosis”:

In the anticipated symbiotic partnership, men will set the goals, formulate the hypotheses, dete...

Table of contents

• Cover
• Contents
• Title page
• Copyright page
• Timeline of Key Events in AR
• Acknowledgments
• Introduction: A Riot in Rhodes
• 1 The Will to Empower
• 2 Surveillance Status
• 3 The Last Days of Reality
• 4 The Web Wide World
• 5 Setting the World to Writes
• Conclusion: No Feint but What We Make
• Index
• End User License Agreement