Seeing into Screens

Eye Tracking and the Moving Image

Tessa Dwyer, Claire Perkins, Sean Redmond, Jodi Sita

• Publisher: Bloomsbury Academic
• Year: 2018
• ISBN:
  9781501329005
• Edition: 1
• Topic: Medios de comunicación y artes escénicas
• Subtopic: Historia y crítica cinematográficas

Seeing into Screens: Eye Tracking and the Moving Image is the first dedicated anthology that explores vision and perception as it materializes as viewers watch screen content. While nearly all moving image research either 'imagines' how its audience responds to the screen, or focuses upon external responses, this collection utilizes the data produced from eye tracking technology to assess seeing and knowing, gazing and perceiving. The editors divide their collection into the following four sections: eye tracking performance, which addresses the ways viewers respond to screen genre, actor and star, auteur, and cinematography; eye tracking aesthetics which explores the way viewers gaze upon colour, light, movement, and space; eye tracking inscription, which examines the way the viewer responds to subtitles, translation, and written information found in the screen world; and eye tracking augmentation which examines the role of simulation, mediation, and technological intervention in the way viewers engage with screen content. At a time when the nature of viewing the screen is extending and diversifying across different platforms and exhibitions, Seeing into Screens is a timely exploration of how viewers watch the screen.

SECTION 1

Seeing the Eye

1

In Order to See, You Must Look Away: Thinking About the Eye

William Brown

Eye-tracking studies of film (as well as eye-tracking studies in general) focus on seeing. In this chapter, I shall propose that the human visual system relies not just on seeing, but also on moments of not seeing. That is, I shall argue that moments in which we do not see are not ‘flaws’ in a visual system that otherwise strives towards total vision. Rather, such ‘flaws’ (moments of blindness) are crucial components of vision and what it means to be human more generally. I shall illustrate this need for temporary blindness by looking at three films that are comprised uniquely (or almost uniquely) of photographs, including Jonás Cuarón’s Año uña/Year of the Nail (Mexico 2007), Chris Marker’s La Jetée (France 1962) and Jean-Luc Godard’s Je vous salue, Sarajevo (France 1993). In films comprised of photographs, we see not continuous action (we do not have a ‘perfect vision’ of events) but wilfully fragmented action (even more than in films comprised of moving images that involve narrative ellipses). These films demonstrate how what we do not see is perhaps equally as important as what we do see – both in film viewing and in life. In this way, I shall argue that these films make clear how eye tracking overemphasizes vision during eye fixation, thereby under-appreciating the importance of the necessary blindness that accompanies vision. As it is cinema that draws out the oversights of eye tracking, I shall reverse the trend of science being used to explain art, providing an example rather of art being used to explain science. In order to do this, let us start with a brief explanation of human vision.

The imperfect eye

Andrew Parker has written about how – as part of the body’s response to atmospheric light – the eye evolved rapidly during, and thus played a key part in, the so-called Cambrian explosion, whereby the diversity of life on earth developed rapidly about 542 million years ago. Although eyes did not necessarily originate with the Cambrian explosion, Parker nonetheless links the development of the eye with predation: being able to see at some distance rather than smelling at close range or touching as a result of proximity enabled life forms to develop new ways of hunting and to find new ways of avoiding being hunted. As a result of these new strategies of survival, many new life forms evolved – in effect, a different species evolved for each different survival strategy (Parker 2003).

While Michael F. Land and Dan-Eric Nilssen suggest that eyes have evolved separately in many different species (i.e. not all species with eyes evolved from the same ancestor; see Land and Nilssen 2002), nonetheless, eyes seem to have evolved from photosensitive proteins found within cells to photoreceptive cells, which themselves clustered to become the first ‘eyespots’ and then, with the addition of lenses, what we might today refer to as eyes. The point is that while Charles Darwin considered the eye to be ‘complex and perfect’ (1998: 143), the eye has in fact evolved to view the world in a heuristic/evolutionary fashion; thus, if the contemporary eye is exceptional in its photosensitivity, being able to pick out details at a distance, even in low levels of light, in some senses the eye cannot be ‘perfect’: it is necessarily imperfect in that it does not always work, nor does it take in all information that surrounds it.

In some senses, to say that the eye is not perfect is obvious: Amos Vogel has pointed out how the eye is sensitive to only about 5 per cent of the light spectrum (2005: 12–19), while the fact that humans cannot see 360 degrees at once would also seem to contradict Darwin. Furthermore, we know that humans regularly fail to spot things in their field of vision (see, for example, work on ‘change blindness’; Simons and Levin 1997) and as a result miss both prey and predators. We might contend in such instances that the eye is working perfectly and that it is the rest of the body and/or brain that is the major contributor to the demise of the human, thereby necessitating a discussion regarding where the eye begins and ends in relation to the brain and the body. However, I wish to bypass such discussions and assert that the eye is not perfect and that it can always get better not just at taking in information but at taking in a greater range of information. Indeed, the very imperfection of the eye might in some respects explain mankind’s development of optical tools, from glasses to microscopes to telescopes, and so on, tools that supplement the eye both on the level of the individual (contact lenses help me to see ‘better’) and on the level of society (knowing about the micro- and macroscopic worlds potentially benefits all of humanity).

To see or not to see

The human eye works by receiving light reflected from the environment, which, upon striking rods and cones in the fovea, is then turned into brain signals that create vision. The blind demonstrate clearly that vision is not necessary in order for humans to live respectable lifespans, with numerous blind people from myth and history even being upheld for their notable achievements, including Tiresias from Homer’s Odyssey, John Milton and Jorge Luis Borges. Furthermore, while most humans are not blind (even if vision is imperfect, including in the sense of not having perfect vision, i.e. 20:20 vision), vision itself is predicated on gaps, or moments of blindness, which themselves often seem invisible to humans in that we tend to overlook them. These moments range from the very short to the relatively long, and they include blinking, sleeping and saccades. Each has a different function, but each also involves moments when vision shuts down. My suggestion here is that these moments of blindness are not flaws in our visual system, but structural necessities of it, as we shall see next.

The reasons why we blink are at least twofold. First, we blink in order to protect our eyes from oncoming objects. Secondly, we blink in order to spread across the surface of the eye moisture from our tear ducts. Blinking thus helps to prevent our eyes from going dry, as excessive dryness might cause blindness. There is an important irony here. As we are told in childhood not to stare at the sun, especially when using binoculars or other telescopic tools, the very phenomenon that enables us to see, namely light from the sun, possesses the power to destroy our vision. The eye needs to be kept moist and in some senses it needs to be kept in the shade (as implied by the use of sunglasses above and beyond their function as a fashion item).

Nonetheless, humans blink more often than is necessary for reasons of ocular lubrication alone, and so a third reason has been proposed for why we do it. Using functional magnetic resonance imaging (fMRI) to measure the brain activity of subjects watching clips from Mr Bean (various directors, UK, 1990–5), Tamami Nakano and colleagues observed that the parts of the brain associated with attention are deactivated when we blink. Nakano et al., therefore, suggest that blinking provides humans with a moment’s respite from the ongoing stream of visual information that we receive, and that this respite allows the brain to process what it has seen in a non-attentive fashi on (2013). This means that blinking potentially serves a similar function to sleep as a major reason for why humans sleep is to allow the brain to consolidate information gathered over the course of a given day (Hobson 1995).

If we need to blink and sleep in order to build memories, then what is the purpose of saccades? Saccades are movements carried out by the eye in order to search the visual field; saccades punctuate and are punctuated by fixations or moments when the eye remains momentarily still. Fixations here are not just spatial; indeed, we can take in information about what is in our peripheral vision when we are not directly looking at something (so-called covert attention). Rather, fixations are a primary temporal mechanism for the eye to take in visual information (even if the fixations are only very brief – as when we quickly get the ‘gist’ of a scene based on only a rapid glimpse; see Rayner 1998). In other words, we may see more in our field of vision than that upon which our eye fixates, but we nonetheless only see when our eye fixates, and not when it saccades.

While we can choose to make them, saccades are primarily involuntary. What happens is that the eye performs small, ‘anarchic’ (i.e. random) movements that allow it to attend to slightly different parts of the visual field, which in turn allow humans to create a more accurate and detailed image of their surroundings (keeping an eye out for predators, prey and mates). It is the ‘anarchy’ of the saccade that allows for precision: ‘Visual searching is free-running (“anarchic”) because commanded, ordered deployment of attention is so much slower than anarchic deployment that it is faster overall to make many anarchic attentional deployments than fewer orderly ones’ (Wolfe, Alvarez and Horowitz 2000: 691). While anarchic, though, the important thing to note for this chapter is that we do not consciously see while we saccade. As Benjamin W. Tatler and Tom Trościanko explain, we can easily verify this by looking at our own eyes in a mirror: ‘Looking from one eye to the next it is not possible to see the eyes moving – they appear always to be stationary’ (2002: 1403). ‘Saccadic suppression’ likely takes place because our visual field would jumble about and we would lose orientation if we saw during eye movement. Fortunately for us, though, our field of vision remains pretty consistent.

Akin to sleeping and blinking, then, saccades provide a necessary moment of blindness that allows us to see what is before us during fixations. ‘Blindness’, in this respect, is not a deficiency or a weakness, but a structuring principle of vision itself. Thus, vision is not an issue of seeing or not seeing, but a necessary combination of the two. Without wishing to sound too ‘Zen’, in order to see, you must not see. In order to see, you must look away.

Persistence of vision

With this (unconventional) understanding of vision in place, we shall soon turn our attention to cinema, including the role that eye tracking plays in our understanding of film. In order to get there, though, an important question must be answered: namely, if we are not conscious of seeing (if perhaps we do not see) while we saccade, then how is it that vision is not fragmented and discontinuous, but continuous across saccades (and even some blinks)? Many scientists have long since worked on this issue, but a good way to answer this question can be found through a theory that, oddly but appropriately enough, has held mistaken sway in film studies for a long time, namely persistence of vision.

Joseph Anderson and Barbara Fisher point out how persistence of vision has often been attributed (erroneously) to physician Peter Mark Roget, also the inventor of the thesaurus (Anderson and Fisher 1978). Roget noticed that a passing cart, when seen through the vertical slits of a window blind, appeared to be jumping from one static position to another as opposed to moving continuously between each slit, and that the spokes of the cart’s wheels, instead of looking straight, in fact looked curved. While the curvature effect is not illusory (the movement of a rotating line seen through a slit will indeed trace a curve), Roget nonetheless struggled to find a convincing reason for the static appearance of the cart and its wheels (Roget 1824). Although he does not use the term, Roget suspected that the effect had something to do with afterimages, or the belief that an image from the outside world remains fixed on the eye for a brief period. For Roget, the occlusion of the cart by the slits in the blind, followed by the cart seeming static when visible beyond the slits, confirmed the existence of afterimages, as the experience apparently demonstrates how vision is really a succession of still images (that linger on the eye), and not a perception of movement itself. Afterimages help to create persistence of vision.

However, afterimages do not explain persistence of vision. If they did, then what Roget and other sighted humans would see in this case would be ‘a plethora of images resulting from the tracings scattered about the retina according to each separate fixation of the eye’ (Anderson and Fisher 1978: 6). That is, the cart would bounce around our field of vision as we see static images upon each eye fixation, with each static image then lingering on the eye for the period of a saccade before being replaced by another image with the cart having jumped to a different point within our field of vision as a result of the movement of our eye (and the movement of the cart).

The important point to make here is that persistence of vision does not take place on the retina, that is, via afterimages lingering solely in the eye – a belief that is the kernel of Roget’s error. Rather, as Anderson and Fisher explain, the continuity that we see takes place not on the retina but in the brain as a result of the ‘direct path from the receptors in the eye to the brain’, with the processing of what we see ‘deferred until the signal reaches the brain’ (1978: 7). As Anderson and Fisher (now Anderson and Anderson) describe in a subsequent paper, the continuity of vision has since about 1900 been treated ‘almost without exception as principally a central ph...

Table of contents

• Cover
• Half-Title
• Dedication
• Title
• Contents
• Acknowledgements
• Introduction: The Blackest and Whitest of Swans Tessa Dwyer, Claire Perkins, Sean Redmond, Jodi Sita
• Section 1: Seeing the Eye
• Section 2: The Eye Seeing
• Biographies
• Index
• Copyright