Imagery, Language and Visuo-Spatial Thinking

Michel Denis, Robert Logie, Cesare Cornoldo, Manuel de Vega, Johannes EngelKamp

• Publisher: Psychology Press
• Year: 2012
• ISBN:
  9781135430931
• Edition: 1
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

Imagery, Language and Visuo-Spatial Thinking discusses the remarkable human ability to use mental imagery in everyday life: from helping plan actions and routes to aiding creative thinking; from making sense of and remembering our immediate environment to generating pictures in our minds from verbal descriptions of scenes or people. The book also considers the important theme of how individuals differ in their ability to use imagery. With contributions from leading researchers in the field, this book will be of interest to advanced undergraduates, postgraduates and researchers in cognitive psychology, cognitive science and cognitive neuropsychology.

CHAPTER ONE

The generation, maintenance, and transformation of visuo-spatial mental images

David Pearson

University of Aberdeen, UK

Rossana De Beni

Università degli Studi di Padova, Italy

Cesare Cornoldi

Università degli Studi di Padova, Italy

INTRODUCTION

In the last 30 years, imagery research has developed in many different directions, with different approaches, methodologies, types of observations, and phenomena. Visuo-spatial mental imagery phenomena may be classified into many different categories, including fantasy, hypnagogic imagery, hallucinations etc. In the present chapter we will focus on the involvement of mental imagery during visuo-spatial thinking. It is well known (e.g., Denis, 1989) that mental imagery can offer critical support to a variety of thinking processes, including spatial reasoning, problem solving using analogical representations, or mental discovery of novel or emergent properties. In order to implement these or similar processes, the human mind often spontaneously generates and manipulates mental images. Such manipulations can be of varying types, including the scanning, zooming, or transforming of images, but all require that the mental image be maintained during the time required for the relative manipulatory operations to be carried out.

In this chapter we will examine different alternatives concerning the cognitive architecture involved in the use of mental imagery, including the potential role played by visuo-spatial working memory. The properties of mental images and the processes involved during their generation will be discussed in relation to the different types of mental image that can occur during thinking, and the nature of the processes that may underlie their maintenance. We will then move on to consider important examples of mental manipulation, including the rotation and scanning of images, and imaged transformations of size and colour. Finally, the chapter will end by examining the involvement of mental imagery in aspects of creative thought, and in particular during the discovery of novel or emergent properties of objects or patterns based on the manipulation of visual mental images.

IMAGE GENERATION AND THE FORMAT OF LONG-TERM MEMORY INFORMATION

In this chapter we distinguish between mental images that consist of visual traces loaded directly from perceptual experience, and mental images that are generated using long-term information without any reliance on external visual support. There are also intermediate cases, such as images that are based on transformed visual input (see later section on mental synthesis), as well as images in sensory modalities other than vision, such as those that occur during auditory imagery (see Reisberg, 1992).

In some cases the image-generation process appears almost automatic and outside the voluntary control of the participant. A famous example of this concerns the author Marcel Proust, who reported being overwhelmed by a complex series of recollections apparently evoked by the experience of eating a cake that went on to form the basis of his great novel cycle Remembrance of Things Past. In other cases participants’ goals and metacognition clearly guide the implementation of an image-generation plan (Cornoldi, De Beni, & Giusberti, 1996). For example, if people have to imagine the shortest way of reaching a particular place, they can direct their image-generation process according to both the nature of the task (i.e., should they drive or walk?), and to their particular metacognition (i.e., what kind of image would be most adequate for answering this question?).

Both automatic and controlled image generation use information retrieved from long-term memory. There has been considerable debate concerning the nature of the long-term information that is used for generating mental images. In the early 1970s two radical and opposing views were presented. The propositional view (e.g., Pylyshyn, 1973) assumed that both long-term information and conscious representation were always based on a unique amodal propositional format. In contrast, the dual-system view (e.g., Paivio’s dual-code theory, 1971) assumed the existence of both linguistic and non-verbal modal formats in long-term memory and conscious representation. Subsequently different authors have proposed an intermediate position which assumes that long-term information can consist of a unique amodal format, while the conscious mental image has a format more related to the properties of the medium used during the implementation of the representation (Kosslyn, 1980; see also Marschark & Cornoldi, 1990).

As regards the format of long-term information, the computer metaphor has often been applied to show that a single type of information can be used for generating representations of different formats (e.g., Kosslyn, 1980). However, any psychological theory of imagery must offer an explanation for why mental images are sometimes directly and automatically generated with either highly specific modal information (such as a specific odour or colour), or else a predefined imaginal organisation (such as a prototypical image of a dog), or even a combination of both characteristics.

We cannot exclude the possibility that long-term memory can maintain to some extent the sensory properties of a stimulus. Cases of involuntary memories being primed (even after very long periods of time) by re-exposure to the same sensations experienced during learning (such as Proust’s experience described earlier) suggest that the memory pattern code can in some way be related to memories for specific sensory information.

The possibility that this sensory information persists, or at least can be retrieved after long periods of time, seems to depend on its original repeated exposure during conditions of high activation. Mandler and Ritchey (1977) have argued that in general visual memories appear subject to decay functions which produce a rapid loss of specific sensory information, while more general schematic information is maintained. This suggests that visual memories may initially maintain elements encoded during perceptual exposure, but that these elements are then progressively subjected to processes of transformation and integration within long-term memory which result in an increasing loss of specific sensory details (see also Hitch, Brandimonte, & Walker, 1995; Cornoldi, De Beni, Giusberti, & Massironi, 1998; differential effects of long-term and short-term memory images were also

found by Ishai & Sagi, 1997).

In summary, it can be seen that understanding the organisation of long-term memory is important to account both for which information is used during the image generation process, and also for how this information is accessed and retrieved. This issue is returned to later on in the chapter, following a discussion of the nature of medium in which mental images are represented.

THE IMAGE MEDIUM: VISUO-SPATIAL WORKING MEMORY OR A VISUAL BUFFER?

In the 1980s different positions were presented for describing the cognitive system(s) involved during the generation and manipulation of mental images. Baddeley (1986) separately considered short-term visual memory and image representation, but seemed to suggest that in both cases the same system was involved, i.e., the visuo-spatial component of working memory. Kosslyn (1980) postulated the existence of a specific system, the visual buffer, that possesses strong analogical properties. Despite the fact that Kosslyn did not consider its relationship with working memory, the two systems appear to share many characteristics (Logie, 1991). Two arguments seemed to support this conclusion: (a) if working memory is the system involved in maintaining information used in mental activity, images must by definition be within this system; and (b) if mental images have specific properties, they must be held within the working memory subsystem that preserves these properties.

However, although it may be apparent that mental images are maintained within the working memory system, it is much less clear specifically which components of working memory may be involved. The model of working memory proposed by Baddeley and Hitch (1974; Baddeley, 1986) is a tripartite system that comprises three separate components; a central executive, a phonological loop, and a visuo-spatial sketchpad. The loop and sketchpad are both modality-specific “slave systems”, the former implemented during the retention of verbal speech-based material, the latter during the retention of visuo-spatial material. The central executive is a modality-free system that supervises the operation of the slave systems, and is also assumed to be involved during strategy selection and the planning of complex cognitive tasks (Gilhooly, Logie, Wetherick, & Wynn, 1993; Toms, Morris, & Ward, 1993).

Both of the slave systems of working memory have themselves been fractionated into two inter-related components. The phonological loop consists of a passive phonological store and an active articulatory rehearsal mechanism (Baddeley & Lewis, 1981). A similar distinction has also been made within visuo-spatial working memory (VSWM), in which a passive visual cache is supported by an active “inner scribe” spatial rehearsal mechanism (Logie, 1995; Logie & Pearson, 1997). Information held in the visual cache is subject to decay unless maintained, and also subject to interference from new visual input entering the store. The active inner scribe mechanism is responsible for rehearsing the contents of the visual cache, and is also involved during the planning and execution of movement. Although spatial locations can be stored within the cache in the form of a static visual image (Smyth & Pendleton, 1989), the storage of sequential locations or movements requires the operation of the inner scribe. The scribe also extracts information from the visual cache to allow for targeted movement. Hence, any concurrent movement to discrete spatial locations can result in a disruption of the visuo-spatial rehearsal mechanism (Baddeley, Grant, Wight, & Thomson, 1975; Quinn & Ralston, 1986).

According to a revised model of VSWM (Logie, 1995; Pearson, Logie, & Gilhooly, 1999), the visual cache is considered a separate component from the visual buffer in which conscious mental images are represented. During the performance of a mental imagery task the visual cache and inner scribe can function as temporary stores for visual and spatial material, providing a means to transfer additional information to and from the visual buffer. Information held in each of the slave systems can be extracted by the central executive component and utilised during the completion of various cognitive tasks, as can semantic information held in long-term memory. Mental imagery therefore occupies the resources of the working memory system as a whole, rather than being specifically the province of the visuo-spatial sketchpad, which functions as a temporary store for information outwith the conscious image.

In fact much evidence has been accumulated showing that different imagery processes differentially involve a working memory system. For example, it has been shown that a concurrent task (spatial tapping) that typically interferes with VSWM activity does not interfere with mental imagery, whereas a task typically interfering with the central executive (random generation) may interfere with imaginal activity (Logie, 1995; see also Bruyer & Scailquin, 1998). Furthermore the passive storage function of the visual cache cannot easily be identified with the active role played by Kosslyn’s visual buffer, and only part of the stored information within VSWM need actually be utilised during imaginal activity. These results may be better interpreted if we use different descriptions of VSWM (see other chapters in this book), and also if we consider different aspects of a generated image.

The generation and maintenance of a mental image involves not only storage processes, but also active processes. The quality and/or quantity of these processes can vary according to the nature of the task both between images and within an image. Three different, though partially overlapping, aspects of an image can be used to explain this last point. First, images, like percepts, can be organised (without voluntary intervention of the participant) into a figure and a background. The figure is more activated than the background, but the background remains included in the representation. Second, an attentional window (Kosslyn, 1980) can emphasise and improve the quality of the representation of certain parts of the image, even if the other parts remain present. Third, this process can be made highly selective by “zooming in” on some parts of the image while excluding other parts from the buffer (and potentially holding them highly accessible in a separate VSWM store; Logie, 1995).

The problem of the capacity of the image medium is therefore related to the problem of defining the systems that are implicated and of defining the critical variables within the system. In principle, exceeding that capacity can imply either a loss of information or a reduction in the quality of the representation.

Kosslyn does not directly specify the cognitive resources underlying image maintenance, although he does state that they are not the same “top-down” hypothesis-testing mechanisms that function during image generation. This theoretical position can argue in favour of either a fractionation of activity related to VSWM and imagery processes (Logie, 1995; Pearson, Logie, & Green, 1996; Pearson et al., 1999), or a continuum-based model that assumes that this activity can be differentiated along a vertical and a horizontal continuum (see Vecchi, Phillips, & Cornoldi, Chapter 2, this volume). The vertical continuum progresses from low-activity processes (such as the automatic retrieval of highly available mental images), through intermediate-activity processes (such as simple image maintenance), to highly active processes (represented by complex manipulations and transformations of mental images). The horizontal continuum describes the range of variations in the format of material and representation, suggesting that information treated in VSWM and mental imagery can be located more or less further away on the continuum from other modalities of representation. For example, Cornoldi et al. (1998) assumed that visual traces are more distant from verbal and conceptual representations than mental images generated from long-term memory. Furthermore, generated images become even closer to conceptual representations if they were generated along a semantic pathway. Cornoldi et al. (1998) asked a group of participants to remember geometrical patterns of different colours (red, yellow, and blue) or of similar colours (different variations of blue). In one condition (visual trace) the figures had been seen earlier, in another condition they were generated from verbal instructions, and in a final condition (conceptual generated image) they were generated with reference to colours of world objects. Figure 1.1 shows how, in the last condition, the recall of shape, size, and colour of the patterns was enhanced and was less sensitive to a visual similarity effect.

Figure 1.1. Mean numbers of characteristics recalled by the generated image group, visual trace group, and conceptual generated image group for stimuli of similar colours (data from Cornoldi et al., 1998). Reprinted with permission.

There are a growing number of studies that have argued for an empirical distinction between generation and maintenance processes within visual imagery (e.g., Cocude, Charlot, & Denis, 1997; Cocude & Denis, 1988; Uhl et al., 1990; Wallace & Hofelich, 1992). Although such a distinction is persuasive, in terms of the working memory model it would a...

Table of contents

• Cover
• Halftitle
• Title
• Copyright
• Contents
• List of contributors
• Preface. Imagery, language, and visuo-spatial thinking
• 1 The generation, maintenance, and transformation of visuo-spatial mental images
• 2 Individual differences in visuo-spatial working memory
• 3 Pictures in memory: The role of visual-imaginal information
• 4 The processing of visuo-spatial information: Neuropsychological and neuroimaging investigations
• 5 The interface between language and visuo-spatial representations
• 6 Language, spatial cognition, and navigation
• 7 Actions, mental actions, and working memory
• Author index
• Subject index