Shifts towards Image-centricity in Contemporary Multimodal Practices

Hartmut Stöckl, Helen Caple, Jana Pflaeging

• Publisher: Routledge
• Year: 2020
• ISBN:
  9780429947506
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Linguistics
• Index: Languages & Linguistics

This innovative collection builds on current multimodal research to showcase image-centric practices in contemporary media, unpacking the increasing extent to which the visual plays a principal role in modern day communication. The volume begins by providing a concise overview of the history and development of multimodal research with respect to image-centricity, with successive chapters looking at how image-centricity emerges over time, unfolds in relation to language and other features in global design strategies. Bringing together contributions from both established and emerging researchers in multimodality and social semiotics, the book presents case studies on a variety of image-centric genres and domains, including magazines, advertising discourse, multimedia storytelling, and social media platforms. The aims of the book are, to interrogate the new multimodal genres, relations, forms of analysis, and methods of production that emerge from a greater reliance on visual components. Refining and broadening current understandings of image-centricity in today's media sphere, this collection will be of particular interest to scholars and students in multimodality, social semiotics, applied linguistics, language and media, and discourse analysis.

1 Shifts toward Image-Centricity in Contemporary Multimodal Practices

An Introduction

Hartmut Stöckl, Helen Caple, Jana Pflaeging

1 Introduction

This volume explores image-centric practices in the contemporary media sphere, a space that images now dominate. It does this through the application of functional-linguistic, social semiotic, text/genre-linguistic and inter-/transtextual approaches to multimodality, and by drawing on both the established and the emerging research expertise of multimodality scholars from Europe and Australia. These scholars were invited to refine and broaden the general idea of image-centricity and put it to the test in a number of different mass/social media contexts, where language, text structure, and text-image relations are dominated and led by images. These include magazines, advertising, news, and social media platforms. We use the term image as a cover term to include a range of visuals: photographs, illustrations, visualizations, and new writing/typography. The resulting volume is a rich investigation of a range of contemporary multimodal practices, examining the relative status of image and language, visual aesthetics, multimodal cohesion/coherence, and the values and ideologies underpinning image selection and presentation.

In this chapter, we chart the path toward image-centricity and review the foundational research that has brought us to our current understanding of the role of images in the contemporary media sphere.

2 Shifts toward Image-Centricity in the Contemporary Media Sphere

What is perceivable as an image-centric contemporary media sphere is the result of a shift toward image-centricity that has taken place, most notably, since the 19th century. Spurred by the Industrial Revolution, but not solely contingent on technological advancements, visual modes have gradually moved toward the center of the semiosphere.

Conditions for image-centricity were initially set through the tremendous material-technological advances of the 20th century, both in the capture and reproduction of images. This has been further enhanced by the advent of computer technology and desktop publishing software that has inspired innovations in type-setting and page composition. A broad range of typefaces, typographical settings, scalable and movable text boxes, vector-based drawing and coloring tools, and now digitally processed photographs are used to create increasingly image-centric page spaces, often on the basis of standardized templates (Johnson & Prijatel, 1999; see also Pflaeging, 2017a).

Sociocultural changes in the values associated with images have also contributed to the shift toward image-centricity. The affective value of the single still photograph, its ability to move public to action, emerged with the documentary photographers of the late 1800s to 1900s. The photography of PH Emerson and Ansel Adams, for example, became instrumental in advocating for improvements in urban living conditions and in the preservation of nature through the national parks system in the United States. Research by Perlmutter (1998) attributes the iconic status of certain news photographs to a range of editorial factors. These include institutional factors such as the photograph’s place on the news agenda, repetition of use, and its transposability to other contexts. Coupled with this are image-internal factors such as the ability of a single photograph to sum up an issue (its metonymic function), its potential for cultural resonance, and its often striking composition. Such factors, along with the ability of an image to project the offer of the ideal, of perfection, have also contributed to the shift in advertising materials toward the visual.

The implications of advancements in technological innovation and in the sociocultural conditions facilitating shifts in the semiotic landscape toward the visual have long been noted by the contributors to this volume. In relation to journalism, Caple and Knox (2015, p. 292) have argued that “we have witnessed a fundamental shift towards visual story-telling”, and Bednarek and Caple (2012, p. 111) are convinced that “story structure has shifted and images now tend to dominate the verbal text”. Acknowledgment of the aesthetic function of photographs has also led to the rise of image-dominated news story genres in print news (Caple, 2013), while the templated structure of online news portals has facilitated an even wider range of roles for photographs in news story structure (see e.g., Knox, 2009a, 2009b; Caple & Knox, 2017). Similarly, contemporary magazines and advertising materials have taken advantage of both technical and cultural shifts to become more image-centric.

The case of the popular science magazine National Geographic vividly illustrates the ‘rise of the image’ afforded by socio-cultural and material-technological developments. In a small-scale study on the development of National Geographic feature articles, Pflaeging (2017a, 2017b) shows that, even though the image-per-page ratio has decreased from 0.91 in 1915 to 0.53 images per page in 2015, the layout space taken up by images increased significantly between 1915 and 2015. In particular, the pattern of covering a double-page solely with an image-caption-cluster has emerged from an insignificant 2.4 percent of the compositional designs in use to a prominent 57.8 percent by 2015.

Similarly, Durrani (2017, p. 163) observes a considerable shift in the size of images in the magazine Time Asia. In the 1980s, less than one percent of images took up a full page to two pages. In the 2000s, this figure increased to 7.5 percent, and overall, 26 percent of the images were used at half a page or larger (7 percent in the 1980s). This shift in image size has been accompanied by the aestheticization of image composition (making use of more axially composed images) and by evolving page design strategies in which images, headlines, stand-first text, and negative space interact much more meaningfully with each other, to create an evaluative stance on the story.

Print advertising is another example of a genre whose history reflects a clear shift toward image-centricity (cf. Stöckl, 2014b, pp. 94–98). Long and descriptive-argumentative copy containing simple illustrations of products or their use gave way to graphic spaces that are dominated by artfully designed images and minimal text, which must directly engage with visual image elements to produce a rhetorically complex, multimodal, pictorial argumentation (Kjeldsen, 2012). When in the past it was semantically subordinated to the text, now the image leads in an anchoring or complementary text-image relation, often appropriating and re-contextualizing images that, at first sight, do not seem to be commercially relevant. Molnar (2018, n.p.) calls this modern multimodal format ‘minimalistic reminder’ to highlight its reduced semiotic form and functionality. Stöckl (2017b, p. 74) emphasizes the visually rich, semantically ambiguous, pragmatically underdetermined, and rhetorically complex nature of modern ads when he calls them ‘enigmatic’ or ‘en-riddled’.

Finally, while increasingly affordable digital cameras have inspired non-professional photographers and text producers since the 1990s, it was the more widespread use of camera phones since the 2010s that has made capturing the everyday in images a common daily routine. The advent of social media platforms, that have, to an increasing extent, afforded the share-ability or curate-ability (Pflaeging, 2015, in press) of online content, and the emergence of online audiences who take quick scrolls through their Facebook timelines, Instagram and Twitter feeds have made image-centric updates a preferred choice for social online interaction (Adami & Jewitt, 2016).

Possibly the most effective way of summing up the shift toward image-centricity is to demonstrate this through a visualization. Figure 1.1 presents a diachronic snapshot of the shift toward image-centricity among the data studied for this volume. Alongside this sits the emergence of new visually-dominated digital media at the turn of the 21st century. Such richly varied and complex modal ensembles in former and contemporary media spheres offer fertile ground for the exploration of image-centric practices, and the contributions to this volume take on this task. In doing so, they build upon the foundational research into multimodality and image-centric practices that precedes this volume.

Figure 1.1 From verbiage-centricity to image-centricity.

3 Researching Image-Centric Multimodal Practices

3.1 From Image Nuclearity to Image-Centricity

As Stöckl (2015b, pp. 51–52) notes, the concepts and methods deployed in research on multimodality generally emanate from text linguistics and semiotics and have been mapped onto multimodal phenomena. One such approach stemming from the study of the functional structure of verbiage-centric news reporting (Feez, Iedema, & White, 2008) is Caple’s (2008, 2013) works on the functional structure of image-nuclear news stories. The prominent position of image and headline in such texts has been posited to build a propositional nucleus and provide a perceptual and cognitive anchor (in) to the text, which also affects the evaluative stance taken and the (news) values a media story will encode. Accompanying text is dependent on this image-headline nucleus, as it functions to elaborate and extend pictorial content and to disambiguate the visual-verbal play that such stories tend to foreground (Caple, 2013).

While Caple’s notion of image nuclearity relates to the functional structure of a particular type of news story, it is inextricably linked to a broader communicative trend in popular forms of mass media. In this volume, we broaden the scope of this term and redefine it as image-centricity in order to account for its potential as a large-scale multimodal design strategy with far-reaching effects on text structure and perception.

3.2 What Exactly is Image-Centricity?

An explanation of image-centricity works best by exploring the concept in relation to some of the mainstream thinking on text-image relations (cf. Bateman, 2014). Text-image relations are understood here as discourse, where “the two modes appear separate yet integrated in both semantics and form” (Martinec & Salway, 2005, p. 338) and “act as a single unit of composition” (Bateman, 2014, p. 28) that constitutes a functional communicative act intentionally designed by a text-maker and consciously attended to by a recipient. Most importantly, arguing that images become dominant and central in a multimodal text implies a shift in the general balance of modes, which is the default assumption in multimodality research. That all modes contribute in their own way and in this sense ‘equally’ to the overall textual meaning has been expressed in a number of terms, such as ‘mutual elaboration’ of modes (Jewitt, Bezemer, & O’Halloran, 2016, p. 91), ‘dialogicity’ of signs (Jewitt et al., 2016, p. 111), ‘co-determination’ of meaning (Spillner, 1982, qtd. in Bateman, 2014, p. 37), ‘complementarity’ or ‘synergy’ (Royce, 1998), and ‘meaning multiplication’ (Bateman, 2014, pp. 5–7). In contrast, image-centricity clearly implies that images become the superordinate mode in a multimodal text, that the directionality of mode elaboration is from image to text/language, and that the modes have different ‘modal intensity’ or ‘weight’ (Norris, 2014, p. 90).

Such mode differences and their varying relative importance have been conceptualized in the term ‘status relations’ (cf. Martinec & Salway, 2005, pp. 343–349). Clearly, any image-centric text or genre primarily entails ‘unequal status’, and revisiting Barthes’ (1964/1977) original terminology, we can classify image-centricity as a text-image relation of ‘anchorage’. Here, mutual mode elaboration is led by a superordinate image, whereas the reverse case, verbiage- or text-centricity corresponds to Barthes’ (1964/1977) ‘illustration’ that implies a direction of mode elaboration from a text to a subordinate image. Martinec and Salway (2005, p. 344) point to the fact that when a text has the leading superordinate status, the image relates to only part of the text, whereas when the image has the leading superordinate status, the text may relate to the whole image or parts of it only. The more strongly image-centric genres, it seems, would be relating a text to an image in its semantic and functional entirety, whereas in weaker forms of image-centricity text may only involve relations to individual image elements.

In addition to an interpretation of image-centricity as unequal status relations, with the superordinate image leading mutual mode elaboration, image-centric discourse can also be seen as a special case of equal status relations. Martinec and Salway (2005, p. 343) say that “when an image and a text are joined equally and modify one another, their status is considered complementary”. This ‘equal/complementary’ status relation is realized by a whole text and a whole image “combining to form part of a larger syntagm” (Martinec & Salway, 2005, p. 344). Equal and complementary status relation echoes Barthes’ (1964/1977, p. 41) classic ideas of relay in that “the words, in the same way as the images, are fragments of a more general syntagm and the unity of the message is realized at a higher level, that of the story […]”. It is interesting to see that while Barthes (1964/1977, p. 41) clearly distinguishes relay from anchorage (and illustration), he also understood that “the two functions […] can co-exist”. Finally, a wider view of image-centricity as set out and endorsed here must rule out only equal and independent status relations, where “there are no signs of one [mode] modifying the other” (Martinec & Salway, 2005, p. 343).

In our conceptualization of image-centric genres, the most vital element appears to be what Martinec and Salway (2005, p. 345) have called ‘text subordination’. In other words, image-centric text-image relations construct a compositional unity of the two modes that is characterized either by an elaboration of the image through text or by a mutual co-elaboration of image and text in a complementary fashion. Consequently, text subordination to us may mean both a semantic centrality of the image that allows it to lead the interpr...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Table of Contents
• List of Contributors
• 1 Shifts toward Image-Centricity in Contemporary Multimodal Practices: An Introduction
• Part 1 Advances in Theory
• Part 2 Historical Developments in Image-Centric Practices
• Part 3 The Relative Status of Image and Language
• Part 4 Image-Centric Practices as Global Design Strategies
• Index