Transmediations

Communication Across Media Borders

Niklas Salmose, Lars Elleström

• Publisher: Routledge
• Year: 2019
• ISBN:
  9781000761306
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Communication Studies
• Index: Languages & Linguistics

This collection offers a multi-faceted exploration of transmediations, the processes of transfer and transformation that occur when communicative acts in one medium are mediated again through another. While previous research has explored these processes from a broader perspective, Salmose and Elleström argue that a better understanding is needed of the extent to which the outcomes of communicative acts are modified when transferred across multimodal media in order to foster a better understanding of communication more generally. Using this imperative as a point of departure, the book details a variety of transmediations, viewed through four different lenses. The first part of the volume looks at narrative transmediations, building on existing work done by Marie-Laure Ryan on transmedia storytelling. The second section focuses on the spatial dynamics involved in media transformation as well as the role of the human body as a perceptive agent and a medium in its own right. The third part investigates new, radical boundaries and media types in transmediality and hence shows its versatility as a method of analyzing complex and contemporary communicative discourses. The fourth and final part explores the challenges involved in transmediating scientific data into the narrative format in the context of environmental issues. Taken together, these sections highlight a range of case studies of transmediations and, in turn, the complexity and variety of the process, informed by the methodologies of the different disciplines to which they belong. This innovative volume will be of particular interest to students and scholars in multimodality, communication, intermediality, semiotics, and adaptation studies.

1
Transmediation

Some Theoretical Considerations

Lars Elleström

In this survey I will first place transmediation within the broader frames of intermediality and transmediality and then recapitulate some of my ideas on transmediation. I will also continually relate all the chapters in this volume to my conceptual framework and to each other. However, it is not my task here to indulge in in-depth discussions of the chapters; I will instead pick out some relevant features in them that make comparison and overview possible.

Transmediality

I define media products as those physical but not necessarily solid entities (objects or processes) that make communication among human minds possible. Intermedial relations can hence be understood as relations among different sorts of media products, in contrast to intramedial relations that are relations among similar sorts of media products. Intermedial relations can be found in all forms of communication. Although one does not always note their existence, they are ubiquitous and unavoidable. Without them, human communication would fall apart into a number of separate cognitive strata.

As intermedial relations are everywhere in all forms of human communication, I prefer to think of intermediality not as specific cases of communication but rather as a deliberate way of researching them with the aid of tailored theories and methods. Hence, intermediality, or intermedial studies, is not just any research on intermedial relations (which would mean that intermedial studies is virtually the same as communication studies) as it is research that applies certain perspectives and concepts that are designed to disentangle the complexity of intermedial relations.

Intermedial studies may be roughly divided using two main perspectives. The first is a synchronic perspective: how can different types of media be comprehended, analyzed and compared in terms of the combination and integration of their fundamental media traits? This viewpoint stresses an understanding of media as coexisting media types, media products and more specifically media modes (such as various material, spatiotemporal, sensorial and semiotic modes that generate meaning together). The second is a diachronic perspective: how can transfer and transformation of media characteristics be grasped and described adequately? Media characteristics are understood here as those things that are represented by media (its “content”, in more casual parlance). This viewpoint stresses an understanding of media that includes the existence of a temporal gap among media products, media types and media traits. This temporal gap may either be actual, in terms of different times of genesis of media, or a gap in the sense that the perceiver construes the significance of a medium on the basis of previously known media.

In the chapters in this volume, the diachronic perspective dominates. However, the authors do not explicitly write in terms of diachronicity. Instead, the diachronic perspective on media interrelations is generally referred to in terms of transmediality. From the most wide-ranging perspective, the term “transmediality” can be understood as representing the general concept that different media types share basic traits that can be described in terms of material, or more broadly physical, properties and abilities to activate mental capacities. All media products, in somewhat similar ways, are physical existences that trigger semiotic activity and can be properly understood only in relation to each other. Thus, physical media properties and semiosis are transmedial phenomena. More specifically, different media types may, largely – although certainly not completely – communicate similar things. In other words, represented media characteristics may be transmedial to various degrees.

It is only a short step from the idea that represented media characteristics may be transmedial to various degrees to recognizing that media characteristics, because of their transmedial nature, can be understood as being transferred among different kinds of media. Inserting a temporal perspective, it very often makes sense to acknowledge not only that similar media characteristics are or may be represented by dissimilar media but also that media characteristics that can in some respect be understood as the same recur after having appeared in another medium. For instance, a written email describing the television news and spoken words retelling the story of a film both include a temporal gap between what might be called source and target media, but also the implicit notion of sameness. We find the relations between email and television news and between speech and film meaningful because the events and other things that they represent are not only similar but are in some respects the same – if one understands sameness as a pragmatic rather than a metaphysical quality – although it is evident that transfers among different media always entail changes. There are no media transfers that do not entail at least some degree of transformation; sometimes, media transfers necessarily involve a great deal of transformation – indeed, which can be an asset as well as a problem.

Transfers like these have of course not been unnoticed by research. Although Roman Jakobson was certainly not the first to dwell on diachronic media interrelations, his brief statement in a linguistic article on translation from 1959 that “intersemiotic translation or transmutation is an interpretation of verbal signs by means of signs of nonverbal sign systems” could be viewed as the starting point of semiotically oriented research on diachronic media interrelations.¹ However, Jakobson was restricted by the one-sided perspective of “verbal” versus “nonverbal” signs and offered no theoretical tools for analyzing intersemiotic translation. Several decades later, in 1989, Claus Clüver developed the notion of “intersemiotic transposition”, which was an attempt to sketch a broader approach to transmedial relations than Jakobson’s.² However, Clüver’s approach is delimited by the misleading dichotomy of “verbal” versus “visual” texts that obscures the complex nature of overlapping media characteristics (some verbal media are visual, others not; some visual media are verbal, others not). Another similar approach can be found in Gunther Kress’s notion of “transduction”, from 1997,³ later developed in a publication by Dinda L. Gorlée.⁴

A few years after Kress’s idea of transduction was launched, Jay Bolter and Richard Grusin published Remediation: Understanding New Media, which has been important in highlighting the wide area of media transfers and transformation in contemporary media culture, although their all-embracing concept of remediation is not designed for analyzing the processes in detail.⁵ Just over a decade after that, Regina Schober used yet another term: “intermedial translation”.⁶ A later article published in 2011 clarified that this term should cover various types of “intermedial transformation processes”.⁷ Because the word translation provides strong associations with transfers among different verbal languages, I prefer to refer to the notion of “intermedial transformation processes” simply in terms of media transformation.⁸ “Media transformation” is thus for me an umbrella term referring broadly and indiscriminately to all forms of media diachronicity.

Transmediation and Media Representation

In my book Media Transformation: The Transfer of Media Characteristics Among Media (2014), in which I try to form a thorough conceptual framework that facilitates analysis of transfers of media characteristics among media, I initially establish a distinction between transmediation (repeated representation of media traits, such as a comic strip that tells a story that can be recognized from a song) and media representation (representation of another medium, such as a review that describes the performance of a dance show).⁹ To explain this in more detail requires the explanation of a few other concepts.

Media products are realized by what I call technical media of distribution, which should simply be understood as those solid or non-solid physical entities of processes that harbour media products. Being a media product is, to be more precise, a function triggered within what one takes to be a communicative situation, and technical media of distribution are those physical entities or processes that enable the function of being a media product – in other words, the function of communicating among human minds.

I use the term “mediate” to describe the process of a technical medium of distribution that realizes presemiotic (potentially meaningful) sensory configurations. For instance, a piece of paper is able to mediate visual sensory configurations that are (once perceived and rudimentarily interpreted) taken to be a food recipe, a bar chart, a scientific article or a musical score. If equivalent sensory configurations (sensory configurations that have the capacity to trigger corresponding representations) are mediated for a second (or third or fourth) time and by another type of technical medium, they are transmediated. In our minds, some of the perceived media characteristics of the target medium are, in important ways, the same as those of the source medium, which allows us to think that the musical score that is seen on the paper is later heard when it is trans-mediated by the sounds of instruments. In other words, the score’s vital characteristics are represented again by a new type of sensory configuration (not visual but auditory signs) mediated by another type of technical medium (not a piece of paper emitting photons but sound waves generated by musical instruments).¹⁰

The concept of transmediation involves two ideas. Transmediation is not only re-mediation – repeated mediation – but also trans-mediation: repeated mediation of equivalent sensory configurations by another technical medium (please note that remediation should not be understood in the open-ended sense of Bolter and Grusin). Hence, the composite term “transmedial remediation” is more accurate for denoting the concept in question. However, for the sake of simplicity, I prefer the brief term “transmediation” (which has been used previously by Charles Suhor to refer to a similar but not identical concept).¹¹ In any case, this term should be understood to refer to notions of both “re” and “trans” – repeated mediation through another technical medium.

Transmediation is the first type of media transformation, and the second type is media representation. Media representation involves the notion of one medium representing another medium. Media representation is at hand whenever a medium presents another medium to the mind of a communicatee. A medium, which is something that represents something in a context of communication, becomes represented itself. Clearly, such a representation may be more or less accurate and more or less fragmentary or complete; similar to transmediation, media representation involves different degrees of alteration. Because I define media representation as a kind of media transformation, I understand it to involve different types of media. Thus, if one wishes, terms such as “transmedial media representation,” “transmedial representation,” or “simply trans-representation may be used to denote the concept that I prefer to call media representation.”¹²

Media representation, such as an autobiography describing a photograph, or a film depicting a person communicating through sign language, may involve the representation of what can be described roughly as both media form and media content; a verbal description of a photograph may focus on both its overall composition and its discrete parts, and a filmic depiction of a signing person may capture both vital interrelations between different body and hand movements and individual actions. Likewise, transmediation can involve both media form and media content (and what is perceived as content from one point of view may be perceived as form from another point of view, and this form contains other content).

Whereas transmediation is about “picking out” elements from a medium and using them in a new way in another medium, media representation is about “pointing to” a medium from the viewpoint of another medium. In other words, whereas in transmediation the “frame” is not included in the transfer from one medium to another, it is in the case of media representation. For instance, a film representing a theatre performance (a film that includes a theatre performance in its story) can be understood in terms of media representation; however, a film whose story closely resembles that of a theatrical play should instead be understood in terms of transmediation.

It is convenient to distinguish these two types of media transformation theoretically, as they emphasize two conceptually different forms of transfer among dissimilar media. However, they may well overlap in practice. What one perceives to be mainly a transmediation may include minor aspects ...

Table of contents

• Cover
• Half Title
• Series Page
• Title
• Copyright
• Contents
• List of Table and Figures
• List of Contributors
• Foreword
• 1 Transmediation: Some Theoretical Considerations
• PART I Transmedia Storytelling
• PART II Ekphrasis
• PART III Transmediation: A Broad Media Perspective
• PART IV Transmediating the Anthropocene
• Index