How the Brain Processes Multimodal Technical Instructions

Dirk Remley

• Publisher: Routledge
• Year: 2016
• ISBN:
  9781351865401
• Edition: 2
• Topic: Languages & Linguistics
• Subtopic: Communication Studies
• Index: Languages & Linguistics

While Aristotle acknowledges the connection between rhetoric, biology, and cognitive abilities, scholarship continues to struggle to integrate the fields of rhetoric and neurobiology. Drawing on recent work in neurorhetoric, this book offers a model that integrates multimodal rhetorical theory and multisensory neural processing theory pertaining to cognition and learning. Using existing theories from multimodal rhetoric and specific findings from neurobiological studies, the author develops a model that integrates concepts from both fields, bridging, if not uniting, them. He also discusses possible applications of the new model, with specific case studies related to training and instruction. These applications include various media used in instructional and training contexts, such as print, slide shows, videos, simulations, and hands-on training. The book thus introduces concepts of cognitive neuroscience to multimodal rhetorical theory and facilitates theorization combining multimodal rhetoric and multisensory cognition, and serves as a vehicle by which readers can better understand the links between multimodal rhetoric and cognitive neuroscience associated with technical communication. Integrating case studies from industry and practice, the text makes explicit connections between academic scholarship and workplace preparation. It also describes how interdisciplinary research can contribute to pharmaceutical research, as well as the development of productive instructional materials. Rhetoric is affected by how the brain of any member of a given audience can process information. This book can promote further research-qualitative and quantitative-to develop a better understanding of the relationship between multimodal messages and how the brain processes such information.

CHAPTER 1
A Gap to Bridge

How does one learn a new technical concept? A teacher or trainer can provide information about that new concept to a student or new employee; he or she can even show the student or employee how to practice or apply the concept. However, what happens in the student’s mind to help him or her learn the concept and its application? Learning involves many attributes and dynamics generally associated with cognitive psychology, education, semiotics, rhetoric, and neurobiology. That is, the ways one presents instructional information and the student’s natural biological attributes associated with processing that information affect how the student learns. However, these are disparate fields of scholarship, and they pursue study of the question, “How does one learn a new technical concept?” differently.

Neurobiologists Calvert, Spence, and Stein (2004) noted that, because the scholarship related to neuroscience is “spread across multiple disciplines, it has become increasingly fragmented in recent years” (p. xii). However, in a special issue of Technical Communication Quarterly, Rivers (2011) encouraged a multidisciplinary approach to research into cognitive science, recognizing the roles that biology and social environment as well as technology play in cognition. Alluding to the convergence of tools, environment, and brain in distributed cognition, he stated that, “those tools and that world are always part of the mind itself” (p. 415). With this book, I attempt to synthesize some scholarship in disparate fields to provide a bridge by which interdisciplinary study that may enhance the development of instructional materials may occur. I introduce a model by which to study the question, “How does one learn new technical concepts?” This model recognizes that learning involves social, biological, and rhetorical attributes that engage many senses. Consider these examples:

It is a frequent experience—having to drive to a doctor’s office for the first time. You contact the doctor’s office, and the receptionist gives you directions to the office, usually identifying particular landmarks—a restaurant or shopping plaza near it and approximate distances from those items to the office building—“about 100 yards after that restaurant, you’ll cross an intersection. On the right you’ll see a gas station; our building is right across the street from that”—perhaps even the color of the building—“it’s grey.” You get into your car on the big day of the appointment, and you pay close attention to these visual and spatial cues, looking carefully for the restaurant, seeing it, and understanding that you are very close to the building. You suddenly see a gas station and you look across the street; there it is—the grey building. Using the information—visual landmarks and spatial relationships—you succeeded in finding the building. Now you have only to navigate the building to find the doctor’s office!

Another frequent experience is that of hearing the blare of sirens from an approaching emergency vehicle before you see it. Sometimes one may never actually see the vehicle—fire engine, ambulance, or police car—understanding when it passes the vicinity, though, from the changing sound of the blaring siren. Sometimes one sees the vehicle a few moments after hearing the sirens and moves their vehicle aside or watches it pass. In either case, one has learned what such a signal generally means and understands that there is an emergency somewhere and acts accordingly. Whether one sees it or not, an audio message is sent, and a listener reacts to it before seeing and may react again upon seeing it. I experienced some surprise when I heard and then saw a fire engine approach my house, though I understood partially the reason. The implication, though, raised further questions that prompted more multisensory experiences connected to an emergency. I relate the experience later in this chapter.

In each of these cases, though, different senses were engaged to help one understand a situation. A number of dynamics related to one’s understanding of the world, cognition, and forms of representation played out so that person could understand and act accordingly. There is a multimodal rhetoric to these experiences. The field of “rhetoric” generally engages the question of how one may convey a message, either in writing or orally, to bring about a particular response from an audience. The field of multimodal rhetoric recognizes that one may convey their message using multiple forms of representation, thereby engaging multiple senses in an audience and creating meaning with combinations of these modes of representation. However, which combinations most effectively facilitate learning depends on a number of social, biological, and cognitive attributes associated with the learner.

The field of social semiotics, further, recognizes that meaning is a social construct, that is, one’s interpretation of various images and objects evolves through interactions with others. One has certain sensory information provided to them based on experiences and knowledge that likely involve social settings and interactions with others: You understand the message and you respond to the message according to its purpose.

The cognitive experience is rhetorical and social. We learn about new concepts by being exposed to information about them and interacting with phenomena associated with the new concepts. This interaction may be through discussion or practical experience. One may provide information to us in a way that will help us to understand a new concept; this is both a social phenomenon—interaction with another—and it is rhetorical—a message is provided to an audience (us) with the purpose of helping us understand something new. It takes some interaction with the world around us to comprehend a situation and the meaning of the information provided. However, it is also biological. Cognitive science generally recognizes these attributes of cognition—social and biological attributes related to facilitating an understanding of our world. However, the discussion of these cognitive neuroscience dynamics is complicated by disciplinary discourses and exclusions.

Each discipline approaches the topic from its own angle, recognizing that literature from that field is needed to support such scholarship. For example, rarely will the author of a scholarly article cite work from outside their own discipline or the discipline of the particular journal. This extends to scholarly books too. For example, in his highly regarded book Cognition in the Wild, Hutchins (1995) limited the discussion of cognition and social semiotics to cognitive psychology and distributed knowledge theory. Also, in How the Mind Works, another highly regarded work of cognitive neuroscience, Pinker (1997) integrated some discussion of neuroscience on cognitive processes, however, he focused on historical development of cognitive processes and psychological evolution. Finally, Gruber (2012) highlights discourse differences in how scholars treat the neuroscientific concept of mirror neurons, neurons that help an audience interpret and copy behavior they view. Such discourse exclusion limits the lens through which studies examine the phenomena.

Further, approaches to empirical study affect the lens through which scholars view phenomena. Typically, empirical study of learning from social science perspectives involves presenting subjects with instructional materials, having them review those materials, and then assessing their performance on a test or other means of learning assessment to ascertain how much they learned from the instructional materials (see studies by Moreno & Mayer, 2000, for examples). Studies from physical science perspectives may ask subjects to review instructional materials while the subject wears equipment that measures and facilitates observation of electrical or blood activity in the brain to ascertain which parts of the brain are involved in processing information (see studies reported in Calvert et al., 2004, for examples). While social scientists appear to be studying how well certain instructional materials facilitate learning—what works and what does not—physical scientists are examining the biological dynamics of cognition and how certain biological attributes may affect learning—why certain approaches work and others do not.

The purpose of this book is to address some of these differences so that scholars in both fields can pursue interdisciplinary research that may enhance our understanding of cognitive psychology and neuroscience generally. Study that recognizes the value of considering what works as well as why it works is important toward facilitating a more comprehensive consideration of cognition. Specifically, I describe a model that integrates neurobiological attributes involved in cognition and social attributes involved in learning new information provided using multiple modes of representation. I refer to it as a neurocognitive model of multimodal rhetoric or as an integrated model of cognition. I focus on it specifically relative to instructional materials and how they can help an audience learn. Attributes of cognitive psychology and neuroscience engage many different functions, including decision making as well as changing perspectives and management (Pillay, 2011). Some of these overlap with learning, so I integrate them here. Any form of communication and rhetoric includes consideration of the purpose of the message. Instruction is just one of many purposes. The focus of the model is on how the brain processes new technical information or concepts toward learning them. The model recognizes that some with a given neurological condition may need information to be provided in ways that differ from how it is presented to others without that condition, but the information and how to apply it can be understood and learned by both groups.

Rhetoric, in a broad sense, examines how the way information is presented affects an audience’s understanding of that information and response to it. Aristotle (1991) and Perelman and Olbrechts-Tyteca (1969) recognized that rhetoric considers the disposition of certain kinds of audiences, and one who wishes to convey an effective message must adjust to their particular audience. Aristotle acknowledged that rhetoric includes “three factors—the speaker, the subject and the listener—and it is to the last of these that its purpose it intended” (p. 80). The purpose of a message and its audience are intertwined. The message must consider the audience’s disposition in order to accomplish its purpose. This disposition can be theorized relative to social disposition or biological/physical disposition. Indeed, Aristotle noted that this likely involves an audience that may have “limited intellectual scope and limited capacity to follow an extended chain of reasoning” (p. 76). Such a statement includes physiological attributes in the rhetoric equation. If the audience’s cognitive capacities are not considered in developing the message, the meaning of the message will be lost.

Perelman and Olbrechts-Tyteca (1969) recalled this emphasis on the audience, pointing out that “it is in terms of an audience that an argumentation develops” (p. 5). Indeed, they compared one who does not consider the audience to a rude visitor (p. 17). They asserted that the most important rule of rhetoric is to adapt the message to the audience (p. 25). A message is not automatically understood just because it is articulated; it must be conveyed in a way that suits the audience’s background and understandings, their experiences and practices, their capacity for cognition.

Scholarship in rhetoric draws on studies from the social science and humanities disciplines of cognitive neuroscience—social semiotics, social psychology, and language theories. Rhetoric is certainly a social dynamic. However, rhetoric has been left out of much of the discussion of cognitive neuroscience and is not considered among those fields. Jack (2012) as well as Jack and Appelbaum (2010) provide some introductory material for connecting rhetoric with biological fields of neuroscience in her edited collection about “neurorhetorics.” Jack and Appelbaum (2010) identified two approaches to “neurorhetoric.” One involves studying the rhetoric of neuroscience, in which one considers how different discourses treat neuroscientific scholarship. Gruber (2012), for example, takes the first approach and describes the discourse differences related to how different fields treat a particular concept of neuroscience—the concept of “mirror neurons.” He observes that institutional dynamics at work within disciplinary scholarship limit the ability to arrive at a common language to describe the concept, further illustrating this problem. They also state that a

second approach might be the neuroscience of rhetoric, drawing new insights into language, persuasion, and communication from neuroscience research. Findings such as this study of noncommunicative patients can prompt us to broaden our very definitions of rhetoric to include those with impaired communication (such as autism, aphasia, or “locked-in syndrome”), asking how communication occurs through different means, or how brain differences might influence communication. (Jack & Appelbaum, 2010, p. 412)

I attempt to close some of the discourse disconnections Gruber (2012) and Calvert et al. (2004) identified while using the second approach to synthesize scholarship in multimodal rhetoric and neurobiology, particularly with respect to multisensory neural processes, explicitly in the discussion of cognitive neuroscience.

Gruber (2012) formulates four “pillars” by which interdisciplinary research involving rhetoric and neuroscience can occur by facilitating a means of “translation” between discourses. These pillars are very much a building tool applied in this book. The first pillar, he explains, is the “field-familiar spokesperson” (p. 237). This is a scholar who is knowledgeable about neuroscience and a second field—a sort of intermediary between discourses. I represent this person in the context of this book. The second pillar is that of the spokesperson’s support—a mechanism by which the spokesperson from the first pillar establishes ethos, or credibility, as well as logos for establishing the connection with the neuroscience community (p. 239). As I explained in the Author’s Preface, I have consulted with a neurobiologist to ascertain that I understood concepts of neurobiology that I present in this book and applied them correctly. This neurobiologist acts as the second pillar in the context of this book.

The third pillar is that of nature; Gruber (2012) indicates that nature connects neuroscience with the particular field being applied to it or vice versa. I have already alluded to Aristotle’s (1991) and Perleman and Olbrechts-Tyteca’s (1969) references to the links between rhetoric and biology. These and the scholarship in neurobiology that I cite contribute to establishing this pillar for this book. The last pillar Gruber identifies is that of “objective writing practice.” He explains that this is a practice that makes writing transparent rather than an exercise in creativity; it is an effort to represent an objective reality rather than corrupt reality.

The model that I propose and develop here rests on these pillars and is open to further construction. As scholars in rhetoric and other disciplines interact with this model, they act as additional field-familiar spokespersons, lending their credibility to the model’s development and applications. When two or more researchers from different fields join to study a given phenomenon, a synergistic effect occurs within the dynamic of those pillars to strengthen the model and allow for further development.

Cognitive Neuroscience and Rhetoric

The field of neuroscience has experienced a boom in scholarship that integrates several disciplines. Generally, this scholarship ranges across the five general disciplines that are connected with cognitive neuroscience: cognitive psychology, philosophy, linguistics, biology, and chemistry. Physics is also somewhat involved. Most of these are recognized as humanities-related areas, while the others are specifically connected to science—biology, chemistry, and physics. As mentioned above, each discipline theorizes neuroscience and cognition by applying its own research methods and theories to analysis and discussion. However, the disconnection across disciplines is problematic, especially as institutions attempt to find ways to connect disciplines with interdisciplinary programs and research projects. Cognition is associated closely with perception: How one perceives information affects their understanding of that information. The field of cognitive neuroscience devotes much attention to understanding how one processes information toward cognition. Humanities scholars tend to examine how language and social interactions affect our understanding of the world. Reid (2007) noted that “cognitive scientists termed the 1990s ‘the Decade of the Brain’ for the startling advances made throughout their discipline” (p. 14). Indeed, Hutchins (1995) and Pinker (1997) theorized cognition as a series of developmental processes that include historical dynamics as well as how people treat training and actual practice and the social dynamics therein. This has helped to generate subfields of distributed cognition and cognitive psychology as well as social semiotics. In each case, research in cognitive neuroscience has found that cognition is a multisensory process. Social interaction engages multiple senses—visual, aural, spatial orientation, and relationship—as well as gesture—touch and smell. Likewise, language is generally recognized as being aural/oral or visual, and print-linguistic text is a visual representation.

Science disciplines have been studying connections between perception, behavior, and neural dynamics. Available technology affects how this study occurs. Until recently, most of this involved looking at electrical activity within the brain. Neurons send electrical messages across the brain, and the different parts process that information toward doing something with it. However, recent technology has made it possible to look into other physical attributes of the brain and how the brain processes information related to perception and cognition. In particular, 2-photon microscopes and magnetic resonance imaging (MRI) technology facilitates such research. A 2-photon microscope permits the imaging of areas of the brain that are excited during tasks, suggesting neural activity. Some MRI technology allows researchers to see how blood flows to certain parts of the brain while one performs a particular task—viewing a given film or doing certain work, for example. This technology is called “functional MRI,” or “fMRI.” Biologists and chemists have begun examining the relationship between blood flow and neural processes. As humanities scholarship has done, many of these studies also link cognition to multisensory processes (e.g., see collecti...

Table of contents

• Cover
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Acknowledgment
• Chapter 1: A Gap to Bridge
• Chapter 2: Multimodality and Neurobiology
• Chapter 3: The Neurocognitive Model of Multimodal Rhetoric
• Chapter 4: Framing Cognition with Media
• Chapter 5: Historical Case Study: TWI Training Practices
• Chapter 6: New Media Applications: Slide Shows and Simulators
• Chapter 7: Instructional Machinima
• Chapter 8: Comparative Neurorhetorical Analyses
• Chapter 9: Research Implications
• Chapter 10: Conclusion
• Appendix A: Challenges in Historical Research of Multimodal Rhetoric and Cognition
• Appendix B: Interview Questions
• Appendix C: The Arsenal of Democracy Speech: December 29, 1940
• References
• Index