Multimodal Texts

11 Key excerpts on "Multimodal Texts"

• 

  eBook - ePub

  Literacy Theories for the Digital Age

  Social, Critical, Multimodal, Spatial, Material and Sensory Lenses

  • Kathy A. Mills(Author)
  • 2015(Publication Date)
  • Multilingual Matters

    (Publisher)

  4 Multimodal Literacies A multimodal approach to technology-mediated learning offers a way of thinking about the relationship between semiotic resources and people’s meaning making (Jewitt, 2006: 16) Key Concepts of Multimodal Literacies Multimodal approaches to literacy are currently prolific in educational research, indicated by the steady increase in the number of research studies of digitally mediated literacy practices (Mills, 2009). The popular terms ‘multimodal’ and ‘literacy’ now appear together in over 30,000 scholarly texts accessible to Google Scholar, though definitions of multimodality are not always alike. The dominant theory of multimodality addressed in this chapter is positioned within the theoretical framework of social semiotics (Kress & van Leeuwen, 2006). Social semiotics explicitly attends to meaning-making of diverse kinds, whether of words, actions, images, somatic meanings or other modes (Thibault, 1993). Therefore, by definition, social semiotics acknowledges the role of non-linguistic modes in human social meaning. Multimodality is defined as ‘... the use of several semiotic modes in the design of a semiotic product or event’ (Kress et al., 2001: 20). Reading and writing have always been multimodal, since these literate practices involve the decoding and encoding of words, while similarly attending to spatial layout of the text, images and other modes of representation (e.g. gestural meanings of represented characters, material features of the book). Yet undeniably, people-driven technological developments of communication have given rise to a much more diverse range of texts and textual practices

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Narrative Inquiry in Language Teaching and Learning Research

  • Gary Barkhuizen, Phil Benson, Alice Chik(Authors)
  • 2013(Publication Date)
  • Routledge

    (Publisher)

  The literacy narratives by Barry and Porter both included visual components to complement and enrich their written narrative texts. In language learning research, also, we have begun to see the use of multimodal narrative texts as alternatives or supplements to the more traditional written and spoken narratives covered in the previous two chapters. Multimodal Texts can be used as subject (e.g., Barry, 2002) or as tool (e.g., Porter, 2002). As subject, the multimodal text is meant to be the end product and is the main set of data for narrative analysis. This is similar to the collection of written data for analysis, with additional photographs or drawings or hyperlinks inserted. As tool, the multimodal text is used as an artifact during interviews, in which the author (usually a language learner) discusses language learning with reference to the multimodal text. Here, the multimodal text is a prop, and it may or may not be analyzed independently. Compared to oral and written elicitation, the number of narrative analysis studies adopting Multimodal Texts either as subject of investigation or tools for investigation is comparatively small. However, there is a growing number of studies which are based on multimodal narratives in second language education. In this chapter, we look at digital and non-digital Multimodal Texts as subjects of and tools for narrative inquiry in language teaching and learning.

  4.2 Contexts in which Multimodal Narrative Texts are Used

  This chapter is concerned with multimodal data collection in narrative inquiry. We adopt Kress’s definition of mode as that of “culturally and socially produced resources for representation” (2005: 6). Kress gives the examples of image, writing, music, layout, and others as different modes that people use to make and communicate meanings. Therefore, in this chapter, we consider narratives constructed by more than one mode as multimodal narratives; for instance, written narratives embedded with hyperlinks or photographs, or oral narratives supported or supplemented by photographs. In addition, “multimedia text” has been used by narrative researchers, a term used frequently in Computer-Assisted Language Learning research. Beatty (2010) defines multimedia texts as texts that include several media types; e.g., text, images, sound, or video. Multimedia text embraces both “the non-linear organization of text” and “non-linear and multiple information formats” (p. 45). Throughout the chapter, we will use both terms interchangeably. Definitions aside, our main focus is on the different ways narrative researchers appropriate multimodal/multimedia texts for data collection in narrative research.

  As in studies using oral and written data, multimodal data are produced and collected in different contexts. The following outlines in more detail the contexts in which such data are produced and collected. The studies chosen here are categorized according to two criteria: either the authors themselves used the term “multimodal” or “multimedia” to describe their data, or we deemed the data to be “multimodal” or “multimedia.” Studies that have been referenced as examples are discussed in more detail later in the chapter.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Designing Learning for Multimodal Literacy

  Teaching Viewing and Representing

  • Fei Victor Lim, Lydia Tan-Chia(Authors)
  • 2022(Publication Date)
  • Routledge

    (Publisher)

  3 Multimodal Literacy Learning with Print Texts

  DOI: 10.4324/9781003258513-3

  3.1 Why Teach Multimodal Literacy with Print Texts?

  As you read the lines on this page, you would notice that the meanings are not just expressed through the content of the words in the paragraphs. The heading of the chapter at the top of this page, the subheadings introducing new paragraphs, the numbering at the bottom of the page, and even the materiality of the book before you – be it physical or digital – offer additional information about the content of what you are reading and about to read.

  The texts that we encounter in our daily lives are always multimodal – expressing meanings not only with words but also how words are shown in terms of their fonts, size, and colour. Meanings of formality, mood, and style can be expressed through choices in these semiotic resources (van Leeuwen, 2010 ). Consider also how the placement of words on a page and their juxtaposition with other elements in a layout can express different meanings of prominence and importance (Kress & van Leeuwen, 2021). Language is inherently multimodal in its expressions, both with written words through choices in the visual system and speech through choices in the sound system (Matthiessen, 2007 ). Words are also often accompanied by other visual semiotic resources, such as images (photographs, drawings, computer artwork), figures, and emojis.

  Given the rich multimodal nature of the texts in our world, the sole focus on language in literacy learning and development is arguably inadequate in helping students engage with the full suite of communicative resources around them (Jewitt & Kress, 2003 ; Kress, 2003 ; Lim, 2021 ; New London Group, 1996 ). Indeed, as the US National Council of Teachers of English (NCTE) declares,

  literacy is more than just reading and writing … literacy is the way that we interact with the world around us, how we shape it and are shaped by it. It is how we communicate with others via reading and writing, but also by speaking, listening, and creating.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Literacy and Education

  • Kate Pahl, Jennifer Rowsell(Authors)
  • 2012(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  Chapter 5 , we talk about this as the notion of ‘sedimented identities in texts’, which is the idea that meaning makers, whether singly or collectively, sediment parts of themselves into texts (Rowsell & Pahl, 2007). In this chapter, we look at identity in terms of how learners use the principles and practices of multimodality when they make meaning with texts across a variety of contexts.

  MULTIMODALITY: AN OVERVIEW

  Multimodality is a field that takes account of how individuals make meaning with different kinds of modes. The shift from written to visual texts in contemporary culture means that a focus on semiotics, the study of signs, is important (Kress & Van Leeuwen, 1996). There has been so much dynamic research on multimodality over the past decade (Lancaster, 2003; Walsh, 2003; Siegal, 2006; Stein, 2007; Jewitt, 2009; Vasudevan, 2009; Wohlwend, 2009; Kress, 2010; Pahl & Rowsell, 2010). We also highlight Gunther Kress’s book, Before Writing. The book maps out a language and logic of multimodality entirely premised on how children make meaning in the world: through senses, gut instincts, improvisations, and fearless creativity. In his eloquent words, ‘we cannot understand how children find their way into print unless we understand the principles of their meaning making’ (Kress, 1997: xvii). In this chapter, we build on this work.

  Multimodal research unveils aspects of English and literacy that have been held back by a linguistic analysis. Take the research of Pippa Stein and her work with young children and their multimodal meaning-making and transformation of ideas into hand-made artifacts (Stein, 2007). Or the work of Kate Pahl (2004), who connects the sign to the sign-maker by tracing subjectivities embedded into texts. Or Carey Jewitt’s detailed work analyzing diagrams of scientific processes in science class (2009) as preferred ways of understanding material. Or Charmian Kenner’s (2004) interpretation of spatial organization and the framing of writing on the page, its directionality, the angle of its scripts, and the shape and size of script. These studies, individually and collectively, redress an over-emphasis in the past on the written word.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Multimodality in English Language Learning

  • Sophia Diamantopoulou, Sigrid Ørevik, Sophia Diamantopoulou, Sigrid Ørevik(Authors)
  • 2021(Publication Date)
  • Routledge

    (Publisher)

  340). The New London Group singled out five different types of design (Cazden et al., 1996): visual design (e.g. photographs, drawings), linguistic design (written words), audio design (music, sound effects, etc.), spatial design (e.g. architectonic meaning), and gestural design (e.g. gesture, behaviour, sensuality) in addition to subsuming the combination of modes by the term multimodal design. The result will be a multimodal text that is not necessarily an alternative way of conveying the ‘same meaning’ as in a more traditional verbal text: information is carried differently in visual and verbal mode and consequently a new code of writing and image combined is produced (Unsworth, 2008). 11.3 Options and Constraints in Students' Text Production 11.3.1 Considerations of Context L2 teaching since the mid-1970s has emphasised the interrelationship between language use and its context. On a general level, any choices related to language use and text production are the product of a complex system of options and constraints. The context and situation in which the multimodal text is constructed will allow for a set of choices, but properties of context will also act as constraining factors. What complicates this issue even more is that there is no such thing as ‘a direct context-text relationship’ (Weiss & Wodak, 2003, p. 95). Instead of talking about the social situation in which a particular text is produced, it makes sense to see the concept of social situation in terms of a mental model. Interpreted this way, ‘a context is not a social situation but a subjective mental model that participants construct of the relevant properties of the social situation’ (Weiss & Wodak, 2003, p. 95)

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Elaborating Multiliteracies through Multimodal Texts

  Changing Classroom Practices and Developing Teacher Pedagogies

  • Geoff Bull, Michèle Anstey(Authors)
  • 2018(Publication Date)
  • Routledge

    (Publisher)
• All texts are multimodal.
• Texts can be interactive, linear or non-linear.
• Texts may be intertextual.
• Texts are tending to become more screen-like as design and designing become more central to the production of texts.
• Texts can be created by the consumer using the links in digital texts to produce hypertexts.
• The social and cultural background of individuals influences the production of, and engagement with, text.
• A text may have several possible meanings.
• The consumer interacts with the text to actively construct the meaning of the text.
• The complexity of Multimodal Texts means that consumers have to consciously differentiate the focus of their attention across the semiotic systems.
• No text is neutral.
It is also useful to review historical approaches in terms of how they might inform a pedagogy that teaches the consumption and production of text as a process of design. Review Figure 6.2 , which has been previously presented and depicts the processes of producing and consuming text as a process of design.

Reflection Strategy 6.4

• The purpose of this Reflection Strategy is to review the historical approaches to the teaching of writing in terms of developing multiliterate individuals, developing a multiliterate pedagogy, developing concepts about Multimodal Texts, understanding the consumption and production of text as design.
• Draw up a table in which you can compare and contrast historical theories and approaches to the teaching of writing.
• Re-read the characteristics of a multiliterate person and understandings about multimodal text and use this knowledge to review the approaches in terms of the following general question: ‘What is necessary but not sufficient about this research and pedagogy for producing (designing and redesigning) texts in a multimodal, multiliterate world?’
• Use the following minor questions to examine the historical research and approaches in terms of the stages on Figure 6.2 . How does the research that informs this approach and related pedagogy and practice:
  • Inform understandings about the role of purpose, audience and meaning when producing a multimodal text?
  • Contribute knowledge, skills and processes that become part of a repertoire of resources that aid the production of Multimodal Texts?