Technology, Literacy, Learning

A Multimodal Approach

Carey Jewitt

• Publisher: Routledge
• Year: 2012
• ISBN:
  9781134282623
• Edition: 1
• Topic: Pedagogía
• Subtopic: Educación general

With the recent explosion of technology into the world of education across the globe, this book sets out a framework for rethinking the three key areas of schooling that are most affected by technology's impact on education today: knowledge as curriculum; learning and pedagogy and literacy across the curriculum. A well-known author in this field, Jewitt takes the reader through an analysis of teaching and learning with materials such as CD-ROMs, websites, the Internet, computer programming applications and computer games, relating each in turn to the main curriculum topics.

Through this detailed scrutiny the following questions emerge:

• How do the new technologies reshape knowledge as curriculum?
• How does the use of new technologies in the classroom reshape learning pedagogy?
• As writing moves from page to screen, what is the impact on students' situated literacy practices and how does it effect learning?

Through these questions, this book demonstrates that mode, technology and curriculum knowledge are fundamentally connected and describes how teacher and student roles in the classroom could be altered in the face of new technologies.

Chapter 1

Introduction

Through this book I explore a fundamental question that I think remains unanswered, that is ‘what real difference can the use of new technology make for learning?’. My first and central aim in the book is to provide a better understanding of the relationship between the multimodal meaning-making resources that new technologies make available (image, sound-effect, movement and so on) on the one hand and school knowledge, learning and notions of literacy on the other.

A fundamental question

After all the research that has been conducted into new technologies and learning the most positive case scenario is that some new technologies help some people learn in some instances. Nonetheless, the governments of ‘developed’ and ‘developing’ countries around the world direct educational policies and huge amounts of funding at ‘technologizing’ the school classroom (Harrison et al., 2001; Somekh et al., 2001a). There seems to be an overwhelming belief or hope that new technologies will change learning. And change it for the better. This is reflected in the technological wave of hardware and promises that have been instantiated in educational policy and funding in many countries since the early 1990s.

The difficulties of introducing new technologies into schools are well known. These have been written about for years. But global investment in new technologies does not appear to have been restrained by this knowledge. Schools and teachers across the globe are under increasing pressure to incorporate new technologies into teaching programmes – a demand that is made at all levels of education. The failure to incorporate technology into the school effectively has rarely been attributed to the design of technologies. Rather it has been seen as a failure of government policy, inadequate funding, lack of educational vision, and or the apathy and resistance of teachers.

Against this complex backdrop of optimism and failure it is fair to say that the benefit of new technologies in the classroom remain vague — especially for many teachers.

New technologies make a wide repertoire of representational and communicational modes available. These include image, colour, movement (often in the form of animation), music, sound-effect, speech and writing. Alongside this the potential structure and interactivity of digital media provide new possibilities for interaction. In other words, new technologies offer different potentials for learning. I want to suggest these different or ‘new’ potentials require a re-thinking of what it means to learn.

The almost habitual conjunction of ‘language’, speech and writing, with learning is I think especially paradoxical in relation to technology-mediated learning. Here speech and writing are clearly a part and often a small part of a multimodal ensemble. Educational research has tended to focus on language. In the case of research on technology and learning, it has primarily looked at spoken interaction around the technology (Littleton and Light, 1999; Mercer et al., 1999). I think this has led to the development of methods of data collection and analysis that ‘strip away’ the multimodal features of contexts and practices. The question of how multimodal representational and communicational resources shape and reshape education does not arise or is not foregrounded. This is the case regardless of the technology used. It also masks the contradictory desires and demands embedded in educational policy and practice on technology and other aspects of education. In the context of Britain, for example, there is a ‘conflict’ between policies on technology and literacy. Policy on new technology aims to produce a population that is ‘technologically literate’. Literacy policies attempt to maintain a traditional notion of literacy rooted in language and writing, and the social dispositions that go with that.

In this book I set out a framework for re-thinking learning from a multimodal perspective in order to explore what real difference the use of new technology can make for learning. This framework, the central ideas and concepts that I use and more generally how I think about learning and technology are described in Chapters 2 and 3. This framework is described and elaborated on through its application to illustrative examples of technology-mediated learning in Chapters 4 through to 7. This includes learning in school English, mathematics and science (CD-ROM, websites, computer programming applications and so on). I address four central questions in the book. My first question is how new technologies reshape knowledge as curriculum. What might it mean for learning to represent a curriculum concept visually as opposed to in writing, for example. The second question concerns what it means for learning when the resources available to students change. How does the use of new technologies in the classroom re-shape learning? The relationship between mode and practices inform my third question although this time my focus is on writing and literacy. What is the impact of the changing role of writing on screen on students’ situated literacy practices? The final question that runs through the book asks ‘how does the use of new technologies in the classroom impact on teaching?.

A framework for thinking

In order to answer these questions, I combine multimodality (Kress and van Leeuwen, 2001) and activity theory (Engestrom, 1987; Daniels, 2001). Multimodality allows me to focus on all the different resources that are displayed on the computer screen and are part of classroom interaction. Activity theory offers a useful framework for situating people’s semiotic choices and use of technologies within the context of a curriculum subject and the classroom. I bring these two ways of thinking together to help ‘locate’ people’s use of representational and communicative modes in the complex social interaction of the classroom.

Multimodality

Multimodality in which all modes of communication are attended to as part of meaning making is founded on Halliday’s social semiotic theory of communication (Halliday, 1978, 1985). The main idea that underpins social semiotics is that language is social. Language is the way it is because of the social functions it has come to serve in people’s lives (Halliday, 1978). So rather than seeing language as a ready-made code, social semiotics understands language as a result of people’s constant social and cultural work. Through this ‘work’ the material of sound in the case of speech is shaped into a set of resources for representation. The resources of language are understood as being both material and socially and culturally shaped. These resources are ‘signifier-material’ for making meaning (making of signs).

Multimodal social semiotics extends Halliday’s theory of meaning beyond language to understand meaning as realised in a range of modes. It starts from the position that all modes have been shaped through their social use into semiotic resources. (Although some modes are more fully articulated than others.) Multimodal theory develops social semiotics to show that all modes have been developed as sets of resources for making signs. This includes music, speech and sound (van Leeuwen, 1999) and action (Martinec, 2000), as well as visual communication (O’Toole 1994; Kress and van Leeuwen, 1996). Multimodal theory has also focused on the ways in which modes are combined and designed to make meaning (Hodge and Kress, 1988; Martin and Veel, 1998; Baldry, 2001; Kress and van Leeuwen, 2001; Kress et al., 2001).

There is a significant body of work on multimodality that addresses the theoretical questions (Kress and van Leeuwen, 1996, 2001; Finnegan, 2002) including several books on methodology and tools for multimodal research (Baldry and Thibault, 2004; Norris, 2004). Having said that, multimodality is an emergent theory and many of the theoretical concepts and tools are still unsettled.

Although multimodality is underdeveloped as a theory, educational researchers have actively taken it up. Multimodality has been applied to a variety of learning contexts. This includes pre-school learning (Pahl, 1999; Lancaster, 2001), bilingual language learners (Kenner, 2000, 2003, 2004), literacy practices (Street, 1998; Moss, 2001; Unsworth, 2001; Gee, 2004), English and media education (Burn and Parker, 2001, 2003a; Kress et al., 2004), learning with video games (Gee, 2003) and science and maths education (Lemke, 1998a; O‘Halloran, 1998; Kress et al., 2001).

One of my aims in this book is to contribute to the elaboration of multimodality and its use to new technology and learning. Specifically I want to provide a better understanding of the multimodal resources offered by computer applications and how they impact on what can be ‘done’ with them. I analyse the design and use of the representational and communicative modes that technology-mediated learning makes available. I show how these multimodal resources reshape curriculum, learning and pedagogy as compared with the static ‘monotone print’ of the page. I explore the fundamental connection between mode, technology and curriculum knowledge. In particular, I examine how the design and use of image, sound, writing and movement on screen contributes to the construction of school knowledge and potentials for learning.

Activity theory

Throughout this book I am primarily using activity theory to think about meaning making in the classroom and to link signs to the broader issue of mind and learning. Activity theory builds on the cultural historical approach to learning of Vygotsky (1981, 1986). The focus is on how people’s consciousness is formed through their activity with tools. Through their activity with shared tools people are brought in contact with and connected to the history, values and social relations of society. Activity theory represents the social relations that underpin semiotic mediation as an activity system. An activity system is a group of people who share a common object and purpose over time and who share tools to act on and realise their purpose. The framework of an activity system provides useful tools for thinking about learning in the classroom: the norms, rules, community and roles, and how a change in tool impacts on learning. I use activity theory to draw in social, historical cultural elements beyond the immediately observable classroom interaction. This includes among other things the teacher’s purposes in the lesson, the focus and demands of the curriculum and the traditions and histories of school subjects.

A focus on schools

The ideas and questions that inform this book emerged from my work as a researcher. In particular, my research with Kress and colleagues on the multimodal production of school science (Kress et al., 2001) and school English (Kress et al., 2004). This research showed how a range of modes contribute to the construction of curriculum knowledge, learning and pedagogic practices. New technologies did not feature in the classrooms that I observed during this research. The focus was on the ‘old’ technologies of the classroom: pen and paper, blackboard and chalk (sometimes whiteboard and pen), wobbly crystal models, Bunsen burners, clamps and beakers.

Alongside my research work I watched the children of friends and my young daughter ‘play’ and ‘work’ on computers (game boys and playstations). I watched them ‘read’ books, ‘play’ music, write, draw, build and destroy cities, and kill armies. I listened to the sound–effects of the software, watched the visual display of animated (revolving) icons and characters, the use of colour on screen, the spatial construction of the screen environment, and all the time I wondered what was going on.

As these children sat at the computer, they looked intently at the screen, moved the mouse, clicked on icons, pressed keys and pointed at things on the screen (or jumped around waving madly– but purposively – if playing ‘Eye Toy’ games). I could use multimodality to think about my daughter’s drawings and writing on paper, the books we read together, the three-dimensional objects that she makes from a variety of materials – beads, boxes, clay, cardboard, her songs and ‘performances’. I wondered whether the multimodal theory of meaning making that I could apply to my daughter’s other kinds of ‘work’ could be of any use in understanding this ‘world’ on the computer screen. So to some extent the focus of this book originated from my interest in children’s computer use in the home. In a sense, my interest and experience of the school led me to think about and focus on students’ use of new technologies in the school. During seven years of classroom-based research I had only seen computers stuck in the corner of a classroom, usually with the screen saver frantically beaming stars like an ironic sign of travel to the outer limits.

Despite the difference between the use of computers at home and school, when students use new technologies in the school or the home they are engaged with a range of representational and communicative modes. This makes both the home and the school equally suitable sites for asking the kind of questions I am interested in and to develop multimodal theory. In some ways school provides a more appropriate context to examine the relationship between mode and learning.

The use of any technology is shaped by the social context of its use, and children’s use of new technologies in the home and the school differs. At the most ‘basic’ level, children spend three times as long on computers in the home as they do in school on average (Harrison et al., 2001). When at home they are usually engaged in more challenging and innovative uses of new technologies than they are in the school. The kind of technologies and applications that are usually available at home and school are also different. Technologies are also used differently (with different intentions) to do different things (Facer et al., 2001; Harrison et al., 2001; Somekh et al., 2001b). Much of what children and young people get to do with new technologies at school can be summarised as very mundane. This is the case even in schools that are held up as beacons of excellence with a high standard of equipment and technical support. New technology in primary and secondary schools is most often used to produce ‘smart-looking’ word processed versions of writing – often for display (Harrison et al., 2001). Rather depressingly there is plenty of evidence that schools continue to find it problematic to incorporate new technology into the curriculum. Government reports indicate that Information Technology is the weakest national curriculum subject. In contrast, some children and young people are doing some interesting things with new technologies in the home and informal sites of learning.

Governments have implemented a variety of initiatives to technologise education since the late 1970s onwards. The British government has provided major funding for technology hardware, training for teachers and the National Grid for Learning (launched in 1998 and due to continue through to 2004). The government target for all British schools to be connected has been all but realised; by the end of 2001 nearly all secondary schools, primary schools and special schools were connected to the Internet (Somekh, 2001a). New technologies are seen by governments as a primary way to raise standards and to offer more inclusive education. In Britain, for instance, new technologies are considered to have a direct effect on improving student attainment in key curriculum areas (Pittard et al., 2003). Despite this, politicians’ hopes that technology would transform education have not been realised. Beyond the accessing of ‘information’ there does not seem to be a real sense of how new technologies transform (or might transform) schooled knowledge and learning.

This bland backdrop is the ‘technological revolution’ as it is realised for most students. Perhaps understandably, many researchers interested in new technologies have evacuated the school in search of the real revolution. Educational research on young people’s use of new technologies often takes place in arts, media and youth centres with specialist technology provision, including game making, digital video production and Web production. Research is increasingly focused on game playing and learning in the home. The ‘creative’ potential of games and computer applications used in the home have been the focus of much cultural and media studies research (Lachs, 1999, 2000; Buckingham, 2000; Sefton-Green and Parker, 2000). The ways in which young people’s engagement with new technology can give expression to young people’s self-identities (Turk...

Table of contents

• Cover
• Halftitle
• Title
• Copyright
• Dedication
• Contents
• List of figures
• List of tables
• Preface
• Acknowledgements
• 1 Introduction
• 2 Turning a multimodal lens on technology-mediated learning
• 3 Towards a multimodal analysis
• 4 Reshaping of curriculum knowledge
• 5 Multimodal learning
• 6 Multimodality, ‘reading’ and literacy
• 7 Pedagogy as design
• 8 Some final thoughts on representational and communicational resources, technology and learning
• References
• Index