Composing Audiovisually

Perspectives on audiovisual practices and relationships

Louise Harris

• Publisher: Focal Press
• Year: 2021
• ISBN:
  9781000407365
• Edition: 1
• Topic: Computer Science
• Subtopic: Digital Media
• Index: Computer Science

What does the Coen Brothers' Barton Fink have in common with Norman McLaren's Synchromy? Or with audiovisual sculpture? Or contemporary music video? Composing Audiovisually interrogates how the relationship between the audiovisual media in these works, and our interaction with them, might allow us to develop mechanisms for talking about and understanding our experience of audiovisual media across a broad range of modes. Presenting close readings of audiovisual artefacts, conversations with artists, consideration of contemporary pedagogy and a detailed conceptual and theoretical framework that considers the nature of contemporary audiovisual experience, this book attempts to address gaps in our discourse on audiovisual modes, and offer possible starting points for future, genuinely transdisciplinary thinking in the field.

Section 1
Thinking audiovisually

This section deals with some of the relevant contextual background for this book. It introduces and dissects some relevant texts, and presents a brief survey of current thinking and approaches to audiovisual modes from a range of disciplinary perspectives. It also presents and summarises the thinking of contemporary artists, sound designers, directors and collaborators, through responses to an open-ended questionnaire and a series of semi-structured interviews.

However, before getting into any of these areas in detail it is important to lay out what the ethos of this book is and how it informs the chapters that follow. As I have discussed earlier in the introduction, the language we use here matters, and the use of the term ‘audiovisual’ is important in both clearly defining the work being addressed and describing the holism of the approach to addressing work involving both sounds and images. Indeed, what is being sought here are ways of interrogating and exploring our understanding of audiovisual experience without attempting to bracket particular works or practices into specific disciplinary boundaries or reductive terms. That is not to say that this cannot be a useful way of encountering audiovisual works, but for the purpose of this volume these aesthetic, conceptual, theoretical, disciplinary or simply genre-based categorizations’ will, where possible, be side-stepped, to allow consideration of a work and our response to it through other means.

As an example, an immediate question in approaching this volume might be “why use the term ‘audiovisual composition’ and not something like ‘visual music’?”. This is a question I have been asked on more than one occasion, and indeed my own audiovisual work has historically been described as visual music (though not, it must be said, by me). It is perhaps best to begin here by first addressing why I don’t use the term in describing my own work, before considering why the term ‘audiovisual composition’ is being utilised here rather than a more specific term such as visual music.

Having spent some considerable time in my career exploring the history and practices broadly associated with visual music, there is little doubt that much of the work I do both draws heavily from and is closely related to, in ethos and practical development, some visual music practices both contemporary and historical. However, the principal element I react against in the term itself is the use of the word ‘music’, as I do not really consider the auditory component of my work to be specifically musical.

This might seem like a peculiar acknowledgement for an academic working within music higher education, but in avoiding the term I am attempting to step around the baggage it brings with it. Whether intentional or not, describing a work as music involves evoking a large number of expectations and preconceptions, many of which I don’t feel are particularly useful in encountering my work. In describing my pieces as audiovisual compositions, I am attempting to define them as being works in which auditory and visual elements have been combined (usually in a temporal form), but beyond that I am attempting to remove any particular preconceptions associated with the terminology I use to define my work. Each of the strands of my audiovisual practice, and indeed each of the works I compose, is different – in how they are developed, the processes and materials they use, their compositional ethos, their structure, form, texture, timbre, colour, shape, rhythm, etc. – the one thing that unites them is that I describe them as being audiovisual compositions. This is the reason why I don’t use the term visual music, as I fundamentally don’t consider them to be so, and I don’t want audiences to approach them as such.

It might be useful here to reflect on an example I encountered in recent years whilst working on both conceptualising my own compositional ideas – specifically those related to algorithmic aleatoricism (Harris, 2020) – and simultaneously developing another (as yet unfinished) research project on the history of the single-shot music video. Exploring the work of OK Go, whose music videos often feature single-shot or apparently single-shot compositions, I became intrigued by the tale of Mario, the Echo Park gander (who the band dubbed ‘Orange Bill’) who inserted himself into the video for End Love and became quite a prominent feature. The video itself is a single-shot, 18-hour take which uses time-lapse techniques in both compression and expansion to allow the band to perform to the song in the park, including at one point spending the night in sleeping bags and awakening intermittently to lip sync. Mario was a gander familiar to regulars of Echo Park, who tended to prefer the company of people to that of other geese. Having initially been quite territorial with the band during the practice run for the shoot several days earlier, including (anecdotally) biting one of the band members, Mario eventually seemed to almost adopt the band and appears frequently throughout the single-shot take – including spending quite a lot of time sitting at the band’s feet or circling them as they perform their elongated dance moves with other park goers.

The intrusion of Mario felt, intuitively, somewhat familiar and sympathetic to the ways in which I had been seeking to structure my own audiovisual work through algorithmic aleatoricism. For me, this process was about both controlling and not controlling elements of my work – of allowing it to be defined in fundamentals – in this case, algorithmically – but having some of the fine details left to chance. The extraordinary control that OK Go have historically demonstrated in their music video work, often involving elaborate visual environments that are constructed or deconstructed as the works progress, finds resonance in the processes I utilise in writing algorithms to tightly control certain aspects of my work. The intrusion of Mario, then, in this video is the aleatoric element; though not specifically sought, as it was in my own work, this element of unpredictability fundamentally shaped and altered the trajectory of the work itself. Though the outcomes of the work and, indeed, the works themselves might be somewhat different, there were elements of the audiovisual construction that were nonetheless related and felt, intuitively, similar. This recognition of similarities in process resonates with my understanding of audiovisual modes more broadly; that the separation or categorisation of audiovisual works into specific genres or ‘types’ might be useful in some ways – in addressing, for example, the historical or technological contexts that facilitated them – but might also push us towards overlooking points of correspondence or commonality between audiovisual modes that could impact on our interpretation of them.

Fundamentally therefore, in this volume I am seeking to address works that involve the combination of auditory and visual elements across a range of practices and modes, yet not limited to being from within one particular designation such as visual music. In Section 3 – Analysing Audiovisually – I will further break down the broad heading of audiovisual composition into specific categories or modes in order to facilitate the close analytical reading of a number of examples, but these categories will be functional and descriptive of the mode of the work, i.e. live performance, installation, music video, rather than genre-based or relating to the work’s aesthetic or conceptual underpinnings. Within each of these categories it would, of course, be possible to further subdivide these modes into disciplinary boundaries, but that is beyond the remit and, bluntly, interest of this volume at the present moment.

Approaching this volume in this way may be, in itself, potentially problematic. In attempting such a broad and holistic approach to audiovisual work – in trying to speak across a very wide range of modes, practices and artists – I run the risk of revealing nothing meaningful about any of them; as a mentor of mine has often said, “when everything is important, nothing is”. Indeed, there are audiovisual works that might be difficult to account for, or more complex to encounter, under the frameworks that will be developed in each subsequent section of this book; narrative film and theatre, for example, and the relative weightings of their sensory components might pose problems that are insurmountable for this particular volume, and the expansive nature of existing discourse relating to their audiovisual experience might render an attempt to add anything to the discourse somewhat fruitless. Nonetheless, what is perhaps most important here is the endeavour, an attempt to reconsider how we encounter audiovisual work outside of disciplinary, aesthetic, conceptual, theoretical, historical or modal boundaries which, even if not successful in this volume, might ultimately provoke the kind of discourse that will allow us to continue this conversation in fruitful directions.

1 Discourse on audiovisual experience

As I have described in the introduction, the impetus for writing this volume comes from a number of areas, one of which is the relatively limited discourse in the field of audiovisual practice outside of specific disciplinary boundaries. The following chapter will provide a brief summary and consideration of some relevant discourse, both positioned from the perspective of addressing ‘audiovisual practices’, broadly defined, and from discipline-specific perspectives that might have a bearing on how we encounter audiovisual work. It will consider a number of the ‘key texts’ exposed through the questionnaire, specifically those by authors who were cited by numerous of the questionnaire respondents. It will also consider branches of philosophical, phenomenological and perceptual theoretical thinking that might be relevant to this volume, or which are often referenced by authors writing on audiovisual practice when addressing either their own practice or that of others.

Texts that attempt to deal with the multisensory nature of audiovisual experience within the context of various art forms have become increasingly prevalent, particularly in the last 10–20 years or so, as the term has gained currency within arts practices and academic settings. Whilst the majority of these are still situated within particular disciplinary brackets (the recent ‘Sound Image’ collected edition (Knight-Hill, 2020b) and, indeed, this publication will appear as part of Routledge’s ‘Sound Design’ series, for example), I will begin with the open access publication, The Audiovisual Breakthrough, which both through the way in which the text is constructed and the fact of its open access ethos is one text that has effectively spoken across disciplinary boundaries.

Perhaps the most succinct way of explaining the central ethos of the text is to directly quote the introduction to the book from its host website:

Visual music, expanded cinema, live cinema, VJing, live audiovisual performance—these are concepts enough to create some confusion in the wide realm of today’s artistic audiovisual production. While each of these concepts is widely propagated and suggestive of its own line of history and shared practices, they are not as yet sufficiently defined for theoretical debate and clear practical use. To untangle this confusion was the aim of our project, The Audiovisual Breakthrough, which brought together six international researchers to solve the problem in a collaborative effort. This book is the result. Its main purpose is to make the entangled complexities of the field manageable by putting forth and elaborating on definitions for the five main concepts named above. The theoretical texts are complemented by the design of the book, a graphic take on audiovisuality that includes visualizations of the results from an international survey among the practitioners in the field of audiovisual (art) practices. This book is addressed to artists, curators, researchers, students, and teachers within these practices, and to anyone working at their intersection with other fields of knowledge.

(Carvalho and Lund, 2015, para. 1)

This succinct exposition is a neat encapsulation of what the book does, drawing from a range of authors from different backgrounds and fields of expertise to offer both definitions of the terms visual music, expanded cinema, live cinema, VJing and live audiov...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• List of illustrations
• Introduction
• Section 1 Thinking audiovisually
• Section 2 Composing audiovisually
• Section 3 Analysing audiovisually
• Reflections
• Epilogue – final reflections
• Bibliography
• Index