Musical Sense-Making

Enaction, Experience, and Computation

Mark Reybrouck, Graham Welch, Adam Ockelford, Ian Cross

• Publisher: Routledge
• Year: 2020
• ISBN:
  9781000260878
• Edition: 1
• Topic: Media & Performing Arts
• Subtopic: Music

Musical Sense-Making: Enaction, Experience, and Computation broadens the scope of musical sense-making from a disembodied cognitivist approach to an experiential approach. Revolving around the definition of music as a temporal and sounding art, it argues for an interactional and experiential approach that brings together the richness of sensory experience and principles of cognitive economy.

Starting from the major distinction between in-time and outside-of-time processing of the sounds, this volume provides a conceptual and operational framework for dealing with sounds in a real-time listening situation, relying heavily on the theoretical groundings of ecology, cybernetics, and systems theory, and stressing the role of epistemic interactions with the sounds. These interactions are considered from different perspectives, bringing together insights from previous theoretical groundings and more recent empirical research. The author's findings are framed within the context of the broader field of enactive and embodied cognition, recent action and perception studies, and the emerging field of neurophenomenology and dynamical systems theory.

This volume will particularly appeal to scholars and researchers interested in the intersection between music, philosophy, and/or psychology.

1 Introduction

What is music? Can we conceive of it as an ontological category, as something “out there,” or should we think of it in epistemological terms, in the sense that music is only music when it is heard and listened to? Is music a mere concatenation of sounding events that impinge upon our body and our brain? Or does it call forth an active process of sense-making by the listener? The questions are challenging and have elicited discussions for decades, revolving about the distinction between music as a structure and music as a process. This dichotomy, however, is somewhat ill-defined as music, as a temporal art, is a dynamic structure that is evolving over time. Music, in this view, is not to be equated with a static structure or artifact, which seems to be petrified as something “out there” existing in a virtual space outside of the time of actual unfolding. It is possible to conceive of the sixth symphony by Bruckner or a Beatles song, but each actual performance or new rendering calls forth a sonorous articulation that is characterized by the idiosyncrasies and specifics of each particular performance. Does this invalidate the traditional preoccupation of much musicological research with a major focus on analysis of the score? Much depends here on the stance we take about “sounding” music. Listening in real time implies consumption of time, and making sense of sounding music involves real-time listening as well. But music is also an ephemeral art. Each sound disappears while it sounds, leaving only a trace in memory instead of actual sounding stimuli. As such, we can only have a summary or global overview of a musical work as a whole by recollecting the moment-to-moment history of all actual sounding now-moments in memory. There is, so to say, a dynamic tension between the actual auditory sensation which is very short-lived and the representation in memory and imagination, calling forth a dynamics of representation which reflects the transition from a stream of auditory stimuli to some kind of objectification, allowing us to think of a sounding flux in terms of different temporal representations, from real-time and fine-grained temporal windows to more encompassing global and synoptic overviews (Godøy, 1997a).

This global recollection is characterized by a kind of virtual simultaneity, which goes beyond the inexorability of the unfolding of time. We can, in fact, mentally recall some moments which are not contiguous in time and relate or compare them to each other. This is the plasticity and reversibility of mental operations, which is so typical of symbolic play: being to some extent disconnected from the actual sensations, we rely on mental replicas rather than on the sounds themselves (Reybrouck, 2006b, 2016a). Yet, it is possible to mentally navigate through these recollections in memory and to reconstruct the pathway of the successive sounding events. The immaterial and ephemeral character of these mental reconstructions, however, makes it difficult to experience them in depth. It can be helpful, therefore, to freeze these virtual impressions in time by using some lasting trace or record, such as a score or a computer-generated visualization (a waveform or spectrogram are typical examples). Such frozen visualizations have the advantage of presenting the music in a synoptic way, with the materials being presented all at once, but the approach is also limited, as it presents the “structure of the music” and not “our structuring” (see below). It can be questioned, therefore, to what extent we can conceive of a kind of representation that depicts the structure of the music in a dynamic way as well as the ongoing reactions and interactions that take place in our body and our brain in a real-time listening situation.

This book is an attempt to provide an answer to this question. It revolves around the mapping between the sonorous articulation of the music as it unfolds in time and the continuous and ongoing processes of sense-­making by the listener. These processes are conceived in their broadest sense, embracing not only “disembodied” rule-based or information-processing models of cognition, but also considering “enactive” and “embodied” models of the mind that emphasize the self-organizing aspects of cognition as an ongoing process of dynamic interactivity between an organism and its environment (Schiavio et al., 2017b). Musical sense-making, in this view, is not to be reduced to a conception of musical experience as a kind of abstract, decontextualized, and disembodied process as advocated by the cognitive approach to music listening and analysis. It should address, on the contrary, the actual lived experience of music, which involves more than internal cognitive processing and detached aesthetic appraisals (Maeder & Reybrouck, 2016). Music understanding involves embodied, pre-linguistic, and emotional-empathic forms of understanding, communication, and social cognition as well, as evidenced by the primordial interactions between infants and primary caregivers, somewhat related to the primary intersubjectivity and participatory sense-making that is necessary for developing social bonds through embodied-affective means (Trevarthen, 2002; De Jaegher & Di Paolo, 2007). As such, our body plays a central role, both explicitly and covertly, in shaping the way how we experience music in terms of embodied interactivity, as an ongoing history of organism–environment coupling that affords the enactment of meaningful worlds.

All this has consequences for the process of musical sense-making and the broader field of musical semiotics. Semiotics, which can be defined as the “science of signs,” is a rather young discipline. It has its own disciplinary history with an initial opposition between two traditions which were greatly independent of each other (Reybrouck, 2017a). There was, on one side, the “Anglo-Saxon tradition,” oriented mainly to the theoretical framework of Peirce, and the “continental tradition” (mostly Italian, French, and Slavic) with a principal orientation to the schools of de Saussure and Hjelmslev. Both positions, which were rather hostile to each other at first, have started to come nearer to each other as a result of the pragmatic turn in philosophy (Bernstein, 2010; Egginton & Sanbothe, 2004; Parret, 1983; Ralston, 2011; Rorty, 1982). This has been the case also for the domain of music (Cram, 2009; Kunst, 1978; Maeder & Reybrouck, 2015, 2016) with a growing role for the “experiential dimension” in the study of music in all its aspects. This is obvious from theories of externalization, which embrace issues such as embodiment, corporality, individual biographies of composers/performers/listeners, as well as other disciplines such as theories of performance, neurosciences, cognitive sciences, and other methodological approaches, which are based on the subjectivity of expression (Helbo, Bouko, & Verlinden, 2011).

Musical semiotics has seen a similar shift in scope. Starting from earlier studies, with the main emphasis on structural, phenomenological, and hermeneutical approaches, there has been a widening of the field by relying mainly on the contributions of Morris and Molino. Morris' division of semiotics in three dimensions, which he termed the syntactical, semantical, and pragmatical dimension of semiotics, are based on the relations between signs, their objects, and interpreters. To quote his words:

One may study the relations of signs to the objects to which the signs are applicable. These relations will be called the semantical dimension of semiosis. Or the subject of study may be the relations of signs to interpreters. This relation will be called the pragmatical dimension of semiosis […] and since all signs are potentially if not actually related to other signs, it is well to make a third dimension of semiosis co-ordinate with the other two which have been mentioned. This third dimension will be called the syntactical dimension of semiosis.

(Morris, 1975[1938], pp. 6–7)

Molino's levels of description, on the other hand, distinguish the esthesic, poietic, and neutral level of description, referring respectively to the process of creation (poietic), reception (esthesic), and the form and content (neutral) of the music (Molino, 1975; see also Nattiez, 1976). This tripartition, which has been contested also to some extent (see Agawu, 1991, p. 12, for an overview), has enabled semioticians to free themselves from certain constraints that were imposed by mere structural analyses which conceive of music as a closed system.

Much former music analysis has focused on the syntactic level of description, leaning heavily on the contributions from linguistics (see, e.g., Molino, 1975; Nattiez, 1976, 1990; Ruwet, 1975). Starting from a neutral level of description, an attempt was made to classify the sounds objectively and scientifically by selecting and identifying the classes of objects that can be arranged in terms of similarity and difference. Central in this “taxonomic-­empirical approach” were procedures of division and extraction of structural elements, with decoding strategies that work “from text to code” with structural units that are describable in a formal way. To quote Nattiez:

…[i]t is no longer a question of knowing whether one of the fragments […] is a motif or a cellule: it becomes an a, or A, or x, no matter which, possessing certain characteristics, which are defined by a group of features (melodic, rhythmic) which make it possible to compare it and classify it, that is to place it in a hierarchy with all the other segments of the piece. At the level of the metalanguage of the analysis, one can guess what the immediate tasks of musicology will be: to develop fully a formal, artificial, explicit language which can take into account all the units one can find in music and their combinations.

(Nattiez, 1973, p. 63)

There have been contributions to the level of musical semantics as well (Laske, 1973; Kühl, 2007; Reybrouck, 2008, 2013), with a distinction between musical meaning being defined as referring to something outside of the music (“external” or “real” semantics) or as referring merely to itself (“internal” or “self-referential” semantics) (see Cariani, 2001b for definitions and Koelsch, 2011 for empirical grounding). The conception of self-referential semantics, however, is somewhat ill-defined, as it conflates to some extent with the syntactic level. As such, it calls forth the syntactization of semantics as advocated already in the 1930s—the logical semantics of Carnap (Carnap, 1934a, b) and the model-theoretic semantics of Tarski (Tarski, 1956)—with an approach that is accomplished by completely encoding the world so that the elements (mostly formal symbols) are seen in relation to completely logical-­symbolic structures without the need of specifying any set of observables and without the need of verifying their truth-values concerning an outer world. In Saussurian terms, this should mean that “signifier” (i.e., the sign that signifies) and “signified” (the thing that is referred to) blend, and that musical signifieds are internal to the musical system, without any reference to something outside of the system. The signifieds, in this view, are not denotative or lexical but are self-­reflective (Imberty, 1979) which means that they refer mainly to themselves. What matters in this view is the identification of sonic events and their interrelations, without any relation to the external world. Music, then, is a carrier of immanent meaning, with sounding elements as recognizable entities that can be assigned some meaning or semantic weight. Unlike language where attention is directed away from the text to grasp the meaning outside of the written text—this is the “centrifugal” tendency of linguistic meaning—music is characterized by a “centripetal” tendency with a focus on the auditory material itself (Kyndrup, 2011).

The distinction between internal and external semantics, however, is not so radical as it may seem. Music as a sounding phenomenon relies on both, in the sense that elements that are referring to themselves may trigger processes of sense-making that refer to the external (the sounding environments) or internal world of the listener (bodily resonance). To the extent that we experience a particular sound as a real sounding thing that originates in the external environment, there is an aspect of external reference and external semantics. However, as soon as I start doing mental computations on this sound, there is a shift from presentational immediacy to cognitive mediation. We, then, no longer conceive of the sound in its experiential qualities but conceive of it at a symbolic level of representation, with processes of recognition and identification that replace the fullness and richness of an actual real-time experience. Music, in that case, is conceived in absentia and not in praesentia, to use de Saussure's terms (see Reybrouck, 2004). The reference to the internal environment of the listener, on the other hand, has received considerable impetus from the hard sciences, in particular from cognitive neuroscience and the neurobiological research with a special focus on the inductive power of music and its effects on the body and the brain (see Reybrouck & Eerola, 2017, for an overview). Stimuli, in this view, do not necessarily originate from the outer environmental world. They can have their origin in our proper body with all kinds of sensory or motor reactions to the sounds as well. This echoes somewhat the distinction between distal and proximal stimuli in perception (see Reybrouck, 2017c): the former correspond to what is considered an actual object or event in the environment; the latter are more narrowly defined as the patterns of energy impinging on the observer's sensory system. The energy is associated with the distal stimuli, but the observer depends most directly on proximal stimuli for perceiving the world. For certain perceptions, however, there is little distinction between the two. Touch is a typical example, as the distal stimulus that is responsible for the tactile sensation is created when the object that serves as a distal stimulus is in physical contact with the observer (Snyder, 2001). The distinction, however, needs further elaboration as proximal stimuli are situated mostly at the boundary (mostly the skin and special sense organs) between the inside and the outside of the body. Yet, there is also the visceral part of our body, together with our bones, muscles, and connective tissues which all can trigger reactions to the sounds in the sense that they are resonating to these sounds. This is, in fact, the province of “vibro-acoustic medicine” (Gerber, 2001; Schneck & Berger, 2010), which investigates the bodily and visceral reactions together with the kind of information processing that is tuned at monitoring the internal environment of our body. It seems that sound vibrations may be organized and targeted to arouse certain bodily functions to induce particular physiological responses.

This brings us to the pragmatical dimension of musical sense-making, which investigates the relations between sign vehicles—these are the material signs—and their users and the processes which are involved in the interpretation of the signs. Pragmatics, as originally conceived by its founder (Peirce, 1965a, 1965b), proceeds from effects to causes. As Peirce puts it in his typical cumbersome style:

Consider what effect that might conceivably have practical bearings you conceive the concepts of your conception to have. Then, your whole conception of those effects is the whole of your conception of the objects.

(Peirce, 1965b, p. 293)

Central in this definition is the major role of “conceiving” and “interpreting” by the sign user. Translated to the realm of music this should mean that we as listeners make sense of the music rather than the music forcing its structure on us (Reybrouck, 1999; Rösing, 1981). As such, we can conceive of an interaction between the sonorous articulation and our sense-making, not in a behavioristic way with normalized stimuli and standard responses, but rather in a neo-behavioristic way in the sense that our responses are not restricted but constrained. Meaning, in this view, is not to be defined in terms of ontological categories but in terms of dispositions to react to external stimuli. It includes the listener, or more in general, the music user—to use a generic term that encompasses all agents who deal with music in some way or another (see Laske, 1977; Reybrouck, 2005)—as a principal participant in the semiotic process, both at the level of reception, action, and mental processing and computation. As such, it calls forth dimensions that go beyond a mere object-centered, “esthesic,” or “poietic” approach, to use Molino's terms. The configuration of our body and our cognitive faculties determine not only our ways of listening but also the execution and creation of the music, which make it possible to understand and to live a musical experience.

As a discipline, however, musical pragmatics is still in continuous development (see Maeder & Reybrouck, 2015, 2016). Starting to some extent from the conceptual framework by Peirce and Morris, it has made considerable efforts to describe the music in a richer and more complex way. This is, even more, the case nowadays with multiple contributions that are borrowed avidly from other disciplines such as the cognitive sciences, psychology, neurosciences, and even philosophy and neuropragmatics (see Bertuccelli Papi, 2010; Brown, 2006; Huron, 2006; Nussbaum, 2007; Bambini, 2010; Bara & Tirassan, 2000; Stemmer, 2000). The bulk of music and emotion studies, as well as studies on the effects of music and its inductive power, are also likely to provide substantial empirical grounding for this approach (Koelsch, 2011, 2013; Juslin & Västfjäll, 2008). As such, it seems that the “pragmatic turn,” as initiated in philosophy and semiotics, could be complemented with an “affective turn” that defines musical meaning not only in terms of understanding but also in terms of bodily reactions to the music.

The division in syntactics, semantics, and pragmatics, finally, is interesting as a framework. It can be questioned, however, whether it is also applicable in the case of a real-time listening situation. Though useful as an analytical tool, it seems that a lived experience cannot be captured in terms of these distinct categories. The image that shows up, on the contrary, is that of a kind of blending of levels of sense-making with a lot of freedom given to each individual listener. Where the first level of contact with the sounding music may be a level of immersion in the sounds, it is possible also to take distance with respect to the sounding flux and to conceive of it at a level of abstraction in a more detached and disembodied way. Between these two extremes, there is a whole continuum that spans the whole spectrum of sensorimotor or epistemic interactions with the sounds, with syntactic, semantic, and pragmatic levels of sense-making popping up in mutually overlapping ways. It is important, therefore, not to impose processes of sense-making in a rather compelling way as has been done in much earlier music research, but to assess the processes of dealing with the sound in a rather “clinical” way. What do we hear and attend to when we listen to music in a real-time situation? What are the underlying mechanisms of our sense-making? How can we assess them and can the listeners articulate them also for themselves? This calls forth a “phenomenological” approach to musical sense-making that embraces the three semiotic levels, but which takes seriously also the first-person experience of the music. It can be questioned, further, whether we ca...

Table of contents

• Cover Page
• Series Page
• Title Page
• Copyright Page
• Dedication
• Contents
• List of illustrations
• Series editor’s preface
• Foreword
• 1 Introduction
• 2 Musical sense-making: enaction, experience, and computation
• 3 Sense-making and the enactive approach
• 4 Musical meaning: representational-computational versus 
dynamic-experiential approach
• 5 Experience and interaction: ecological, cybernetic, 
and embodied claims
• 6 From interaction to sense-making
• 7 Music and the extended computational approach
• 8 Perspectives and future epistemology: social cognition, 
dynamical systems theory, and neurophenomenology
• References
• Index