Signal Processing in Auditory Neuroscience

Temporal and Spatial Features of Sound and Speech

Yoichi Ando

• Publisher: Academic Press
• Year: 2018
• ISBN:
  9780128159392
• Topic: Biological Sciences
• Subtopic: Neuroscience
• Index: Biological Sciences

Signal Processing in Auditory Neuroscience: Temporal and Spatial Features of Sound and Speech discusses how the physical attributes of different sounds manifest in neural signals and how to tease-apart their different influences. It includes EEG/MEG as additional variables to be considered when studying neural mechanisms of auditory processing in general, specifically in speech.

• Focuses on signal processing in human auditory-neuroscience
• Contains information that will be useful to researchers using a MEG/EEG recording of brain activity to study neural mechanisms of auditory processing and speech
• Gives an important overview and methodological background for techniques that are useful in human auditory-neuroscience

Chapter 1

Introduction

When designing enclosures (or any environment) for spoken communication, the acoustic properties of the sound field should be taken into account. This volume describes the human hearing system and the possible auditory mechanisms responsible for the rise of subjective preference. Because subjective preference is a primitive response that steers the judgment and behavior of the organism in the direction of maintaining life, we investigated corresponding cerebral activity in the slow vertex response (SVR) and brain waves in electroencephalogram (EEG) and magnetoencephalography (MEG). The results suggest an auditory signal-processing model that yields primary percepts and a theory of subjective preference for the sound field.^1-4 The temporal and spatial primary percepts may be well described by temporal factors extracted from the running autocorrelation function (ACF) and spatial factors extracted from the interaural cross-correlation function (IACF), respectively. These mechanisms are the bases for automatic speech recognition and the design of hearing aids in the context of this volume.

Auditory Temporal and Spatial Factors

For the past five decades, we have pursued a theory of architectural acoustics based on acoustics and auditory theory. In the summer of 1971, I visited the III Physics Institute at the University of Goettingen, where Manfred R. Schroeder encouraged me to investigate the aspects of spatial hearing that are most relevant to the design of concert halls. Peter Damaske and I were, at that time, interested in explaining the subjective diffuseness of sound fields using the IACF. The maximum magnitude interaural cross-correlation (IACC) of this function is an indication of the level of subjective diffuseness of a given sound field perceived by an individual due to binaural effects. We reproduced sounds in a room using a multiple-loudspeaker reproduction system and recorded the signals at the two ears of a dummy head.⁵ Because the IACC was known to be an important determinant in the horizontal localization of sounds, we also believed it to be significant in subjective perception of diffuseness. Two years later, in 1974, a comparative study of European concert halls performed by Schroeder, Gottlob, and Siebrasse, showed that the IACC was the most important factor in the incipient subjective preference reactions that established a consensus among individuals.

In early 1975 at Kobe University, we observed a superior sound field with a speech signal that was achieved when adjusting the horizontal direction and the delay time of a single reflection. A loudspeaker in front of a single listener reproduced the direct sound. The angle of the single reflection was about 30 degrees in the horizontal plane measured from the front, the delay time was about 20 ms, and the amplitude was the same as that of the direct sound.⁶ These working hypotheses were reconfirmed in the fall of 1975 while the author was an Alexander-von-Humboldt Fellow in Goettingen.^7,8 We were also able to explain the perception of coloration produced by the single reflection in terms of the envelope of the ACF of the source signal.⁹

In 1983, a method of calculating subjective preference at each seat in a concert hall was described by four orthogonal factors of the sound field.¹⁰ Soon after, a concert hall design theory was formulated based on a model of the auditory system. We assumed that some aspects depended on the auditory periphery (the ear), while others depended on processing in the auditory central nervous system.¹¹ The model takes into account both temporal factors and spatial factors that determine the subjective preference for sound fields.¹ The model consists of a monaural ACF mechanism and an IACF mechanism for binaural processing. These two representations are used to describe monaural temporal and binaural spatial hearing operations that we presume to be taking place at several stations in the auditory pathway, from the auditory brainstem to the hemispheres of the cerebral cortex.

Special attention was given to computing optimal individual preferences by adjusting the weighting coefficients of four orthogonal factors (two temporal factors and two spatial factors), which were used to determine the most preferred seating position for each individual in the room.² “Subjective preference” is important to us for philosophical and aesthetic reasons as well as for practical, architectural acoustics reasons. We consider preference as the primitive response of a living creature that directs its judgment and behavior in the pursuit of maintaining life—of body, of mind, and of personality.¹² Thus, neural evidence obtained could be used to identify the auditory system’s signal-processing model.

Correlation Model for Temporal and Spatial Information Processing

To develop a theory of temporal and spatial hearing for room acoustics that is grounded in the human auditory system, we attempted to learn how sounds are represented and processed in the cochlea, the auditory nerve, and in the two cerebral hemispheres. Once effective models of auditory processing are developed, designs for concert halls can proceed in a straightforward fashion, according to guidelines derived from the model.¹ In addition, understanding the basic operations of the auditory system may lead to a new generation of automatic systems for recognizing speech,⁴ analyzing music,² and identifying environmental noise and its subjective effects.¹³ In more general terms, the first book on a brain-grounded theory of temporal and spatial design in architecture and the environment was published in 2016.¹⁴

It is remarkable that the temporal discharge patterns of neurons at the level of the auditory nerve and brainstem include sufficient information to effectively represent the ACF of an acoustic stimulus. Mechanisms for the neural analysis of interaural time differences through neural temporal cross-correlation operations and for analysis of stimulus periodicities through neural temporal autocorrelations were proposed over half a century ago.^15-17 Since then, many electrophysiological studies based on single neurons and neural populations have more clearly elucidated the neuronal basis for these operations. Binaural cross-correlations are computed by axonal tapped delay transmission lines that feed into neurons in the medial superior nucleus of the auditory brainstem and act as coincidence detectors.¹⁸ If one examines the temporal patterning of discharges in the auditory nerve,¹⁹ one immediately sees the basis for a robust time-domain representation of the acoustic stimulus. Here, the stimulus autocorrelation is represented directly in the interspike interval distribution of the entire population of auditory nerve fibers.^20,21 This autocorrelation-like neural representation subserves the perception of pitch and tonal quality (aspects of timbre based on spectral contour).^22,23

In our laboratory, we have found neural correlates of spatial hearing in the left and right auditory brainstem responses (ABRs). Here, the maximum neural activity (wave V) corresponds to IACC, that is, the magnitude of the IACF.²⁴ Also, wave IV for the left and right side brainstem responses (IV_l,r ) nearly corresponds to the sound energies at the right- and left-ear entrances. SVRs are averaged auditory-evoked responses computed from scalp EEG signals. We carried out a series of experiments aimed at developing correlations between brain activity, measurable with the SVR and the EEG, and subjective sound field preference. Subjective sound field preference is well described by four orthogonal acoustic factors, two temporal and two spatial. The two temporal factors are (1) the initial time delay gap between the direct sound and the first reflection (Δt₁), and (2) the subsequent reverberation time (T_sub). The two spatial factors are (1) the listening level and (2) the maximum magnitude of the IACF (IACC). The SVR- and EEG-based neural correlates of the two temporal factors are associated with the left hemisphere, whereas the two spatial factors are associated with the right hemisphere.

Higher in the auditory pathway, we reconfirmed by MEG that the left cerebral hemisphere is associated with the delay time of the first reflection Δt₁. We also found that the duration of coherent alpha wave activity (effective duration of the ACF of the MEG response signal) directly corresponds to how much a given stimulus is preferred, that is, the scale value of individual subjective preference.^25,26 The right cerebral hemisphere was activated by the typical spatial factor, that is, the magnitude of IACC.²⁷ The information corresponding to subjective preference of sound fields was found in the effective duration of the ACF of the alpha waves of both EEG and MEG recordings. The repetitive feature of the alpha wave as measured in its ACF was observed at the preferred condition. This evidence can be pragmatized by applying the basic theory of subjective preference to music and speech signals for each individual’s preference.²

We also investigated temporal aspects of sensation, such as pitch or the mis...

Table of contents

• Cover image
• Title page
• Table of Contents
• IFC
• Copyright
• Acknowledgments
• Preface
• Foreword
• Chapter 1. Introduction
• Chapter 2. Human Hearing System
• Chapter 3. Analysis of Sound Signals
• Chapter 4. Formulation and Simulation of Sound Fields in an Enclosure
• Chapter 5. Magnetoencephalographic Evoked Responses to Factors Extracted from the Autocorrelation Function (ACF)/the Inter-Aural Cross-Correlation Function (IACF) Factors
• Chapter 6. Neural Evidences Related to Subjective Preference
• Chapter 7. Temporal and Spatial Primary Percepts of Sound and the Sound Field
• Chapter 8. Subjective Preference of the Sound Field
• Chapter 9. Temporal and Spatial Features of Speech Signals
• Index