Higher Level Language Processes in the Brain

Inference and Comprehension Processes

Franz Schmalhofer, Charles A. Perfetti

• Publisher: Psychology Press
• Year: 2007
• ISBN:
  9781135605650
• Edition: 1
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

Higher Level Language Processes in the Brain is a groundbreaking book that explains how behavior research, computational models, and brain imaging results can be unified in the study of human comprehension. The volume illustrates the most comprehensive and newest findings on the topic. Each section of the book nurtures the theoretical and practical

1

Editors’ Introduction: Mind and Brain in Higher Level Comprehension

Charles A. Perfetti
University of Pittsburgh

Franz Schmalhofer
University of Osnabrück

The study of text comprehension has developed from its fundamental but highly general question-how do people understand what they read?-to include more refined questions about the components and mechanisms of comprehension. What are the conditions that enable inferences and related processes that involve going beyond the literal text information? How do readers immediately link the meaning of a word in relation to its context? How does understanding proceed from a text-propositional level to a situation model level? (Or does it?) Is comprehension grounded in biological experience? What kinds of explicit models can account for comprehension processes?

Such questions have become compelling partly as a consequence of research progress in comprehension (and cognition generally) and partly because more methods have been brought to bear on comprehension. Computational modeling and neuroimaging methods have emerged to complement experimental behavioral methods (and these methods have expanded themselves) in the study of comprehension. Together, these three general methods have the potential to provide converging evidence for the components of comprehension and their mechanisms. This volume is a reflection of how this potential is being realized. In bringing together this small sample of work, we aimed both to assess the state of the art and to illustrate the kinds of new studies that are advancing the study of comprehension.

States of the Arts

The first section is three state-of-the-art chapters. Singer and Leon review the state of the art in experimental behavioral studies. Their review leads them to conclude that behavioral-empirical studies have led to much progress in a relatively short time, while also exposing controversies that are resistant to simple resolution. One of their more intriguing conclusions is that text and discourse research has paved the way for broader generalizations concerning the principles of cognition. Singer and Leon establish a strong foundation for the recurring themes of this volume, including the nature of situation models, propositional representations, conditions on inference making, perceptual symbols, and limitations of various methods of discourse study.

Goldman, Golden, and Van den Broek review the state of the art in computational modeling, detailing its usefulness for explaining comprehension processes. As they define it, “a model refers specifically to a representation of the psychological processes that comprise a component or set of components involved in human text comprehension.” A computational model is then a representation that is expressed in forms that can be run on a computer. They explain the value of computational modeling for developing theories of text comprehension, for testing explanations and accounting for unexpected findings, and for supporting communication within the comprehension field and with other areas. In making the case for these values, Goldman et al. provide some interesting comparisons of the some of the best-known models of comprehension.

Ferstl reviews the state of the art in neuroimaging research on comprehension. Keeping with integrative focus of the volume, the chapter does not merely review results of neuroimaging studies, but rather establishes a general framework for comprehension based on the class of models proposed by Walter Kintsch. The chapter provides a relatively nontechnical review of brain imaging studies that can be related to this framework and establishes results that are relevant for the work of later chapters on neuroimaging studies of integration, inferences, and cerebral functions. Ferstl concludes that the main value of neuroimaging is a better understanding of how cognitive processes work together to produce comprehension. She concludes that for this to happen, research will move from identifying single brain regions to discovering the contributions of components of a discourse comprehension network.

Computational Models

The second section presents two chapters on computation models. Dennis and Kintsch demonstrate the application of string theory to inference rule generation for simple texts. This application, the Syntagmatic Paradigmatic (SP) model, is based on simple comparison processes across unstructured (no syntax) strings of words stored in memory through experience. The SP model is intended to capture the relations that are usually represented in propositions, but without extracting explicit relationships. It provides a new inference mechanism, inference by coincidence, which allows inference-like retrieval process based on a corpus of similar strings. Dennis and Kintsch point out limitations of SP as a general language model.

Frank, Koppen, Noordman, and Vonk present a model of multiple levels of text representation, the Distributed Situation Space (DSS) model. The DSS creates situation models without first extracting propositions and allows the emergence of inferences from knowledge applied to story statements. The chapter extends the model with a simple recurrent network to simulate word-by-word reading. Multiple levels of discourse representation automatically arise from learning to transform sentences from a textual representation of a sentence into a situational one. Frank et al. note various limitations of the current DSS model that prevent it from creating an overall interpretation of a story from the incremental transformation of word sequences.

Integrative Processes in Text Comprehension

The third section shifts to studies of integrative processes in comprehension, with a special emphasis on inferencing as a kind of integration process. Schmalhofer and Perfetti combine ERP, fMRI, and behavioral studies of word-to-text integration. They report evidence with strong convergence across fMRI (German) and ERP (English) studies that forward inferences produce less immediate integration compared with text-based integration. For example, in the ERP results, integrating a word with a preceding sentence is less likely when the integration depends on an inference in the preceding sentence than when the preceding sentence contains a paraphrased antecedent of the word. This conclusion is reinforced by results of a parallel fMRI study. Theoretically, Schmalhofer and Perfetti suggest that the process of linking a word to a representation of the prior text includes a resonance process that compares the word with semantic contents (as opposed to the verbatim words) of recent text memory.

Singer and Remillard emphasize a distinction between explicit and implicit text ideas. They show that the process dissociation method can be used to clear up discrepancies between the distinction in text processing between explicit and implicit ideas and the failure of behavioral evidence to capture this distinction and to address the role of automaticity in inference processing. Singer and Remillard compare the retrieval profiles for explicit text ideas, bridging inferences, and elaborative inferences, examining the contributions of recollection and familiarity. Their experiments indicate that there are measurable differences between people’s representation of explicit and implicit text ideas.

The chapter by Reichle and Mason exemplifies the goal of cross-method integration that can bring progress to theories of comprehension. They combine computational modeling and neuroimaging in developing a model to explain results from behavioral and fMRI studies of causal inferences. They demonstrate how the results of an fMRI experiment can inform understanding of the cognitive processes that lead readers to make causal inferences during reading. Their strategy is to link the patterns of fMRI-measured activation during the reading of short texts with the functional roles that have been attributed to the observed brain areas. From this, Reichle and Mason develop a small number of simple computational principles to make predictions about the outcomes of both brain-imaging and behavioral experiments.

McNamara, de Vega, and O’Reilly examine individual differences in comprehension skill, reviewing both behavioral and ERP studies. They conclude that ERP data show that less skilled readers rely more on superficial aspects of reading and do not efficiently manage the use of background knowledge. They conclude also that this picture from ERP data corresponds well with behavioral results that poor readers are less likely to make inferences during reading.

Cognitive Representations

Section IV is six chapters that concern the cognitive representations of comprehended language. The new look in comprehension-that comprehension is grounded in experience and embodied in biological systems-is seen in the chapter by Kaup, Zwaan, and Lüdtke. They hypothesize that understanding negation requires two mental simulations, one of which is the actual state of affairs as described by a text, and the other of which is a simulation of the pragmatically expected state of affairs; they report four experiments testing this hypothesis. Kaup et al. propose a mechanism by which negation is implicitly represented in the experiential simulations created during language comprehension and argue that this mechanism is necessary if the experiential program is to produce a general theory of language comprehension.

In their chapter, Graesser, Louwerse, McNamara, Olney, Cai, and Mitchell examine the link between computational linguistics and the cognitive processes that create a reader’s situation model. Because the construction of a situation model is heavily influenced by a text’s linguistic features, including text cohesion, Graesser et al. identify the factors that contribute to text cohesion and thus to psychological coherence.

They describe a computer system, Coh-Metix, that organizes measures of text properties and allows their application for research that links text properties to comprehension.

Therriault and Rinck examine the general representational issue of the nature of the reader’s situational model for narrative texts. They conclude that, contrary to its privileged position in demonstrations of situation models, space is not a fundamental dimension of a reader’s situation models. Instead, readers are sensitive to fundamental aspects of narrative events, including time, which provides the critical information about the duration, order, and progression of unfolding events. They argue also that goals and causality, which have been accorded privileged status in some accounts, are actually derivative dimensions, dependent upon protagonist information (in the case of goals) and time (in the case of causality). Therriault and Rinck suggest that two relatively neglected dimensions, emotion and perspective, are important in situation models, but leave open the possibility that these dimensions may also be derivative rather than basic.

Long, Baynes, and Prat examine the representation issue through a highly integrative perspective on hemispheric functions. Building on research that has established an important role for the right hemisphere in understanding language, they summarize their priming experiments, which use divided visual-field presentation and which were guided by important distinctions in current theories of discourse processing. Their priming results across four studies show left hemisphere effects at the level of propositional relations, except for left-hemisphere-damaged aphasics. Based on both patient and nonpatient populations, Long et al. conclude that each hemisphere represents explicit information from a text, but each organizes the information differently-within-sentence relations are coded in the left hemisphere only, whereas the right, as well as the left, hemisphere codes lexical-semantic information.

The question of hemisphere functions is taken up as well by Tapiero and Fillon, who address emotional inferences. Like Long et al., Tapiero and Fillon used divided visual field presentation, with lexical decisions made on words that were either congruent or incongruent with an emotional state that could be inferred from a passage that was being read at the time of the decision. Their results confirmed hemisphere differences, with the right hemisphere showing more priming by emotional words. Whether a word was congruent with the inferred emotion and whether the emotional valence was positive or negative had differential effects on the two cerebral hemispheres.

Magliano, Radvansky, and Copeland address the representation issue from the perspective of autobiographical memory. They argue that situation models are general mechanisms that function across a variety of experiences that people have beyond written texts. They develop their argument through examples from movies and virtual reality settings. They argue that among the shared properties of situational models across various experiences are causal relationships. They conclude with the generalists’ proposition that “there are a myriad of ways that the humans can experience events, and it seems reasonable that a common mental apparatus underlies the comprehension of all of them.”

Final Words

Most of the authors of this volume participated in a conference (Higher Level Language Processes in the Brain: Inference and Comprehension Processes) held at the Hanse Advanced Study Institute in Delmenhorst, Germany and organized by the editors of this volume. The conference focused on interaction and research integration around comprehension issues and included several specific steps (prior circulation of critical questions, discussion sessions on conference topics) to bring this about. A distinctive outcome was the emergence of several joint chapters from individual participants. For example, Paul van den Broek joined with Susan Goldman and Richard Golden to write a state-of-the-art modeling chapter, turning two conference presentations into one well-integrated chapter. Erik Reichle and Robert Mason also teamed up for a collaborative chapter, specifically choosing to combined their respective expertise in neuroimaging and computational modeling for an integrative chapter that neither would have been inclined to write individually. Following separate presentations at the conference, Franz Schmalhofer and Charles Perfetti came to agree that their differences on inferences had been significantly reduced, allowing them to work on a collaborative chapter that could become more integrative than their separate chapters could have been.

Charles Perfetti
Franz Schmalhofer

I

State of the Art

2

Psychological Studies of Higher Language Processes: Behavioral and Empirical Approaches

Murray Singer
University of Manitoba

Jose Leon
Universidad Autonoma de Madrid

During the past 35 years, higher language processes have been subjected to intensive study within an information-processing framework. This brief time span has witnessed remarkable progress in the discovery of basic phenomena of language comprehension; the identification of cognitive mechanisms underlying these phenomena; and the proposal of detailed, testable theories in this domain. Much of this progress has been driven by the development and application of behavioral methods for the scrutiny of people’s comprehension of spoken and written language. These techniques have generated a steady stream of empirical evidence to challenge cognitive scientists.

This chapter documents and elaborates on the conversations of the Behavioral/Empirical discussion group at the International Hanse-Conference on Higher Level Language Processes in the Brain. The group focused on areas of distinct progress within this realm, and qualifications and constraints of the behavioral approach. This chapter addresses these issues, using a parallel structure.

Progress in the Study of Language Processes

Information-processing research concerning language comprehension has been characterized by the identification, reconsideration, and regular refinement of the understanding of numerous phenomena central to this domain. Furthermore, these advances have been continuously guided by detailed theoretical formulations. In this section, several phenomena and the pertinent theoretical constructs are reviewed.

Propositional Representations

Early in the enterprise, there was agreement that progress in language comprehension research required a well-defined system of semantic analysis. What emerged from this consensus was the proposal that one or more elementary ideas, or propositions, underlie each clause or sentence (e.g., Clark, 1969; Kintsch, 1972; Schank, 1972; Winograd, 1972). According to Kintsch’s (1972) influential analysis, each proposition consists of a predicate plus one or more arguments. Predicates typically correspond to a verb or other relational term, and arguments correspond to nouns. Accordingly, the propositions underlying sentence (1) are shown in (2), with the predicate of each proposition appearing first:

1. The car pulling the trailer climbed the steep hill.
2. 1. (PULL, CAR, TRAILER)
   2. (CLIMB, CAR, HILL)
   3. (STEEP, HILL)

Equally importantly, these propositions were proposed to be organized in networks (Anderson & Bower, 1973) or propositional schemas (van Dijk & Kintsch, 1983).

Propositional analysis was an effective tool for identifying units of meaning. However, a considerable body of empirical work was needed to establish that propositions functioned as psychological elements. Kintsch and Keenan (1973) showed that reading time for complex sentences varied systematically as a function of (a) their number of propositions rather than word length and (b) the number of propositions that the readers recalled from those sentences. Likewise, reading time varies with the number of different arguments underlying a message (Haberlandt &...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright
• Contents
• 1. Editors’ Introduction: Mind and Brain in Higher Level Comprehension
• I: Stat e of the Art
• II: Computational Models
• III: Integrative Processes in Text Comprehension
• IV: Cognitive Representations
• Author Index
• Subject Index