Connectionist Models of Development

Developmental Processes in Real and Artificial Neural Networks

Philip T. Quinlan

• Publisher: Psychology Press
• Year: 2004
• ISBN:
  9781135426590
• Edition: 1
• Topic: Psicología
• Subtopic: Historia y teoría en psicología

Connectionist Models of Development is an edited collection of essays on the current work concerning connectionist or neural network models of human development. The brain comprises millions of nerve cells that share myriad connections, and this book looks at how human development in these systems is typically characterised as adaptive changes to the strengths of these connections. The traditional accounts of connectionist learning, based on adaptive changes to weighted connections, are explored alongside the dynamic accounts in which networks generate their own structures as learning proceeds.
Unlike most connectionist accounts of psychological processes which deal with the fully-mature system, this text brings to the fore a discussion of developmental processes. To investigate human cognitive and perceptual development, connectionist models of learning and representation are adopted alongside various aspects of language and knowledge acquisition. There are sections on artificial intelligence and how computer programs have been designed to mimic the development processes, as well as chapters which describe what is currently known about how real brains develop.
This book is a much-needed addition to the existing literature on connectionist development as it includes up-to-date examples of research on current controversies in the field as well as new features such as genetic connectionism and biological theories of the brain. It will be invaluable to academic researchers, post-graduates and undergraduates in developmental psychology and those researching connectionist/neural networks as well as those in related fields such as psycholinguistics.

CHAPTER ONE
A connectionist perspective on Piagetian development

Sylvain Sirois

Department of Psychology, University of Manchester, UK

Thomas R.Shultz

Department of Psychology and School of Computer Science, McGill University, Montreal, Canada

The significance of Jean Piaget’s contribution to the study of cognitive development has gained the status of a truism. Even today, some of his original ideas such as object permanence spawn significant theoretical debates (e.g. Baillargeon, 2000; Bogartz et al., 2000), generating contributions from connectionist and other computational approaches as well (see Mareschal, 2000, for a review). However, Piaget’s heuristic value is not limited to the identification of several robust empirical regularities, however useful these may be to benchmark developmental theories.

Piaget’s contribution to the study of cognitive change can be divided, for convenience, into four distinct areas. First, as alluded to in the previous paragraph, his work led to the identification of several robust findings about infants’ and children’s cognitive abilities. Notions such as object permanence, conservation, and seriation, for example, spawned hundreds of studies that replicated, extended, or refined Piaget’s earlier work.

Second, his many original findings stemmed from a novel methodological approach that stressed underlying cognitive operations rather than the usual success or failure assessment of performance. In the clinical interview, for instance, a child’s errors prompt researchers to inquire about the justifications for the child’s behaviour. These justifications provide insights into the underlying conceptual system of the child that elude traditional, quantitative assessment approaches.

Third, Piaget proposed a unifying framework to discuss cognitive competence. His structural theory argued that any level of cognitive functioning from infancy into adulthood, was a function of a general information-processing structure, the nature of which changed over the course of a person’s interaction with the environment.

Finally, Piaget proposed mechanisms of cognitive change. Notions such as assimilation, accommodation, equilibration, and abstraction refer to the processes involved in the various changes observed over infancy, childhood, and adolescence. It may prove useful to bear these four aspects of Piaget’s legacy in mind when assessing his contribution to developmental psychology.

Piaget’s theory as a whole is no longer tenable nowadays, which is the second truism of this introduction. However, rejecting the whole of Piaget’s contribution on the basis of criticisms of specific aspects of his work could prove detrimental for current research. Piaget has been proved wrong in many respects, but quite right in others, and sometimes just too vague for proper assessment (Boden, 1994). For instance, some of his empirical findings rely heavily on the specific methodology he used. Diamond (1985) showed how performance on object permanence tasks can be altered by modifying the delay between disappearance and search, for example. Furthermore, Piaget’s suggestion that a general structure, or structure d’ensemble, sustains all of cognition proves untenable (Karmiloff- Smith, 1992). That is, progress in different domains can follow different trajectories and, at times, suggest different levels of competence. However, despite these and other criticisms, current connectionist research independently supports one of Piaget’s central claims: that cognitive change is a function of self-organisation and adaptation.

This chapter examines how neural network research might help to clarify the crucial yet vaguely specified mechanisms of cognitive change in Piaget’s theory. It has been argued that notions such as assimilation and accommodation were too vague to be useful (Klahr, 1982). Neural networks escape such criticism because they are fully specified, and it may prove fruitful to assess whether Piaget’s vague intuitions have correlates in these dynamical mechanisms. We focus on a particular algorithm—cascade correlation (henceforth the CC algorithm), which has been used successfully to model a host of developmental phenomena, including Piagetian tasks. It must be stressed that the purpose of the chapter is to draw analogies between aspects of Piaget’s theory and aspects of connectionist models. We are not attempting to express one theory in terms of the other. Rather, Piagetian research can benefit from the specification and computational power inherent to neural network research, and connectionist work can benefit from the broad theoretical framework laid down by Piaget.

The first section of the chapter outlines the mechanisms of cognitive change discussed by Piaget, and stresses how these mechanisms relate to learning and development. The second section presents the CC algorithm (Fahlman & Lebiere, 1990) and suggests how aspects of the model are analogous to Piagetian mechanisms. The third section discusses the success of the CC algorithm at modelling number conservation, a landmark Piagetian task. The fourth section, to highlight the broad applicability of the model, discusses the application of the algorithm to a typical empiricist problem: discrimination shift learning. Finally, the discussion argues that generative neural networks such as the CC algorithm can provide the building blocks of a general framework for cognitive development and, in the process, provide a novel level of specification to some of Piaget’s important but illspecified ideas.

PIAGET’S MECHANISMS OF CHANGE

Piaget’s developmental theory is articulated around the notion that cognition is a function of a general knowledge structure. Within a given structure (i.e. a level of development), three processes strive to optimise representations: assimilation, accommodation, and equilibration. When an optimised structure nevertheless fails to adequately represent information from the environment, the process of reflective abstraction generates a higher-level structure aimed at improving the child’s understanding of the world.

Assimilation is the process through which information from the environment is distorted to fit the current cognitive structure of the organism, and accommodation is the adaptation of the structure to environmental input. Assimilation prompts accommodation, and accommodation improves further assimilation (Piaget, 1980). Equilibration is the process that maintains a balance between assimilation and accommodation, ensuring that (only) enough accommodation takes place to promote satisfying assimilation. Piaget called these three processes (assimilation, accommodation, and equilibration) functional invariants. He assumed that they were part of the infant’s innate endowment (along with a few motor reflexes), and that the processes never changed over the course of life. That is, he considered the functional invariants to be impervious to experience. These invariant functions built the mind, using experience and the current state of the mind as materials.

The process of equilibration, with respect to assimilation and accommodation, produces a state of equilibrium. For Piaget, equilibrium was the goal of a self-organising, adaptive mind (Boden, 1994). But this equilibrium is only temporary if it is the current best solution from an inadequate structure. That is, a child can be satisfied with his or her solution to a problem only as long as it is his or her best comprehension of the problem, even in the face of failure. But the repeated conflict between the child’s assimilationaccommodation achievement and disconfirming environmental feedback will eventually prompt a structural reorganisation.

Structural changes are possible through the process of abstraction, of which Piaget (1980) distinguishes two forms¹ whose functioning is also regulated by a process of equilibration. Reflective abstraction involves a functional reorganisation of a cognitive structure in order to promote higher-level assimilation-accommodation. Reflective abstraction itself consists of two distinct processes: (1) reflecting, which is a projection to a higher level of what is at a lower level; and (2) reflexion, which is reorganisation at a higher level. The other form of abstraction is reflected abstraction (or reflected thought), which concerns making explicit and integrating functional structures generated through reflective abstraction. For Piaget (1972, 1980), the semiotic function (i.e. the capacity to represent objects through symbols) is essential for abstraction, and language is especially important in reflected abstraction.²

Recent attempts to discuss Piaget in neural network terms (McClelland, 1995; Shultz, Schmidt, Buckingham, & Mareschal, 1995) did not quite succeed at this endeavour, mainly because they focused on assimilation and accommodation, ignoring the notion of abstraction. This focus on the twin processes of assimilation and accommodation may reflect a common belief; namely, that these are the fundamental developmental mechanisms in Piaget’s theory. In his critique of Piaget, for example, Klahr (1982) described assimilation and accommodation as the “Batman and Robin” of cognitive development. With this analogy, Klahr stressed the mysterious nature of the processes, and thus the need to go beyond vague statements with respect to the mechanisms underlying cognitive change.

Although legitimate, Klahr’s severe evaluation of Piaget is surprising for two reasons. First, for many Piagetians, equilibration—the “Superman” of development—is far more mysterious than assimilation and accommodation (Furth, 1972; Gallagher & Reid, 1981). Second, the process of abstraction is probably the most important developmental aspect of Piaget’s structural theory (Case, 1999), and not assimilation and accommodation as many believe (Nersessian, 1998; Siegler, 1998). An outstanding question, not addressed by the Batman and Robin analogy, concerns the nature of learning. Piaget acknowledged that there was learning and not just development, but never clearly outlined how learning was achieved or how it interacted with development (Gallagher & Reid, 1981).

To discuss Piaget’s mechanisms of change with respect to learning and development, we first need to introduce definitions we have outlined

¹ Actually, there is a third form of abstraction called empirical abstraction (Piaget, 1980), which concerns the acquisition of object properties (i.e. content rather than competence acquisition). Empirical abstraction allows the storage of factual information from the environment.

² This section presents a significantly simplified account of Piaget’s theory, on which he worked for over 50 years. Conservative estimates suggest that on cognitive development alone, Piaget published over 40 books and 100 articles (Miller, 1989). This section presents a summary of key elements in Piaget’s theory based mostly on one of his later papers (Piaget, 1980), where his work-in-progress was possibly most explicit.

elsewhere (Sirois & Shultz, 1999). We defined learning as parametric change within a processing structure in order to adapt to the environment. This broad description was meant to be compatible with general statements about learning, such as found in nativistic (Fodor, 1980) and developmental (Carey 1985; Piaget, 1980) accounts. A key element in this definition of learning is the quantitative nature of the process. In contrast, development was defined as change of an existing structure to enable more complex parametric adaptations. This definition highlights a key idea for most developmental theories: a qualitative change in the structure supporting cognitive activity. For instance, our definition of development is compatible with Carey’s (1985) conceptual change, and Karmiloff-Smith’s (1992) representational redescription. We argued that such functional definitions of learning and development allow for useful distinctions between the two processes. There is no overlap between the processes, which constrains their unique contribution to cognitive change: learning as parameter adjustment within a given structure, and development as structural change that enables further learning.

Discussing the mechanisms of change in Piaget’s theory within this formal distinction between learning and development, a specific interpretation emerges. Assimilation and accommodation can be construed as the learning component in Piaget’s work, a different interpretation than typical Piagetian accounts that view accommodation as a developmental mechanism (Gallagher & Reid, 1981; Siegler, 1998). We suggest that accommodation only results in quantitative changes within the existing cognitive architecture. Essentially, information is assimilated in the system and, if the discrepancy between the internal representation and the external information is important, the system accommodates (i.e. adjusts its parameter values). The process of equilibration, which strives to move the system towards a state of equilibrium, ensures that enough accommodation takes place given the current level of assimilation. Learning is thus the quantitative process that adapts the current representational structure to the input it receives. A system in equilibrium is one where further accommodation would not improve assimilation.

Development is the more radical process by which an inadequate architecture is qualitatively transformed in order to promote further learning. Reflective abstraction describes this developmental process. The current representational structure is reformulated at a more adaptive level (reflecting), whereby the new structure can be used for further parametric adaptations (reflexion). Furthermore, the construction of the new structure makes use of the same mechanisms involved in learning. The new structure assimilates the previous representations and accommodates towards a new level of equilibrium. Whereas learning consists of adapting the current structure, development involves learning new structures. This suggests a rather compact model with few primitives (the three functional invariants) implementing two functionally different processes (external adaptation versus internal restructuring).

This interpretation of mechanisms of change in Piaget’s theory marks a departure from recent attempts (McClelland, 1995; Shultz et al., 1995). These approaches focused on assimilation and accommodation as developmental mechanisms, which misrepresents Piagetian development. Assimilation and accommodation do not produce higher-level representational structures; rather, these mechanisms work within the confines of the current structure. The key developmental process in Piagetian theory is reflective abstraction (Case, 1999). What we propose in this chapter, then, is a framework that explores the full range of functional change mechanisms in Piaget’s work.

We will discuss the usefulness of this interpretation in a later section. For now, we should acknowledge that our description of Piagetian learning and development remains rather abstract. Piaget was vague about how his theory could be implemented and his lack of mechanical specification is reflected in our review. However, it is useful to turn to neural network models for the level of specification we ultimately strive for and which renders the theory operational and testable. The next section presents a neural network algorithm that exhibits some unique properties to aid in this enterprise.

THE CC GENERATIVE ALGORITHM

The cascade correlation (CC) algorithm is a feedforward neural network algorithm. Such networks are trained to acquire the functional dependency that exists in pairs of input and output patterns (i.e. that a set of target output activations Y is some function of a set of inputs X). At any time in training, network error is defined by the discrepancy between the target output activations and the network’s current output. The structure of the network (the number of modifiable weights, the activation functions of the units, and how units are connected to one another) defines a multidimensional error surface. Training involves navigating on this error surface towards (ideally) its global minimum.

The learning rule in CC is called quickprop and, as the name implies, it is both similar to and quicker than back-propagation. Connection weights are changed as a function of the discrepancy between actual and desired output, as in back-propagation, but it uses the second-order derivative of the unit activat...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• List of contributors
• Modelling human development: In brief
• Chapter One A connectionist perspective on Piagetian development
• Chapter Two Connectionist models of learning and development in infancy
• Chapter Three The role of prefrontal cortex in perseveration: Developmental and computational explorations
• Chapter Four Language acquisition in a self-organising neural network model
• Chapter Five Connectionist modelling of lexical segmentation and vocabulary acquisition
• Chapter Six Less is less in language acquisition
• Chapter Seven Pattern learning in infants and neural networks
• Chapter Eight Does visual development aid visual learning?
• Chapter Nine Learning and brain development: A neural constructivist perspective
• Chapter Ten Cross-modal neural development
• Chapter Eleven Evolutionary connectionism