The SAGE International Handbook of Educational Evaluation

Katherine E. Ryan, J. Bradley Cousins

• Publisher: SAGE Publications, Inc
• Year: 2009
• ISBN:
  9781483343457
• Edition: 1
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Bringing together the expertise of top evaluation leaders from around the world, The SAGE International Handbook of Educational Evaluation addresses methods and applications in the field, particularly as they relate to policy- and decision-making in an era of globalization. The comprehensive collection of articles in the Handbook compels readers to consider globalization influences on educational evaluation within distinct genres or families of evaluation approaches.

Key Features

• Discusses substantive issues surrounding globalization, and its implication for educational policy and practice and ultimately evaluation;
• Includes state-of-the-art theory chapters and method chapters within scientific, accountability-oriented, learning-oriented, and political genres of evaluation approaches;
• Provides real-world case exemplar chapters to illustrate core concepts within genres;
• Extends dialogue on controversial topics and contemporary educational evaluation tensions in the context of globalization;
• Summarizes, by means of an integration chapter, the issues, tensions and dilemmas confronting educational evaluators in an era of globalization.

Serving as a state-of-the-art resource on educational evaluation, this volume is designed for graduate students, evaluation scholars and researchers and professional evaluation practitioners with an interest in educational program and policy evaluation.

PART I

The Educational Evaluation Context

The chapters in Part I provide the context for the volume, including a sense of the current milieu surrounding educational evaluation in an increasingly globalized society. Moreover, critical concepts and issues involving the volume’s framework, such as globalization, educational evaluation foundations, and how globalization may be influencing historical evaluation dilemmas (e.g., evaluator identity and role), are addressed.

In Chapter 1, Rizvi presents a sweeping overview of how current educational policy research, and in turn educational evaluation, is being changed by globalization. Among other matters, he briefly reviews various meanings of globalization and proposes that a transnational framework is essential for an examination of these changing national and international educational policy processes. In Chapter 2, Schwandt addresses educational evaluation inquiry foundations, including philosophical, ethical, and political assumptions and how they are being influenced by globalization discourses and practices. Notably, he begins with a critical examination of the definitions of educational evaluation—a concept that is characterized by multiple meanings. After analyzing challenges to the Western evaluation identity posed by globalization, Schwandt concludes that these challenges can be viewed productively by turning the tensions to opportunities for reexamination of educational evaluation aims. In the last chapter in Part I, Smith (Chapter 3) proposes historical evaluation dilemmas (e.g., what is acceptable evidence?) as a lens for considering how educational evaluation is changing in response to the circumstances of globalization. After elaborating a variety of definitions of the globalization of evaluation (e.g., globalization as spread), Smith proposes that educational evaluation theories and practices will need to become more flexible and accommodating around global issues. Such issues include, for instance, the value of written versus oral communication and individual versus group interests.

CHAPTER 1

Globalization and Policy Research in Education

Fazal Rizvi

This chapter explores some of the ways in which policy research in education—conceived broadly to include studies of the context in which policy options are explored, the political dynamics associated with policy development and implementation, and the issues surrounding policy and program evaluation—is affected by the contemporary processes of globalization. It suggests that globalization has destabilized the traditional conception of policy research as territorially bounded. Public policy research in particular can no longer take for granted the exclusive link between a given territory, such as the state, and its political authority (Sassen, 2007). The globalization of economy and various transnational political and cultural forces have altered both state and non-state institutions. The state has become a fragmented policy arena, permeated by policy networks that are both domestic and transnational, whereas the work of non-state institutions is no longer confined to national boundaries, but potentially stretches across the globe.

These transformations demand a new perspective on educational policy research, including educational evaluation. In evaluating policies and programs, for example, we need to examine how they are now produced and legitimated within a broader transnational framework, and how they are steered by the global forces of capital, various corporate interests, as well as transnational relations of more informal kinds. We need to ask how global policy networks affect and are utilized in the processes of policy development and evaluation. From a broader perspective, we need to consider the ways in which these transnational dynamics have possibly created conditions for the global convergence of educational policies.

Within this shifting context, this chapter suggests that globalization has led to the production of a new social imaginary of education, created as a result of the growing pressure on states for more market-friendly policies, better aligned to the requirements of the global knowledge economy. There is, however, nothing inevitable about this social imaginary of education. If it expresses values that are not in line with the popular sentiments for democracy and social justice, and specific national interests, then it is argued in this chapter that a major contemporary task of educational evaluation is to “name” this contradiction and suggest alternatives that contest the dominant neoliberal policy discourses now circulating around the world.

The argument in this chapter is structured around a number of steps. First, it shows how the nature and sources of authority underlying public policy in education is changing. Second, it suggests that these changes are influenced by the contemporary processes of globalization. It is argued that globalization can be viewed both as an objective and subjective phenomenon, and that, as a subjective phenomenon, it describes how policy authority is institutionalized through the “work of social imagination” (Appadurai, 1996). Third, this chapter shows how recent processes of globalization have thus steered educational systems around the world toward a particular social imaginary of education, predicated on a range of neoliberal assumptions. However, this imaginary has emerged in neither a spontaneous nor a deterministic fashion, but through processes that are inherently political and work in various locally contingent ways. The final section of this chapter discusses some of the implications of this argument for the theory and practice of educational evaluation.

Public Policy and the Sources of Its Authority

---

It is now more than 50 years since David Easton, regarded by many as one of the architects of the field of study we now call “policy research,” provided a most succinct definition of the concept of policy. Easton (1954) argued that policies expressed “authoritative allocation of values.” Policies are always normative: They presuppose certain values and direct people toward action, but in ways that are authoritative. Their legitimacy is derived from an authority, be it a government, a corporation, or a social institution such as a school system or church. Easton’s definition has been highly influential; it has been assumed by policy researchers working within the positivist and rationalist traditions, and it has formed the basis of interpretive and critical approaches.

Stephen Ball (1994), for example, regards an exploration of the nature and scope of authority to be a central task of policy research. Policy research, he argues, involves an examination of three key aspects of policy: texts, discourses, and effects. Policies, he suggests, are always contested, value-laden, and dynamic, and they are a product of various compromises. They are encoded in representations of what is mandated and what should be done. They are often expressed in a textual form, but within the framework of a broader discourse. Policy analysis thus involves the decoding of texts, in relation to both the context in which they are embedded and the effects they have.

If Easton’s definition provides a basic framework for interpreting and analyzing policies, then the questions arise as to where the authority underpinning a policy comes from and how authority is exercised through that policy. With respect to public policies in education, the answer to the first of these questions would appear to be the state. Without the authority of the state, public policies can neither be supported with public funds nor have the symbolic value to guide action. But how does the state allocate its authority through policy, seeking to manage community expectations and to develop subjects who are sufficiently vested in its political priorities? How do certain values become authoritative? Expressed in these terms, the idea of authority would appear to be central to the notion of policy. Yet few policy researchers address it explicitly. So, for example, although Ball’s discussion of the idea of policy highlights the complexities of the various ways in which policies are constructed and interpreted, it does not problematize issues surrounding the nature of policy authority itself.

Like most other theorists of policy, Ball assumes policy authority to be located within the structures of the state, from where policy texts and discourses get their purchase. Indeed, public policies in education have traditionally been thought to emanate from a national government and its agencies, designed by the state to deliver educational provision in a most effective and efficient, and sometimes equitable, manner. Not surprisingly, therefore, most policy researchers assume state authority to be sovereign. This assumption is based on a Westphalian understanding about the nature of political authority (Krasner, 2000), which includes the view that authority can only be exercised by a state over a defined geographical territory, that each state has the autonomy to develop its own policies, and that no external actor can direct that state’s priorities.

Modern states are assumed to hold ultimate territorial jurisdiction, organized around a specific set of administrative functions. This assumption of the taken-for-grantedness of territoriality can be found in the work of most 20th-century social theorists and policy researchers. Indeed, it has been considered a fundamental feature of modernity on which the political architecture of the modern state system is based (Mann, 2000). The modern state is represented as a kind of container that separates an “inside” of domestic political interactions from an “outside” of international or interstate relations (Brenner, Jessop, Jones, & Macleod, 2003). The state is thus given an authoritative monopoly over the subjects and institutions located within its territory, allocated through a system of international relations.

However, this conception of authority, involving the institutional, territorial, and centralized nature of the state, cannot be sustained without popular consent. It requires a “social imaginary” (Taylor, 2004) concerning the nature and scope of the state’s political authority. It demands people’s consent to view national formations as inevitable, timeless and natural, territorially bounded, and entirely legitimate. This view is in line with Anderson’s (1991) contention that nations are “imagined communities” that were brought into existence in the early modernization processes initially by intellectuals, artists, political leaders, and others and only later became infiltrated into the whole society through myths, stories, songs, and the like. Processes of formal schooling played a major role in developing and sustaining national imaginaries. Assumptions surrounding national authority came to be widely accepted, sometimes through the exercise of violence, but mostly through the inculcation of a social imaginary without which people could not even conceive how things could be otherwise. Most policy research in education continues to operate within this Westphalian framework.

Globalization has destabilized this framework. In what follows, I argue that even if the authority of the state has not entirely declined, and even if many states remain influential and strong, its nature and functions are changing. The state is no longer the only site of policy development and source of political legitimacy; transnational processes intersect in a variety of complex ways with the mechanisms of policy development, dissemination, and evaluation at the national level. If the assumption that policy authority is always located within the structures of the state can no longer be taken for granted, then it follows that, in analyzing the ways in which values are allocated in and through policy, we can no longer merely attend to issues internal to the state, but also need to ask how the interior of the state is being reconstituted by forces emanating from outside its borders, becoming “relativized” (Waters, 1995) by the processes of globalization.

Globalization

---

In less than two decades, the idea of globalization has become ubiquitous, widely used around the world in policy and popular discourses alike. It is used to describe the various ways in which the world is becoming increasingly interconnected and interdependent, referring to a set of social processes that imply “inexorable integration of markets, nation-states and technologies to a degree never witnessed before—in a way that is enabling individuals, corporations and nation-states to reach round the world farther, faster, deeper and cheaper than ever before” (Friedman, 2000, p. 7). Such integration is far from complete and clearly benefits some communities more than others. This suggests that globalization is not experienced and interpreted in the same way everywhere.

Not surprisingly, therefore, globalization is a highly contested notion. Historically, it is articulated with a range of colonial practices (Rizvi, 2007). But its more recent forms are associated with technological revolutions in transport, communication, and data processing. These developments have transformed the nature of economic activity, changing the modes of production and consumption. Harvey (1989) provides perhaps one of the best descriptions of economic globalization. He argues that globalization describes “an intense period of timespace compression that has had a disorienting and disruptive impact on political-economic practices, the balance of class power, as well as upon cultural and social life” (p. 9). In this new era, global capitalism has become fragmentary as time and space are rearranged by the dictates of multinational capital.

Improved systems of communication and information flows and rationalization in the techniques of distribution have enabled capital and commodities to be moved through the global market with greater speed.

At the same time, there has been a shift away from an emphasis on goods to greater trade in services—in business, educational, and health services, as well as entertainment and lifestyle products. The rigidities of Fordism have been replaced by a new organizational ideology that celebrates flexibility as a foundational value, expressed most explicitly in ideas of subcontracting, outsourcing, vertically disintegrated forms of administration, just-in-time delivery systems, and the like. In the realm of commodity production, argues Harvey, the primary effect of this transformation has been an increased emphasis on virtues of speed and instantaneity.

Considerable significance is now attached to information and global networks. According to Castells (2000) the new economy is knowledge-based, postindustrial, and service-oriented. Cultural and political meanings, Castells argues, are now under siege by global economic and technological restructuring. He speaks of an “informational mode of development” through which global financial and informational linkages are accelerated: They convert places into spaces and threaten to dominate local processes of cultural meanings. According to Castells (2000), networks constitute “the new social morphology of our societies”; and the diffusion of networking logic, substantially modifying “the operation and outcomes in the processes of production, experience, power and culture.” The new economy, he maintains, is “organized around global networks of capital, management, and information, whose access to technological know-how is at the roots of productivity and competitiveness” (p. 82). This view implies, then, that states are no longer the only or even the major drivers of the global economy.

Within the system of modern states, considerable cultural importance was attached to education. Educational systems carried the narratives of the nation. As Gellner (1983) points out, it was the mass educational systems that provided a common framework of understanding, which enhanced the processes of state-coordinated modernization. Through the diffusion of ideas, meanings, myths, and rituals, citizens were able to “imagine” the nation and filter conceptions of their “other.” Although education continues to serve this function, according to many globalization theorists (e.g., Steger, 2003), the nation-state can now be imagined in a number of different ways, and the lives of its citizens are now inextricably linked to cultural formations that are produced in far-away places.

Under the conditions of glob...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• Introduction
• Acknowledgments
• PART I. THE EDUCATIONAL EVALUATION CONTEXT
• PART II. THE ROLE OF SCIENCE IN EDUCATIONAL EVALUATION
• PART III. EDUCATIONAL EVALUATION, CAPACITY BUILDING, AND MONITORING
• PART IV. EDUCATIONAL EVALUATION AS LEARNING AND DISCOVERY
• PART V. EDUCATIONAL EVALUATION IN A POLITICAL WORLD
• PART VI. EDUCATIONAL EVALUATION: OPPORTUNITIES AND NEW DILEMMAS
• Author Index
• Subject Index
• About the Editors
• About the Contributors