Participatory Evaluation In Education

Studies Of Evaluation Use And Organizational Learning

Lorna M. Earl, J. Bradley Cousins

• Publisher: Routledge
• Year: 2004
• ISBN:
  9781135718848
• Edition: 1
• Topic: Education
• Subtopic: Education General

This text focuses on "participatory evaluation", an approach that involves teachers and educational administrators as partners with researchers in a broad range of school and school system-based evaluation tasks with the explicit goal of using such data to improve practice.; Participatory evaluation is a natural, suitable and effective approach to school improvement and educational change, and has been practiced by the editors and several colleagues for many years. Though participatory applied research strategies are growing in popularity, there is a paucity of documented empirical support for the approach. presenting a set of original empirical studies and a critical analysis of them this book will add to our knowledge about variations in the approach, conditions that support it, its viability within the culture of schools and school systems and its likely impact defined in terms of the use of research data and organisational learning.; The book will be useful for educational practitioners interested in critically evaluating the potential of participatory evaluation as an integral part of their own approach to educational reform. It will also clarify an agenda for research to further our understanding of the organisational benefits of this type of collaborative systematic enquiry.

Part 1
Why Participatory Evaluation?

In 1992 we published an article in Educational Evaluation and Policy Analysis that provided a conceptual foundation for our ideas on participatory evaluation. In the paper, we described participatory evaluation and prepared a justification for it from a review of empirical and theoretical research. We then presented some preliminary thoughts about the conditions necessary in order for participatory evaluation to be effective, and an agenda for research. One of the blind reviewers for that article called it the best paper in evaluation he or she had seen in several years. Many of the ideas presented in that 1992 article are revisited an chapter 1, but the chapter extends them by grounding the case more directly in the context of considerations important to the process of change in educational organizations. Also, we elaborate our ideas about organizational learning, a theoretical orientation with natural appeal for utilization-oriented participatory evaluation. Finally, we end up with a set of questions that require investigation, a natural point of departure for the remainder of the book.

Chapter 1
The Case for Participatory Evaluation: Theory, Research, Practice

J.Bradley Cousins and Lorna M.Earl

‘Show Me’

The teacher’s role has and will continue to evolve at a rather sharp pace as schools persevere into the next century with a variety of tensions and conflicts. There seems to be little question that norms of privacy and isolation from peers and colleagues are under siege. At the same time, teachers’ resolve to protect longstanding interests and to maintain a territorial stance on classroom decision making is pervasive and strong. Teachers are just plain unwilling to make a fundamental shift simply because a new idea sounds good. Blind acceptance of new direction and expansion in the role is not enough. Not unlike Missouri state automobile licence plates that proudly convey to the world the state’s nickname and a people’s disposition toward ‘high falutin’ proposals and ideas, teachers are saying ‘show me’.

This book is about the acceptance of a relatively new direction and expansion of the role for educators. In particular, the focus for discussion, debate and deliberation is teachers’ and principals’ involvement on school- and system-based applied research projects; activities that clearly fall outside traditionally defined teaching roles. All things considered, why on earth would teachers and principals want to become involved in applied research activities, ventures that are entirely likely to spell hard work, heightened anxiety, tension and stress, and general disequilibrium? We have some distinct ideas about the answer to this question. The basis for our ideas is several years of experience working in schools and school systems with educators on applied research projects of local interest. We have seen the benefits and we have seen the pitfalls. We have hurdled obstacles and we have run into brick walls. But we continue to pursue with alacrity, opportunities to work in partnership with our practicebased colleagues and we absolutely delight in their invitations to come back. We understand and accept that our delights are not sufficient to persuade others. We also appreciate that by merely presenting our rationale for embracing collaborative research projects, no matter how coherent, attractive or ‘high falutin’, we will not even begin to persuade the uninitiated. We know that the proud message of the hearts and licence plates of Missourans defines the path that we must take. That message provides both the starting point and the impetus for this book.

Professionalization as Inquiry Mindedness

The tensions, pressures and influences confronting schools are many, highly varied and pervasive. As others have cogently considered these influences (see, for example, Glickman, 1993; Leithwood, Begley, and Cousins, 1992; and Murphy, 1991) we will not belabor them here. Suffice it to say that such forces as: rapid movement from industrial to technology-based economies; aging populations; cultural, religious and ethnic diversity; individual rights and freedoms; and the evolving role of the family and its implication for children’s educational experiences, provide the dynamic and turbulent backdrop for contemporary schooling. These forces continue to have significant implications for schools, especially in defining the impetus for change and reform.

‘Restructuring’, yet another over-used and overworked term in educational circles, provides the current handle for intended significant and sustained planned change and school improvement. Restructuring and reform, of course, are by no means peculiar to educators; organizations in both the public and private sector are globally embracing the rhetoric for change. Restructuring in business and industry, however, implies something different from the common understanding in education. There the term suggests a focus on innovation and new product development with the expressed purpose of enhancing competitiveness in the global marketplace. In education, on the other hand, restructuring tends to imply the reconfiguration of organizational roles, relationships, and structures, often in the context of some locally defined and valued end. Elmore and associates (1990) describe what they see as three basic thrusts in educational restructuring:

1. raising educational standards, reflected predominantly in calls for ‘back to the basics’, closer monitoring (testing) of student achievement, and heightened accountability demands on educators;
2. involvements of members of the school community as legitimate and true partners in the educational process, reflected in shared governance initiatives, provision to parents of latitude in selecting their children’s school, and general efforts to engage families as working partners in their children’s educational experience; and
3. the professionalization of teaching.

This third thrust provides the overarching framework for the present book.

The so-called professionalization of teaching implies significant reform in what teachers do and think. The concept is a slippery one that means different things to different people. Perhaps fundamental to our way of thinking about professionalization is the cultivation and development of a posture of ‘inquiry-mindedness’ regarding technical core activities, manifest in teachers’ genuine participation in the determination of school goals and the means adopted to achieve those goals. The thinking among teachers as professionals transcends the bounds of procedural knowledge or the drive to develop more fully an understanding of ‘how’ one embraces instructional tasks, and extends more deeply to questioning the very reasons for doing the task at all. Calls for reform in classroom practice and the introduction of educational innovations are not accepted at face value, nor are they rejected out of hand within a general mood of frustration and cynicism. Rather, new ideas are critically evaluated against professional wisdom, a collective understanding of educational purpose and school-specific goals, and a clear and articulated sense of underlying assumptions and values. The emphasis on asking ‘why?’ invariably precedes questions of technical fidelity.

Organizations as Learning Entities

If we embrace an image of professionalization as inquiry-mindedness, what are the implications for our view of what organizations are and how they operate? One set of theoretical principles called ‘organizational learning’ provides a suitable framework within which to consider this question.

Although relatively new to the study of educational administration, organizational learning concepts have been considered in the broader study of organizational theory for quite some time (Cousins, in press; Huber, 1991; Louis, 1994). They are premised on the assumption that learning in organizations is not merely the sum of organization member learnings. Herbert Simon (1991, pp. 125–6) comments on the synergistic qualities of learning at this level:

Human learning in the context of an organization is very much influenced by the organization, has consequences for the organization and produces phenomena at the organizational level that go beyond anything we could infer simply by observing learning processes in isolated individuals.

We see here the natural link to the fundamental notions of social interactionism and Bandura’s foundational work on social learning theory (1977; 1986). The underlying premise is one of an interactional model of causation in which personal factors associated with individuals, environmental events and behaviors operate as interacting determinants of one another. To follow, we summarize the key features of organizational learning theory as a conceptual context for considering strategies or interventions designed to enhance the learning capacity of organizations.

• Knowledge representation Knowledge is represented in organizations in a variety of ways. Theorists (e.g., Argyris and Schön, 1978) differentiate between espoused theories such as one would find in organizational policy documents and spoken utterances and theories-in-use, the image of organizational processes and structures and the causal relationships among them that are held by organization members. When such mental representations are widely held among members, organizational learning capacity is greater. Knowledge in organizations is also captured by organizational routines, codes, documents, stories, jokes, and other symbolic representations. Theorists differentiate between locally created or ‘generative’ knowledge and knowledge acquired from the environment or ‘adaptive’ knowledge.
• Actions versus thoughts Some organization theorists maintain that organizational learning is reflected in the ‘change in the range of potential organization behaviors’ (Huber, 1991), but others argue that learning occurs through repetitive error detection and correction, and as such cannot occur unless observable organizational actions are apparent (Argyris, 1993).
• Levels of learning Low level, incremental, or ‘single loop’ organizational learning occurs when the organizational response to stimulus for change is manifest in attempts to build upon existing mental conceptions of operations and their consequences. High or ‘doubleloop’ learning occurs when organization members surface, articulate and reflect on deeply held assumptions about purposes and processes (Argyris and Schön, 1978; Fiol and Lyles, 1985; Lundberg, 1989). This sort of learning is non-incremental because the organizational response will occur within a newly formulated ‘mental map’. At an even higher level, one might consider the organization’s capacity to learn how to learn, or what Argyris and Schön (1978) call ‘duetero learning’.
• Structural versus interpretive influences Organizational learning theorists maintain that an organizational response to stimuli for change can and, depending on circumstances, will vary from highly rational, deductive and logical to highly interpretive, non-linear and non-rational (Daft and Huber, 1987; Lovell and Turner, 1988). This perspective provides a better fit with what we know about how organizations operate and is a distinct improvement on the highly rational image of organizations portrayed in much of the program evaluation literature, for example.
• Organizational memory A significant feature associated with organizational learning is the organization’s capacity to order and store information for future retrieval and, indeed, its capacity to retrieve desired information as the need arises. Organizational memory and production systems are held by organization members and thus susceptible to rapid decay with personnel turnover and forgetting and by physical record keeping and management information systems set up to perform the function (Levitt and March, 1988; Simon, 1991; Tiler and Gibbons, 1991). While the efficacy of storage and retrieval systems is generally regarded as a key dimension in explaining organizational learning, an organization’s ability to ‘unlearn’ is also viewed as being critical (Hedberg, 1981; Nystrom and Starbuck, 1984).

Organizations such as schools can engage in a variety of strategies and processes designed to enhance organizational learning capacity and generative and adaptive knowledge bases. Strategies designed to enhance a school’s generative knowledge might include, for example, local experimentation or systematic trial and error; ongoing monitoring of performance; simulation and gaming; and general strategies designed to improve internal communications. Strategies designed to enhance adaptive learning, on the other hand, include personnel recruitment; general and focused searches of the school’s environment; vicarious learning by observing other schools through, for example, inter-school personnel exchange and visitation; and imitative or mimetic learning (copying). Some strategies such as program evaluation and needs assessment can be thought of as organizational strategies designed to add to either generative or adaptive knowledge bases.

Collaboration as the Key

The implications for schools of moving toward an image of professional and organizational inquiry-mindedness are considerable. Key to this movement is a fully developed conception of teachers’ joint work reflected by collaborative curriculum decision making; genuine and direct participation in non-curricular, managerial or organizational governance processes; frequent collegial exchange; and the general dissipation of norms of isolation and privacy when it comes to classroom-based activities. Zahorik’s (1987) observation that ‘collaboration stops at the classroom door’ (p. 391) would give way to norms of professional sharing, joint implementation, and collegial observation and feedback. But if educators accept this image and embrace it as worthy of pursuit they will find the transition to contemporary norms to be fraught with obstacles and challenges.

Researchers in many countries have described the persistence in schools of teacher norms of privacy (Little, 1990), noninterference (Feiman-Nemser and Floden, 1986; Huberman, 1990; Lortie, 1975; Nias, Southworth and Yeomans, 1989), individualism (Hargreaves, 1990), and lack of commitment to opportunities for school-wide decision making (Duke and Gansneder, 1990; Duke, Showers and Imber, 1980; Hallinger, Murphy and Hausman, 1991). These conditions are not particularly consistent with the establishment of collaborative culture as a vehicle for school reform. Huberman (1990) characterized teachers as ‘tinkerers’ who operate independently in adherence to norms of noninterference, and rely more on their personal, practical knowledge in thinking about their teaching than on interaction with peers. Leinhardt and Greeno (1986) amplify the notion of teacher independence by elucidating expert teachers’ preference for on-the-spot decision making with minimal pre-planning. Hargreaves (1990) developed an explanation of persistent teacher isolation based on the merits of individualism rooted in an ethic of care and service. The demand on teachers’ time away from class was suggested to be a sufficient deterrent to collaboration. Some researchers (e.g., Campbell and Southworth, 1990) even go so far as to say that teachers are ill prepared to collaborate and lack the capacity to work in groups.

Some critics of collaborative activity as a reasonable route to desired change in schools, have framed their arguments in terms of reward structures, suggesting that collaborative work will diminish intrinsic rewards available to teachers. Intrinsic rewards are defined as feelings of satisfaction arising from personally meaningful intangibles such as pride in student achievement, collegial stimulation and support, the glow of service, and enjoyment of teaching activities (Feiman-Nemser and Floden, 1986). Extrinsic rewards are defined as organizational mechanisms for benefiting individuals, such as pay awards, and promotion to positions of added responsibility that confer prestige and/or power. Such rewards are virtually nonexistent in the ‘flat’ teaching career (Feiman-Nemser and Floden, 1986). Indeed, leadership strategies focused on the distribution of extrinsic rewards (Blase, 1990; Sergiovanni, 1989), including merit pay and career ladder systems installed to enhance teachers’ performance through incentives have met with less than satisfactory results (Bacharach and Conley, 1989; Shedd and Bacharach, 1991; Tyack, 1990). What is not clear is whether intensified teacher-teacher interaction will act to curtail the availability to teachers of intrinsic rewards, or, indeed, to enhance them.

Some would argue that under the right conditions, teachers’ joint work may either enhance the availability of intrinsic rewards for teachers or provide an additional source of them (e.g., Feiman-Nemser and Floden, 1986). Empirical support for this proposition is beginning to accumulate. In an interview study, for example, Lytle and Fecho (1989) reported that teachers involved in a cross-visitation program found their own classes more intellectually challenging, changed their own routines, tended to learn more from students, received validation of their own skills, became more reflective, and improved their view of the teaching profession. Kushman (1992) came to similar conclusions in stating that, ‘Rewards were derive...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• List of Tables and Figures
• Foreword
• Preface
• Part 1 Why Participatory Evaluation?
• Part 2 Participatory Evaluation in Schools and School Systems
• Part 3 Educational Participatory Evaluation: Variations in Form
• Part 4 Themes and Conclusions
• Exercises
• Notes on Contributors