Measuring the Quality of Education

Paul Vedder

• Publisher: CRC Press
• Year: 2020
• ISBN:
  9781000153965
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Engineering General
• Index: Technology & Engineering

This book contains a selection of articles on measuring the quality of education from the perspective of the importance of theories on education, changing effects of education, curriculum dependent or curriculum independent measurement, product and process evaluation, and global curricula.

Measuring the Quality of Education

An Introduction

Paul Vedder

Institute for Educational Research
The Hague, The Netherlands

This book contains a selection of contributions to a conference on measuring the quality of education which was organized on the occasion of the 25th anniversary of the Institute for Educational Research in the Netherlands (SVO) in 1990.

A global topic

Quality of education is currently an important issue in countries all over the world. In this context, special attention is being given to measuring the quality of education both at national and international level. The Education for All Conference in Thailand in 1990, which was organized by UNESCO together with UNDP, UNICEF and the World Bank, stressed the importance of national assessment studies for monitoring changes in school quality. OECD countries are working on the development of educational indicators for the registration of school performance over time and in various countries. At a more regional, European level the European Commission is interested in standardized school assessment practices throughout Europe. Finally, there is of course the International Association for the Evaluation of Educational Achievement (IEA), which is expecting to make significant progress in mapping school achievement across nations.

The measurement of educational quality is enjoying such great interest because it is seen as a step in the identification of problems as well as a first step towards solving problems and improving education. In this book a variety of national initiatives are described, such as the national assessment system in England and Wales (Whetton; chapter 4) and in Finland (Kangasniemi; chapter 7). Two contributions deal with the measurement of quality at the international level. Rissom (chapter 9) presents the UNESCO perspective. He stresses the importance of assessment studies for monitoring changes in school quality in developing countries, suggesting several ways to facilitate the participation of developing countries in international assessment programmes. Plomp and Loxley (chapter 10) describe how the International Association for the Evaluation of Educational Achievement (IEA) may serve the interests of both developed and developing countries.

The relevance of measuring the quality of education

On a global scale there is a steady increase in participation in formal schooling, reflecting trust and belief in the value of education. Schools are seen as instruments contributing to social and economic progress; they are expected to ‘produce’ more and more well-educated individuals, which is to lead to higher incomes, not only for the individuals concerned, but also for the community of which they are part. Moreover, schools help individuals to become self-supporting in a society which becomes increasingly demanding. The more successful schools are in making people self-supporting, the less resources need to be made available for welfare organizations that take care of those who are unable to support themselves. As long as this trust and belief in the virtues of education dominate the debate on education in a country, the function of quality assessment will not go far beyond the school. In these cases, assessment will mainly be used for grading purposes and for informing students and parents about learning progress.

People in countries with high rates of participation in education, however, have lost something of this trust and belief in the virtues of education. Many have started to doubt the efficiency and effectiveness of schools (California Joint Committee on Educational Goals and Evaluation 1971; Lundgren 1987). Current discussions on the quality of education follow from the experience that good schooling does not necessarily guarantee a well-paid job and that more schooling in a society does not always lead to greater prosperity (Buis, Kloprogge, Vedder & Van der Werf 1990; Jules 1988; Lowe & Istance 1989). In other discussions it is pointed out that some of the knowledge and skills that are taught and learned in schools are hardly useful for solving problems outside of school (Bereiter 1990; Brown, Collins & Duguid 1989; Resnick 1987). In discussions on functional literacy, for instance, it can often be heard that schools in western societies succeed in teaching the technical aspects of reading, but fail in teaching reading comprehension or skills needed to analyze texts critically. The seriousness of this problem was stressed at the sixth European conference of directors of educational research institutes, organized by the Council of Europe in collaboration with UNESCO, which had this topic as its theme. Schools, moreover, are found to be insufficiently concerned with the teaching of skills and attitudes involved in effective communication, problem identification, coping, cooperation and self-confidence (Anderson 1990; Raven 1981, 1988).

Measurements of educational quality play an important role in the identification of these problems and in determining their extent and their intensity. The results of such measurements may moreover serve as an input for decisionmaking in respect of school improvements. Not every measurement has relevance in this respect, however. The relevance of a measurement is determined by the place of the measured object in a complex educational accountability system.

Accountability in education refers to the notion that schools, school boards and educators are held responsible for what they achieve or fail to achieve in schools. In order to inform administrators or financers about school outcomes, school boards and educators should perhaps be obliged to compile reports on their own functioning, or allow others to do so, for instance in the form of national assessment studies. On the basis of such accounts, administrators or financers might then decide to change schools, school staff or the administrative system. Schools, teachers and school boards are thus seen to have a responsibility towards administrators or financers. Viewed in this way, accountability seems a fairly simple concept, based on a clear distinction between those in power and those that respond to the demands of power. In practice accountability is a much more complex process, in which the distinction between those in power and those who respond to the demands of power is not always clear. A more realistic approach, therefore, would regard the issue of power in terms of interaction and dialogue in a continuous political discussion between a variety of groups or forces in education, each having particular interests in publicly-financed education (cf. Vickers 1971). Students, parents, teachers, school boards, employers, religious organizations, pressure groups and the state are the most important groups that participate in this discussion. Each group, however, comprises subgroups representing some ethnic or ideological perspective and pursuing particular objectives in respect of the organization and outcomes of education.

The complexity of the accountability structure is an important reason why it is difficult to discuss “quality of education” at a concrete level. What is meant by “quality of education” is always related to the specific goals held or represented by a particular group. Consequently, the relevance of any measurement of the quality of education is determined by the goals that it covers, and by the weight these goals carry in the educational accountability process. The relevance of a measurement is not always evident before the measurement is conducted; quite often, it will not become clear until after the measurement has been completed, in a discussion on the outcomes. It goes without saying that the obvious persons to clarify the relevance of a measurement in the wider context of the accountability process are the scientists who program, execute and report measurement projects.

Rissom (chapter 9) gives an etymological description of the concept of educational quality. The quality of education, he says, is linked to people, to how they perceive education, and to how they organize it. Shavelson and Baxter (chapter 3) make a distinction between school quality, curricular quality, teaching quality and instructional quality. This is a helpful distinction, which accentuates the complexity of education and the need to give due attention to a variety of aspects of education when dealing with the measurement of its quality. Still, as stated earlier, to give body and content to the concept of “quality of education” it is first of all important to identify the goals that are to be achieved through education and the place of these goals in the educational accountability process. Both Hofstee (chapter 2) and Broadfoot (chapter 8) write about accountability in education and its potential effect on the organization of assessment practices, the organization of education and learning. They suggest that the quality of education has to be created and shaped every day again in every new educational setting. In Hofstee’s words, it is a “creative process of ongoing redefinition”. These authors, then, view educational quality in terms of the educational process that goes on inside schools. In fact, they seem to sound a warning against connecting the quality of education with specific goals and the weight these goals have in an accountability process. Hofstee even argues that any attempt to give “quality of education” a clear definition or a clear shape in measurement procedures is likely to give rise to rigid and restrictive views of quality which jeopardize the hard-to-grasp and dynamic practices to which it refers. Both authors seem to opt for a view of education as a sheltered system, free from pressures from other institutions. In other contributions the accountability system and the weight of educational goals seems to be taken for granted. Kangasniemi (chapter 7) provides a mainly technical description of developments in assessment practices in Finland. A similar approach is adopted by Whetton (chapter 4), who describes new assessment procedures in England and Wales, and by Plomp and Loxley (chapter 10), who discuss IEA assessment projects. Interestingly enough, however, the latter also dwell on aspects of accountability systems, emphasizing that assessment may have different functions in different situations. Aleva (chapter 6) refers to a new educational act that justifies a search for quality of education in terms of students’ social competencies. Shavelson and Baxter (chapter 3) stress the importance of taking account of the place of goals in an accountability structure. Taking goals and assessment procedures in mathematics and science as a point of departure, they suggest that implementing new test procedures may result in changes in the achievement accountability system which will eventually result in educational reform. Scheerens’ contribution (chapter 5) bears even more directly on the discussion concerning the definition of quality of education in terms of educational goals and their weight in the accountability structure. He describes the quality of education from the perspective of the legitimacy of goals of education. In the final chapter Van der Kamp gives some suggestions for the empirical study of goals of education.

The theme that recurs throughout this book is “measuring the quality of education is possible, but we have not yet managed to do so in a satisfactory manner.” Although the majority of the contributions address problems connected with measuring the quality of education, not all describe or stress the same problems. This is first of all due to the breadth of the topic itself, which comprises a wide variety of quite distinct subtopics. Another reason is that the measurement of the quality of education currently figures high on the political agenda of many countries. The authors do not only analyze and discuss this political aspect of educational quality, but some are also quite explicit about their own political stance in the matter.

In the remainder of this introductory chapter I will discuss the contributions in this book from the perspective of five themes:

1) the importance of theories on education and learning;

2) measuring the changing effects of education;

3) curriculum-dependent and curriculum-independent measurement;

4) product evaluation and process evaluation; and

5) global measures and global curricula.

These themes represent foci of attention in the ongoing discussion on measuring the quality of education. By placing the contributions in the context of this ongoing discussion, I hope to clarify and contribute to their significance.

The importance of theories on education and learning

The importance of theories and theory development will be considered for two distinct types of problem. The first is not so much related to measurement in a technical sense, but rather to the problem of theoretical instruments and approaches needed for improving the quality of education. This problem has been presented by Lundgren (1983). He suggests that notions about how to improve education are often no more than reflections of current educational practice and therefore do not provide a suitable basis for educational reform, nor for solving serious problems in the education system. He even suggests that these notions might be part of the problem. Particular pedagogical practices, he claims, sustain or even generate learning problems. If initiatives for improvement are defined in terms of the same practices, they will not help to solve problems; they will only make the situation worse. Examples and empirical evidence of this negative effect of assessment may be found in a recent study commissioned by the commission of European Communities (Weston 1990). Clifford (1990) presents another example with other evidence. In a review of studies on aversive effects of motivation practices she concludes that rewards for initially-liked tasks reduce intrinsic motivation, that extrinsic incentives impair learning and stimulate students to select easy tasks, and that strong extrinsic regulation is associated with low mastery-orientation. She states that “it is the excessive, exclusive, and inappropriate use of these techniques that best accounts for many of the motivation problems characterizing our schools” (p.61). I have already referred above to children’s problems in using skills and knowledge learned in school in out-of-school situations and to the fact that few schools are addressing important goals such as teaching pupils to communicate effectively, to be prepared and able to identify problems and to take initiatives to solve them. These problems cannot be seen nor solved from the perspective of everyday school practice and common sense educational theory. According to Lundgren, there is a need for a wider understanding of how educational processes are formed. In order to create new educational practices, we must start by questioning our present ideas about education. The resulting notions may serve both as instruments for more profound analyses of educational problems and as a starting point for the development of new measurement procedures.

The contributions in this volume that are most clearly in favor of a search for new perspectives on the quality of education are those by Hofstee, Broadfoot and Rissom. Hofstee stresses how the non-specialized public may help to define educational theories and function as a reference group that scientists may find useful in the evaluation of their educational theories. Broadfoot clarifies the background of measurement practices in England. She criticizes these practices by describing their negative effects and presents some stimulating ideas for new approaches that might be more appropriate to contemporary demands for educational quality and its measurement. Rissom presents a view of education in developing countries, pointing to the relevance of non-formal education. How the study of non-formal education may lead to new, or redefined approaches to learning and instruction has been made clear in the work of scholars such as Lave (Lave 1977; Rogoff & Lave 1984).

The second aspect of the need for theory development in the context of measuring educational quality is more of a technical nature and relates to the measurement of progress in learning on the basis of the psychological concept of “developmental dimension” and its psychometric counterpart “dimensionality”. A developmental dimension refers to a particular quality characterizing a person’s activities during a longer period which may have a variety of manifestations or intensities of expression that change over time (cf. Wohlwill 1973). Given the variety of manifestations or intensities of expression, a developmental dimension may be conceived of as a reference point which gives a particular idea of constancy or durability. The notion of a developmental dimension allows a distinction to be made between a change of activities that reflects developmental progress or regression and a change of activities that is...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Measuring the Quality of Education; An Introduction
• Independent Educational Assessment in the Interest of Educational Quality
• On Measuring the Quality of Mathematics and Science Education: Achievement Indicators
• Context-based Measures for the Quality of Education Examples from England
• Evaluating Non-cognitive Aspects of Education
• Assessment Methods for Social Competence in Elementary School
• National and School Level Assessment in Finland
• Measuring the Quality of Education for Parents and Students
• The Search for Quality in Education: Some Comments on the International Dimension
• TEA and the Quality of Education in Developing Countries
• Some Issues and Problems in Measuring the Quality of Education