Social Justice and Educational Measurement

John Rawls, the history of testing, and the future of education

Zachary Stein

• Publisher: Routledge
• Year: 2016
• ISBN:
  9781317366911
• Edition: 1
• Topic: Éducation
• Subtopic: Éducation générale

Social Justice and Educational Measurement addresses foundational concerns at the interface of standardized testing and social justice in American schools. Following John Rawls's philosophical methods, Stein builds and justifies an ethical framework for guiding practices involving educational measurement. This framework demonstrates that educational measurement can both inhibit and ensure just educational arrangements. It also clarifies a principled distinction between efficiency-oriented testing and justice-oriented testing.

Through analysis of several historical case studies that exemplify ethical issues related to testing, this book explores and propounds speculative design principles and arguments in favour of radically democratic school reforms, which address how the future of testing might be shaped to ensure justice for all. These case studies cover the widespread use of IQ-style testing in schools during the early decades of the 20th century; the founding of the Educational Testing Service; and the recent history of test-based accountability associated with No Child Left Behind.

Social Justice and Educational Measurement will be essential reading for academics, researchers and postgraduate students in education, testing and assessment, and the philosophy of education. It will also be of interest to policymakers and educational administrators.

1 Social justice and institutionalized measurement

Inequality before law implies unequal laws or rights in relation to measures: some people decree them, others have to put up with them; everyone has a measure of his own, the strong imposing theirs on the weak. The measure is not impersonal but rather human; it belongs to some, it does not belong to others, and it is dependent upon the will of whoever has the power to enforce it. (Kula, 1986, p. 122)

The great historian of measurement, Witold Kula, points out connections between justice and measurement that are easy to grasp. Imagine a farmer who is taxed a certain number of bushels of grain depending on the size of his land, yet each year both the size of the bushel and the measures of land size change to meet the needs of the local magistrate. Or imagine a marketplace where different vendors use different scales for weighing the same materials and change scales depending on who is buying. Consider being in a local community that has used traditional measurement practices for centuries in the distribution of land, only to have a centralized governmental body enforce the use of newer scientific measures. Where land was once distributed according to a measure that combined both size and probable yield, it is now distributed according to a universal standard for determining area, which is indifferent to the local variations in soil quality that accounted for the stability of the traditional practices. Finally, think about having properties of your own body and mind measured by officials who use the results to determine your eligibility for certain social benefits, yet these systems of measurement are demonstrably biased, scientifically specious, and susceptible to corruption.

These examples from history demonstrate the relationship between systems of institutionalized measurement and social justice. In post-industrial societies, universal standards for physical weights and measures are taken for granted – a metre or a litre is the same for everyone everywhere. But this is a relatively recent state of affairs, and it took centuries, thousands of scientists, and several political revolutions to bring this about (Kula, 1986; Tavernor, 2007). In fact, for the majority of humanity’s civilized existence a debate has raged about the relationship between justice and measurement, creating an archetypal relation best exemplified by the mythical figure Lady Justice, who wears a blindfold and holds a scale, one of the most ancient instruments of objective measurement.

Today, newly created measurement systems are rekindling deep concerns about the relationship between justice and measurement, especially in areas like biometrics, econometrics, and psychometrics – the focus of this work. The social justice issues raised by these new measurement systems have strong analogies to those raised in the past concerning our most basic systems for physical measurement. For this reason I begin building an ethical framework for educational measurement by considering ethical issues in historical metrology, which is the historical study of physical measures and measurement practices.

This chapter is an introduction to John Rawls’s theory of justice by way of certain themes in historical metrology. Rawls offers a set of key concepts as a part of his ethical framework, specifically the idea of society’s basic structures and the principles of justice that ought to guide their design. I will explain this ethical framework with reference to the history of measurement practices in order to clarify the structure of problems at the interface of social justice and institutionalized measurement systems. This is preparation for the central argument that unfolds over the rest of the work, where I address the relationships between social justice and educational measurement in modern societies.

First I compile a set of observations and generalizations about the basic functions of measurement systems in society. I argue that they form an important part of any society’s basic structure, playing a key role in the system of institutions that fundamentally shape social life. A society’s basic structure determines the scope of its members’ liberties and rights, as well as the distribution of the basic goods that result from social cooperation. According to Rawls, these basic structures are what theories of justice should be about. A theory of justice is a framework for determining how to build basic structures that are fair – structures that will create the kind of ‘background justice’ that enables trust, equality, and autonomy. It is easy to see that measurement infrastructures are undoubtedly fit to be the focus of theories of justice. They form a part of basic structures that affect a wide variety of basic goods, including everything from food and money, to job opportunities, healthcare, and self-esteem. As historical metrology shows, justice and measurement have been mutually defining terms since the dawn of civilization.

Having clarified the role of measurement infrastructures as basic structures, I then explore what it means to consider measurement infrastructures in light of a broader theory of justice. Rawls offers a set of philosophical principles that clarify the nature of justice, defining it as fairness in the arrangement of basic structures. He proposes a framework for making decisions about the fairness of basic structures, providing a method for adjudicating between just and unjust arrangements. I demonstrate how this framework works, and specifically how it can be used to survey the ethical complexities of institutionalized measurement systems. Using examples from historical metrology, I show how Rawls’s theory clarifies what constitutes a just use of institutionalized measurement. I bring these insights into Chapter 2, where I turn to focus on educational systems as basic structures, and begin building a Rawls-inspired framework addressing educational measurement.

Measurement infrastructures as basic structures of society

We enter into them by birth and exit only by death … the institutions of the basic structure have deep and long term social effects and in fundamental ways shape citizens’ character and aims, the kinds of persons they are and aspire to be. (Rawls, 1996, p. 68)

In complex societies social action is a highly coordinated affair, involving economic, legal, and political systems. This network of institutions congeals into a basic structure, which sets the terms of social cooperation and distributes the advantages that result from it. According to Rawls, the basic structure of a society establishes the ‘background justice’ that conditions and shapes the lives of each member. Not every social structure is a part of the basic structure because not every institution has deep and pervasive effects on the shape of society. Basic structures are those that touch all members in some way, especially determining their access to basic rights and goods. In the above quote Rawls clarifies why these structures should be the primary focus of ethical frameworks concerning social justice. they set the conditions in terms of which the actions of all individuals, groups, and associations take place. No matter how free and fair a specific interaction between people appears to be, we cannot say it is just without understanding the broader social institutions in which it occurs. This was a lesson learned clearly in the segregated American South, where ostensibly fair and uncoerced interactions at ‘separate but equal’ businesses and schools were, in fact, reinforcing unjust legal structures that grossly distorted human relationships. ‘Thus we seem forced to start with an account of a just basic structure. It is as if the most important agreement is that which establishes the principles which govern this structure’ (Rawls, 1999, p. 257).

Rawls argues that fairness ought to govern the design of all basic structures. This is because basic structures are a non-negotiable precondition of life in a complex society. We do not opt in to or join up with society; we are always already members of it, and will always live in terms of at least some of its basic structures. Of course, individuals can emigrate between societies, but they cannot return to a state of nature. We participate in basic structures of one type or another from the day we are born until the day we die. We participate in them and conduct our lives according to them, yet we did not choose them – we just happened to be born in a certain time and place. They shape our fate as if they are a part of nature, yet they are social constructions (Searle, 1995). Therefore, the arrangement of these basic structures is of special ethical concern, especially their bearing on the life prospects of individuals belonging to different groups, be those economic, generational, religious, or what have you. So while there are many acceptable ways to design institutions that people can freely choose to join, such as a club (which charges for membership) or a scientific association (which excludes non-experts), when an institution is a part of a basic structure that everyone must participate in, different organizational principles should be applied. According to Rawls, justice must be the dominant design principle for institutions, such as legal systems, tax codes, and educational systems, which shape the very fabric of social life by structuring the terms of collaboration and the distribution of benefits.

Measurement practices were some of the first social institutions to function as basic structures and to become the subject of theories about social justice. Measurement practices exemplify what basic structures are and how they function to create the background justice of a society. Lessons from historical metrology will clarify the relation between measurement practices (understood as basic structures) and social justice. This sets the stage for an exploration of the ways in which educational measurement practices have many of the same social justice implications as physical ones, serving as basic structures, as a part of legal codes, and being implicated in the distribution of basic social goods.

On the origins of measurement infrastructure and their role as basic structures

It has been argued that the ancient Egyptians first invented standardized measures in tandem with geometry in order to redistribute land and levy taxes each year after the Nile’s annual flood destroyed the previous year’s plots. While the true origins of geometry may lie elsewhere, it is undoubtedly true that the first standardized measurement practices were invented in response to pressing social needs (Duncan, 1984). Measurement practices – involving measurement instruments and norms for their use – are as old as civilization itself and were some of the first social institutions ever established. Scales and measuring rods have been found among pre-civilized humans, and all ancient civilizations had complex systems of weights and measures. Historical accounts tell of an astonishing diversity of pre-modern measurement practices, all of them built out of necessity and evolving in response to the needs of their creators. Peoples in regions where land was scarce had precise measures of area, whereas peoples in regions with abundant land had more approximate measures. Those who dealt in gold had complex systems of precise scales for trading, while those dealing in oats or hay had systems of bushels and baskets, no less complex, but certainly less precise. Nearly all ancient measures were anthropocentric, often literally involving a part of the human body (such as a thumb or foot), and they were all created to serve cognitive, practical, and communicative needs (Kula, 1986).

Measurement systems emerged and have been institutionalized to address recurring social situations in which coordinated action depends upon the creation of a shared understanding of specific qualities and quantities in the objective world. Innovations in measurement stem from situations in our social life where it is necessary to achieve mutual understanding about a state of affairs that is repeatedly problematic, yet also consistently objectively determinable. These types of situations have a similar epistemological structure. They require multiple parties to be able to verify the amount or quality of some thing or things that concern them. Measurement systems are part of those social practices that require the reliable differential determination of objective traits in objects of concern. Considering even the most basic instance of measurement bears out this account.

I measure a wooden board by myself as I build a table, say by laying down the length of my arm from elbow to finger tips and marking it as one ell. Through the act of measurement I have positioned the board in a space of meaning to which I ascribe universal intersubjective validity. That is to say, in the practice of measuring the board I am, in effect, saying that anybody and everybody who measured this board this way would find the same thing. Of course, my forearm is longer or shorter than yours. So as soon as one friend shows up to help me build, we are thrown into a negotiation about how precise an ell we need and whose arm it will be if a judgment must be made. This begins a process of refinement that, over the long run, results in a tape measure, marked in both metric and United States customary units, which any modern table builder would use off the shelf without a second thought. Usually, commonly used measures slip into the background and become part of our taken-for-granted measurement infrastructures. They become an unquestioned condition for the possibility of a vast amount of highly coordinated social actions.

Measurement infrastructures form a part of society’s basic structure because they shape social life in fundamental ways, specifically by providing a means for coordinating social action in relation to objective realities. Consider the measurement practices involved in scientific research, engineering, and economic exchange, or in the administration of basic governmental tasks, such as taxation. Measurement practices can facilitate these complex social activities because they provide a reliable index of reality that has been codified to consistently generate a broad consensus, ideally universal. Thus measurement infrastructures, like legal infrastructures, are both systems of knowledge and systems for guiding action and administering conduct (Habermas, 1996). They require knowledge about the invariant properties of objects and occurrences, which in turn entail the creation of instruments and practices that reliably differentially respond to those properties. Reflectively (sometimes scientifically) codified measurement practices come to structure broad swathes of social life, often to the point of being woven into systems of law. Collections of measurement practices often congeal into an infrastructure and comes to function as a taken-for-granted aspect of social life, so much so that unjust systems of measurement have been perpetuated for centuries due to sheer force of habit. This was the case with some systems used to administer taxation under feudalism in medieval Europe, a process of social inertia often aided by the rule of law (Kula, 1986; Wallerstein, 1974).

History shows that measurement infrastructures have been, and continue to be, invented and institutionalized for social uses and wedded to systems of law. Aristotle’s research for his Politics included a comprehensive survey of the existing city-states’ constitutions, and while the full fruits of his research were lost, what remains suggests that measurement practices, especially in the market and the field, were a major concern for those seeking to administer justice in the ancient world. It is no coincidence that the first ancient systems of measurement were accompanied by the first legal codes, nor that legal systems and measurement systems have co-evolved since the dawn of civilization (Duncan, 1984). In fact, measurement practices were some of the first institutions to qualify as what Rawls would call basic structures of society.

In Aristotle’s political anthropology, he recounts the societal need for metronomi (commissioners of weights and measures), to be appointed by lot, and tasked with ‘seeing that sellers [in the market] use fair weights and measures’ (Constitution of Athens, Ch. 51: see Ross, 1921). ‘There were also the Sitophylaces (corn commissioners) … who watched over prices and the weights of loaves [of bread sold at market], which they had the power to standardize’ (Duncan, 1984, p. 13). The earliest legal systems functioned to establish, among other things, measurement infrastructures as basic structures by using the force of law to assure that specific measurement practices would be reliably regarded as the ‘true measure’. Like other basic structures, such as those for voting and jurisprudence (which are also the subject of ancient constitutions), the institutionalization of measurement practices serves to facilitate trust between strangers, mutual understanding at a distance, and fairness through the standardized treatment of cases. But this same constellation of law and measurement can be, and has been, used to exploit, systematically discriminate, and fallaciously justify inequity.

Consider the result of different laws governing the jurisdiction and powers of ancient and medieval commissioners of weights and measures, who roamed the marketplaces attempting to regulate the use of metrics, e.g., scales, rods, bushels, and loaves. Kula (1986) reports of wide variations between markets that were only miles apart, with ethnic and religious differences often resulting in different rights with respect to measurement. Wealthy merchant guilds found ways to change legal codes in order to allow them the power to regulate their own measures, which were then able to be changed in response to the needs of the guild. When yields were good, the bushel (or whatever was the standard unit of exchange) is big; when yields were poor then the bushel was small, yet the price of the bushel, its exchange value, remained unchanged (Kula, 1986, pp. 43–71).

This kind of metric manipulation was common practice and remained a ubiquitous part of economic life for centuries. It is important to understand how unit setting differs from price setting as a means for merchants to offset unexpected problems with supply or production. This is an issue we will return to when we discuss the manipulation of educational measurement systems, where units (e.g., cut-off scores) are s...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• Preface: the personal is political
• Introduction: tipping the scales – social justice, the philosophy of education, and standardized testing
• 1 Social justice and institutionalized measurement
• 2 Social justice and education
• 3 A theory of just educational measurement
• 4 Social justice and the origins of educational measurement
• 5 Social justice and the rise of national testing infrastructures
• Conclusion: social justice and the future of testing
• Index