The PISA Effect on Global Educational Governance

Louis Volante

• Publisher: Routledge
• Year: 2017
• ISBN:
  9781315440507
• Edition: 1
• Topic: Education
• Subtopic: Education General

The Program for International Student Assessment (PISA) is an international achievement measure that assesses 15-year-old student performance in the areas of reading, mathematics, and science literacy in over 70 countries and economies triennially. By presenting an in-depth examination of PISA's role in education governance and policy discourses, this book provides the reader with a critical analysis of the educational change process within our increasingly global educational policy environment. Exploring the prominent socio-political drivers of large-scale educational reform across the globe, chapter authors examine PISA's national and global implications from a diverse range of regional contexts. Through the presentation of cross-disciplinary viewpoints and topical issues related to the PISA international survey, this volume explains the degree to which PISA-focused research is linked to national educational policy discourses and international education agendas.

Part I
Introduction

1
PISA, Policy, and Global Educational Governance

Louis Volante and Xavier Fazio

Introduction

The promotion of education policies, priorities, and large-scale reforms by international and transnational organizations has become a contentious topic around the world. Policymakers and academics alike have articulated key arguments for and against the expanding influence of international organizations such as the World Bank, European Union, and the Organization for Economic Cooperation and Development (OECD) on national education systems (Volante, 2016a). The impact of the policies promoted by these international and transnational organizations are welcomed by some academics, but have also been heavily scrutinized, particularly by scholars within the educational academic community. Although an analysis of the impact of education policies promoted by each of the previous organizations is a worthy endeavor, this edited volume will focus exclusively on the OECD. This organization has gradually expanded the scope of its mandate over the last 50 years to include education research and policy activities which are exemplified by the influential Program in International Student Assessment (PISA). Since its inception, the PISA triennial survey has been fiercely debated for the role it has played in influencing educational policy and governance issues.

Proponents of the PISA triennial survey of reading, mathematics, and scientific literacy underline its important role in spurring public education system improvements through the lessons learned from top-performing nations/economies (Schleicher, 2009). Essentially, PISA results have traditionally been promoted as an instrumental way to help governments “borrow” policies from effective education systems, often referred to as “reference societies,” in the hopes of emulating their high-achieving success within their own national education context (see Morgan & Volante, 2016; Sellar & Lingard, 2013). Critics counter system improvement claims by arguing that the expanding influence of the PISA triennial survey has eroded the autonomy of nation-states and promoted the homogenization of educational systems around the world (see Andrews et al., 2014; Meyer, 2014; Lingard, Martino, Rezai-Rashti, & Sellar, 2016). Others promote a slightly more nuanced position by noting the potential opportunities afforded by PISA—provided greater caution is exercised when deriving and applying evidence-based policy; the benchmarks’ testing and methodological limitations are more broadly acknowledged; and reductionist approaches to policy reform are avoided (see Araujo, Saltelli, and Schnepf, 2017; Volante, 2016b). Before summarizing the full range of arguments for and against the utilization of the PISA in international policymaking spheres, a brief synopsis of the rationale and defining characteristics of this benchmark measure is warranted.

The PISA Benchmark Measure

PISA is a triennial international survey which aims to evaluate education systems worldwide by testing the knowledge and skills of 15-year-old students. The average age of 15 was chosen because in most OECD countries students are nearing the end of compulsory education. The OECD asserts that PISA is unique in the way it looks at public policy; literacy; and lifelong learning. With regards to public policy, they note that governments, principals, teachers, and parents all want answers to questions such as “Are our schools adequately preparing young people for the challenges of adult life?,” “Are some kinds of teaching and schools more effective than others?” and “Can schools contribute to improving the futures of students from immigrant or disadvantaged backgrounds?”

In terms of literacy, the OECD contend that PISA looks at students’ ability to apply knowledge and skills in key subject areas and to analyze, reason, and communicate effectively as they examine, interpret, and solve problems rather than master discrete knowledge prescribed in school curricula. With respect to lifelong learning, they argue that students cannot learn everything they need to know in school. In order to be effective lifelong learners, young people need not only knowledge and skills, but also an awareness of why and how they learn. PISA both measures student performance in reading, mathematics, and science literacy, and also asks students about their motivations, beliefs about themselves, and learning strategies. It is worth noting that each of the previous points was taken almost verbatim from the OECD’s website (see www.oecd.org/pisa/aboutpisa/).

Not surprisingly, the OECD asserts that the key knowledge and skills measured by their PISA triennial survey are essential for full participation in modern societies (OECD, 2014a). According to the director of PISA, Andreas Schleicher, PISA results in reading, mathematics, and scientific literacy provide “policy-makers and practitioners with helpful tools to improve quality, equity and efficiency in education, by revealing some common characteristics of students, schools and education systems that do well” (Schleicher, 2007, p. 356). Indeed, one of the key objectives of the OECD moving forward is to “strengthen the policy relevance and analytical power of PISA, including establishing best practice for linking PISA with national assessments” (OECD, n.d., p. 8). For the most part, participating nations/economies have accepted the previous claims and are increasingly turning to PISA as the standard metric upon which they judge their relative standing against their international counterparts and perhaps more importantly, the effectiveness of their national education policies and large-scale reform efforts (Volante, 2015).

An important feature of PISA is that each of the literacy domains: reading, mathematics, and science are assigned a major testing domain on a rotating basis, and as a result are assessed in greater detail every nine years. To date, there have been six cycles of PISA: 2000, 2003, 2006, 2009, 2012, and 2015—with each literacy domain assessed as a major focus twice. In addition to reading, mathematics, and scientific literacy domains, the OECD has also introduced new PISA measures to assess skills relevant to modern society, such as creative problem solving and financial literacy (introduced in 2012), and collaborative problem solving (introduced in 2015). Each of the PISA measures is accompanied by student and principal questionnaires that provide background information about the pupils, their attitudes to learning, and their home and school learning environments. These questionnaires play a pivotal role in making connections between student achievement results and the individual characteristics of students and their schools. Lastly, it is worth noting that PISA test items and questionnaires are developed under the responsibility of the PISA Governing Board, which includes representatives from all member and partner countries (with the exception of Brazil).

The Balance Sheet of Arguments

The initial administration of PISA in 2000 included 43 nations. Over the last 15+ years, participation in PISA has greatly expanded with the most recent administration in 2015 including 72 nations/economies. Given that PISA appears to be continually expanding in terms of the number of participating nations/economies and the measures administered, it is not surprising that it has been referred to as “one of the largest non-experimental research exercises the world has ever seen” (Murphy, 2014, p. 898). The popular media, all over the world, often greet the release of PISA results with a flurry of published news stories, as evidenced by the December 6, 2016 release of PISA 2015 (see Bleiker, 2016; Davie, 2016; Guerney-Reid, 2016; Mortillaro, 2016; Saldana & Cancian, 2016; Ripley, 2016). Not surprisingly, a great deal has been written about the significant, and many would argue, excessive attention that PISA has received from national media outlets around the world, begging the question—is PISA worthy of all this attention? The answer to this question is largely, although not exclusively, divided into two broad perspectives that either supports the expanding influence of PISA versus those that contest its utility in promoting education system improvement. It is worth noting at the onset that the “balance sheet of arguments” is decidedly lopsided given that the academic community, particularly education scholars, which are responsible for the majority of the research and publications on this topic, have been largely critical of the OECD and its role in promoting large-scale educational reform.

The Support Camp

Not surprisingly, the most vocal and ardent support for PISA comes from the OECD and government policymakers that utilize this benchmark measure. The OECD has argued that PISA has created valuable opportunities for transnational policy collaboration and should be credited with promoting high and more equitable learning outcomes across various student populations (Schleicher, 2009). The OECD is quick to point out that many countries have benefited from the use of PISA data, such as Germany, which has made “observable steady progress every three years” (OECD, 2014b). Essentially, PISA has facilitated evidenced-based decision-making so that national governments can improve their human capital, and consequently, their future economic prosperity (Volante & Ritzen, 2016).

In order to provide policy support and guidance the OECD has developed an impressive range of open access resource documents and reports. Its online library contains a number of series such as PISA in Focus, Teaching in Focus, and Education Indicators in Focus which provide comparative data and conclusions that address a variety of questions. Sample titles from PISA in Focus include Is spending more hours in class better for learning? (No. 54), How has student performance evolved over time? (No. 47), and Could learning strategies reduce the performance gap between advantaged and disadvantaged students? (No. 30). These briefs complement other web resources such as their free OECD Data Factbook app and Education GPS website that provides comparable data and analysis on education policies and practices, opportunities, and outcomes. An interesting feature of Education GPS is that it provides the latest information, in real time, on how countries are working to develop high quality and equitable education systems. Collectively, the OECD’s impressive infrastructure has helped make the PISA results more significant for policymakers than data from other international achievement measures such as the Trends in International Mathematics Study (TIMSS) or the Progress in International Reading and Literacy Study (PIRLS) (Volante, 2016b). To date, comparative research has suggested that the OECD has indeed influenced the uptake of recommended education policies stemming from PISA (see Baird, Isaacs, Johnson, Stobart, Yu, Sprague, & Daugherty, 2011; Breakspear, 2012; Grek, 2009; Volante, 2016b).

Despite the previously noted findings, the OECD has argued that many of their policy suggestions are actually not implemented when one carefully examines the global educational landscape. For example, their Education Policy Outlook 2015: Making Reforms Happen report suggested that the reforms “needed” for coming close to the best performing countries and the actual reforms implemented was (very) weak, when they examined some 450 education reforms that were adopted across OECD countries between 2008 and 2014 (OECD, 2015). Thus, from the OECD’s perspective, the influence they have been able to exert on educational policy uptake and reforms should not be exaggerated. When one considers the widespread academic criticisms of the OECD’s international surveys noted in the next section, it is fairly clear there remains diametrically opposed views on the nature and scope of the OECD’s role in the educational policy production process. These widespread criticisms are addressed below and explored further by a number of scholars in this edited volume—particularly in relation to conceptual and methodological issues.

The Contestation Camp

As previously noted, a significant segment of the academic community has contested the role of the OECD and PISA in promoting large-scale educational reforms. Indeed, the list of detractors and the nature of their concerns is far too lengthy to address in detail within this chapter. Fortunately, this litany of concerns is aptly expressed in an open letter in The Guardian (British national daily) newspaper entitled “OECD and PISA tests are damaging education worldwide-academics” which was directed to Dr. Andreas Schleicher, the current director of the OECD program (see Andrews et al., 2014). This open letter was signed by a group of more than 80 high-profile academics from around the world arguing that PISA:

• shifts attention to short-term fixes designed to help a country quickly climb the rankings, despite research demonstrating that enduring changes in education practice take decades to come to fruition;
• takes attention away from the less measurable or immeasurable educational objectives like physical, moral, civic, and artistic development, thereby dangerously narrowing our collective view regarding the purpose of education;
• is naturally biased in favor of the economic role of public schools versus how to prepare students for participation in democratic self-government, moral action, and a life of personal development, growth, and well-being;
• harms children and adversely impacts classrooms with its continuous cycle of global testing, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted “vendor”-made lessons, and less professional autonomy for teachers. In this way, PISA has further increased the stress level in schools, which endangers the well-being of students and teachers (Andrews et al., 2014).

These points were reiterated in another open letter that was published in a special edition of the academic journal Policy Futures in Education (Vol. 12, No. 7) which featured more than 130 signatories (as of May 6, 2014).

Detractors of PISA also focus on the influence of the OECD as a pivot for global educational governance. By global governance, scholars are referring to the process of governing that is not bounded by the nation-state, but involves multiple actors and scales (i.e., global, national, and regional) in the policy production and implementation process (Rizvi & Lingard, 2010). International organizations such as the OECD, through PISA, are able to use a form of “soft-power” that compels nation-states to adopt particular policies (Bieber & Martens, 2011; Meyer & Benavot, 2013; Mahon & McBride, 2008; Morgan & Volante, 2016). Scholars have suggested that these policies are directly associated with the rise of neoliberal forms of governance and the new managerialism that involves steering at a distance (see Lingard et al., 2016). However, it is worth noting that the degree of this shift is openly debated as the OECD’s focus on public ...

Table of contents

• Cover
• Title
• Copyright
• Dedication
• Contents
• Foreword
• Preface
• Acknowledgements
• PART I Introduction
• PART II Conceptual and Methodological Issues
• PART III Topical Issues Within National and Transnational Contexts
• PART IV Conclusion
• About the Editor
• Contributors
• Index