Evidence-based Practice in Education

Functions of evidence and causal presuppositions

Tone Kvernbekk

• Publisher: Routledge
• Year: 2015
• ISBN:
  9781135039189
• Edition: 1
• Topic: Education
• Subtopic: Education General

Much educational debate today is dominated by a "what works" vocabulary, intimately associated with evidence-based practice (EBP). The vocabulary consists of concepts and ideas such as accountability, competency, effectiveness, employability, learning outcomes, predictability, qualifications, and testing. As schooling and education are considered successful when predetermined outcomes have been achieved, education is often believed to require assessment, measurement and documentation.

In this book, Tone Kvernbekk leaves the political, ethical and professional dimensions on the sidelines and focuses instead on further unpacking the core of EBP. Chapters concentrate on several fundamental issues ignored by current literature, including:

• 
  
  

• the character of the evidence that plays a central role in EBP in both practical reasoning and acting under uncertainty

• 
  
  

• the notion of causality presupposed by discussion of the production of desired effects and played out in the basic structure of interventions

• 
  
  

• a system-theoretical look at why interventions might not work.

By considering these key points, Kvernbekk articulates both the legitimate uses and the illegitimate, philosophically problematic misuses of EBP in educational thinking and practice. The book will be of key value for academics and postgraduate students in the fields of educational research and practice, philosophy of education and educational theory, especially those concerned with research methodology, professionalism, and discussions regarding evidence-based practice.

Chapter 1Introduction

DOI: 10.4324/9780203774830-1

This is a book about evidence-based practice in education. There is no shortage of literature about evidence-based practice (hereafter EBP). A huge range of aspects of EBP and its (alleged) implications have been explored, analyzed, endorsed by some, and criticized by others. For example, the nature of EBP; the nature and function of educational research; implications for the teaching profession; what counts or should count as evidence; the EBP understanding of practice; methodological developments such as meta-analyses and systematic reviews; the role of evidence warehouses such as the Campbell Collaboration; the translation of research into practice; and privileging instrumental efficacy at the expense of considerations of ethics and appropriateness. This book is not a systematic review of the literature, but is rather an attempt at further unpacking the core of EBP. This core or nature is inescapably causal. And to the best of my knowledge, causation and its possible implications for EBP are not well covered in the otherwise burgeoning literature.

I would like to begin by telling two short stories. Both stories are intended to convey ideas I take to be important. The first story is a personal anecdote about how my class learned to read. We were some twenty first-graders in my class. The class was kept together at all times, so we all got the same input in school. The teacher thus taught all of us to read in the same way, according to the same method, with no concessions made to ability, learning style or motivation. As I recall it, by Christmas, if not sooner, we had all learned to read. Some faster than others, evidently, some more proficiently and with better understanding than others. But, as I recall it, we all had learned to read. If anybody had asked my teacher why she used the method she did, she might have said that she had taught reading in this way for thirty years, and it worked every time.

I acknowledge that this story can be problematized in different ways. What I want to extract from it is the idea of plasticity. First, the plasticity of the human mind. We have a great ability to learn, even under circumstances that are far from optimal. Thus, a student whose intellectual make-up would imply that learning to read by recognizing whole words would be better or more effective, still learned to read by the method used by my teacher—namely to teach us what the letters look like, what they sound like, and then how to put them together into words. Possibly this kind of plasticity can account for a good many educational achievements. And so, possibly, can the plasticity involved in the teacher's admittedly hypothetical claim (although I sincerely believe that that is what she would have said) that her reading instruction worked every time. Surely there was a certain vagueness built into the expression “they can read;” such that reading with differing degrees of proficiency still fell under the concept of being able to read. When “being able to read” gets more narrowly specified, as it probably has since my school days, the verdict might now be that the method does not work that well after all. Judgments about what works are not straightforward and some leeway or plasticity might be a good thing.

The second story is about Mr. G. He was a geometry teacher in New York in the 1930s and appears in Israel Scheffler's educational memoirs Teachers of My Youth (1995). Scheffler describes Mr. G. as tired, uninspired and indifferent to the students. He had a stack of index cards with geometry problems and he presented them to the class, one at a time. The students were called upon to put their solutions on the board and explain them. Whatever problem remained unsolved was carried over for homework. On occasion there was a substantial backlog of unsolved questions, yet a new homework problem would be added whatever the backlog was. Every class was like this, Scheffler says; the students attempting to solve the daily problem and reduce the backlog. And the teacher?

And all through each class, Mr. G. sat dreamily, leaning back in his chair in the corner of the room, seemingly oblivious and yawning now and then as he turned another card over or asked another pupil to explain the matter at hand.

(Scheffler 1995: 83)

It would seem that Mr. G. violated all principles of good teaching. Good teachers are not supposed to be tired and certainly not indifferent to their students. But while Mr. G. may not have been a good teacher, he certainly was an effective teacher. The students began working on geometry problems at every opportunity, in school and out, and their achievements were splendid. Thus it would also seem that Mr. G., somehow or other, had hit upon a terrific method of teaching. As Scheffler muses,

He just wanted to get on with it. But he had, moreover, worked out an ideal method of getting on, one which both required the minimum of effort by himself, and also, as it happened, produced the maximum educational effect.

(Ibid.: 82)

No doubt the case of Mr. G. is rather eccentric. He just wanted to “get on with it,” without any apparent curriculum, plan, or evidence to inform his method. A brief imaginary experiment where we implement Mr. G.'s method would presumably suggest that his method is too idiosyncratic or even far-fetched to work in geometry classes elsewhere, even if we built a suitable degree of plasticity into our understanding of “works.” We would not recommend it to other teachers. Yet I shall insist that Mr. G. (and others like him) is of great significance in the present educational climate because he reminds us that there are many different paths to good results.

Perhaps, paradoxically, none of these ideas—plasticity and multiple pathways to desired results—will figure prominently in the chapters to come, but they will make occasional reappearances and a sub-section of Chapter 9 will be devoted to Mr. G. I mention them here because I want them to run silently alongside my discussions, perhaps like an undercurrent, as implicit presuppositions.

Evidence-based practice

As Stephen Norris says, “My experience of late is that the mere utterance of the words ‘evidence-based educational practice’ is enough to start a heated debate” (Norris 2006: 251). That is largely my experience also. There is something about EBP that triggers people's passion; maybe the feeling that there are very important things at stake here both for researchers and practitioners, things which largely stem from the political context. Researchers, philosophers, and practitioners are responding in different ways to political demands for better research bases to underpin, inform, and guide both policy and practice. There are enthusiastic advocates who think it is irresponsible and downright unethical to try untested strategies in classrooms and who produce the kind of evidence it is assumed can serve as a base for practice. There are critics who view EBP as positivist, implying both an illegitimate instrumentalization of practice and severe restrictions on the freedom of professionals to exercise their judgment, and who therefore wish to reject it. I shall do my best to walk a middle ground and be, as it were, a critical friend to EBP. On the one hand it might not deserve all the criticisms which are leveled at it; and on the other hand there are still unvoiced criticisms, of which I aim to raise some in this book.

Defining EBP

EBP is practical in nature and is commonly known as the what works agenda. Like most broad terms, “EBP” covers a multitude of meanings and researchers might draw on different ones on different occasions to suit their discursive purposes. Its historical focus, however, has been the use of the best available evidence to bring about desirable outcomes both for client and society; or conversely, the prevention of undesirable outcomes. I have derived my preferred definition of EBP from this historical focus: interventions to bring about desirable outcomes for one or more clients and prevent undesirable outcomes, guided by evidence of how well they work. This definition might seem overly meager compared to other definitions, so let me briefly justify my choice. Definitions for a given term, John Gerring and Paul Barresi (2003: 207) argue, must in principle be assumed to be infinite, since the various attributes can be combined in an infinite number of ways. To be sure, this is true of EBP, which is a highly complex phenomenon referred to by three terms (evidence, based, and practice) which themselves can be defined and understood in different ways. I intend my definition to be a general and minimal definition of EBP. It is general because I do not wish to restrict it to a particular context and then have to worry (as much) about the presumed utility of the concept and its range of application. It is minimal much for the same reason: definitions that are minimal in their attributes are broad in their phenomenal range, whereas detailed definitions necessarily encompass fewer phenomena. As Gerring and Barresi put it, “Minimal definitions identify the bare essentials of a concept, sufficient to bound it extensionally while maintaining all non-idiosyncratic meanings associated with the term” (2003: 207).

Hence, I want a definition which does not specify what kinds of evidence are allowed, what “based” should mean, what practice is, what kind of causation should be involved, etc. I thus opt for a parsimonious definition, whose corollary is a view that EBP, as defined, is flexible in the sense that it can manifest itself differently in different contexts. Sometimes it might manifest itself as a linear, top-down approach to educational improvement; sometimes as a technocratic model assuming that the only worthwhile research questions concern effectiveness of means; sometimes as entailing a limited and specific conception of professional practice, and sometimes as restricting democratic participation and deliberations about the aims of education. All these examples are taken from Gert Biesta's presentation of EBP (2007: 5–7). He seems to view these attributes (and others as well) as simply belonging to EBP or as being logically entailed by it. There are two reasons why I want to resist such a detailed determination of what EBP is. The first is that if we have a maximal definition with a large number of attributes, the range of phenomena covered by the definition will be very limited and in the worst case we might not be able to find any concrete instantiations of our concept at all. The second is that we should distinguish between views of a phenomenon and the phenomenon itself. If somebody holds that all educational research should be experimental research in the service of improvement of student achievements as measured on standardized tests, then we should criticize this view. But we should not understand our criticism as ipso facto pertaining to EBP as well.

A parsimonious definition should by no means be interpreted as simplistic. Viewed from the perspectives that I shall employ, attributes such as causation, induction, stability, and systems (to mention a few) will come to the fore. What I above called the “core” of EBP is amply captured by my minimal definition and its central terms “production” and “prevention” of outcomes, in the form of change. Upon unpacking, this core will be found to be quite complex. To begin with, it is vital to note that EBP is deeply causal, as betrayed by its most common associated buzzwords: what works and effectiveness; and from my own definition: produce and prevent—both causal verbs. My take on the causal basics of EBP is much like that of Gert Biesta: we intervene in a particular situation to bring about certain effects (Biesta 2007: 7). In a later publication (Biesta 2010) he also acknowledges that “what works” is relevant to the professions, since they try to bring about change that is considered desirable (ibid.: 494). But by and large, my take on the rest of the story is different from his. At this point I would just like to add that the situation we intervene in already produces an output. This fact is largely neglected yet it is of considerable significance. Our interventions should (ideally) be based on evidence that shows what works. To say that something works, is basically to say that it brings us the results we want. As my little anecdote suggests, our everyday judgments of whether an intervention works or not at least partly depends on how broadly or narrowly we conceive of the desired result. If our intervention brings about results that we do not want or no perceptible results at all, we conclude that it does not work.

The causal core of EBP

I defer discussion about causal terms such as effectiveness and intervention to Chapters 2 and 6, respectively. The notion of causation will naturally occupy much space in this book, but a brief comment here is nevertheless warranted—I wish to make clear where I stand. Causation is highly contentious among educationalists and philosophers of education. Many think that is has no place in education at all, or at best a very small one. I think the view voiced by Paul Smeyers is widespread:

What can be found in so-called quantitative empirical research is indeed often of very limited use in an educational context. It belongs to a paradigm of causality, which cannot […] give a place for the reasons human beings invoke for doing what they are doing.

(Smeyers 2006: 103)

Thus, it is argued, causation can at best play a peripheral role in the human sciences, education included, because it is associated with a form of (quantitative) research that abstracts illegitimately from context, and because causation is “identified as law-like generalizations paradigmatically used for instance in physics” (ibid.: 107, n.4). Biesta concurs: “…the most important argument against the idea that education is a causal process lies in the fact that education is not a process of physical interaction but a process of symbolic or symbolically mediated interaction” (2007: 8). At the outset here I thus enlist Smeyers and Biesta to represent the causation critics or skeptics, but they are by no means alone.

I, for my part, wish to defend causation in education as a matter of principle. Practical pedagogical language is replete with causal terms such as improvement, bring about, enable, affect, contribute, impact, work, promote, facilitate, precipitate, motivate, depend, influence, encourage, discourage, produce, hinder, reduce, increase, intervention, prevention, omission, learning effect, and many more. EBP advocates and quantitative researchers tend to take causation for granted and presumably see no need to discuss it; but even critics of EBP avail themselves of a causal vocabulary. I take it that in so far as education deals in change, whether we call it learning, organizational development, knowledge acquisition or Bildung, it needs a conception of causation. This is because causation, in its many shapes and forms, denotes a dynamic relation between different kinds of relata, such that one influences the other and produces some change or other in it, or the influence is mutual. This is n...

Table of contents

• Cover Page
• Half Title Page
• Series Page
• Title Page
• Copyright Page
• Table of Contents
• Acknowledgements
• 1 Introduction
• 2 Evidence
• 3 The indirect function of evidence
• 4 The instrumental character of evidence-based practice
• 5 Interlude: on finding one’s way in the causal landscape
• 6 The logic of intervention
• 7 Reproducibility
• 8 What does it take for an intervention to work?
• 9 The best laid schemes
• 10 Conclusion
• Bibliography
• Index