The SAGE Handbook for Research in Education

Pursuing Ideas as the Keystone of Exemplary Inquiry

Clifton F Conrad, Ronald C. Serlin

• Publisher: SAGE Publications, Inc
• Year: 2011
• ISBN:
  9781483305622
• Edition: 2
• Topic: Education
• Subtopic: Research in Education

The SAGE Handbook for Research in Education: Pursuing Ideas as the Keystone of Exemplary Inquiry, edited by Clifton F. Conrad and Ronald C. Serlin stimulates and encourages students, faculty, and educational practitioners, including individuals in Pre K–16 education, government, and the private sector who conduct applied and policy-oriented educational research, to place the pursuit of ideas at the epicenter of their research—from framing meaningful problems to identifying and addressing key challenges to the reporting and dissemination of their findings. As well as supporting readers to place the pursuit of ideas as the keystone of exemplary inquiry, the Handbook draws on the perspectives of scholars representing diverse fields within the field of education—from pre-kindergarten to elementary and secondary school to higher education—as well as qualitative, quantitative, and mixed-methods approaches to inquiry. The chapters are punctuated throughout by the voices of authors who wrestle with the formidable challenges of framing and conducting and reporting meaningful inquiry.

PART ONE

---

EXPLORING IDEAS

Embracing Inquiry and the Craft of Framing Meaningful Problems

SECTION I

EXPLORING THE MULTIPLE PURPOSES OF INQUIRY AND KEY STAKEHOLDERS

Introductory Essay

D. C. PHILLIPS

Stanford University

Webster defines “inquiry” succinctly as “attempting to learn,” an account that unfortunately is silent on all of the thorny issues. Who, precisely, is making the attempt? What prompted or motivated the attempt to learn? Did the inquirer come up with the subject of the inquiry on his or her own volition, or was the inquirer commissioned to try to learn something on behalf of someone else? Are the products of the inquiry intended to be helpful to other inquirers, or to people in other walks of life who make use of such products—practitioners or policy makers, perhaps? How will the success or failure of the inquiry be determined (and by whom)? What means—what methodology—shall the inquiry make use of? Are some methods better than others in the sense that they are likely to lead to the learning of something that is both useful and correct? Is every inquiry unique, or are there “family resemblances” that cut across different inquiries? Will the attempt be solitary, or will it involve a (large or small) team or community? What role shall the results of prior attempts play in directing the current inquiry? Such are the questions that plague a philosopher who dares to consult a dictionary.

But these issues are of more than “mere” philosophical interest; many of them have been bedeviling members of the educational research community, where for many decades the nature of educational inquiry has been the focus of intense debate. Educational research is of low quality because it is too focused on practice at the expense of theory, educational research is of low quality because it is too “ivory tower”—too theoretical—at the expense of focusing on practice or being informed by practice, educational inquiry is of low quality because it lacks scientific rigor, and educational inquiry is irrelevant either because it suffers from “physics envy” and mimics the (inappropriate) methods of the natural sciences or because it pursues the wrong questions—esoteric questions that did not arise from a close understanding of the problems faced by those in the world of practice. And so the debate has gone for decades, back and forth, mostly marching in place. (These various charges against research are documented in Phillips, 2009.)

A few years ago, governments around the world (notably in the United States and the United Kingdom, but also elsewhere) threw their weight in, insisting on “evidence-based policy and practice”—not a bad idea, of course, for who wants “lack of evidence-based policy and practice”? But the small print contained a surprise: “Evidence” was characterized as being that which has been obtained (preferably) by the use of randomized controlled experiments or field trials (RFTs). Qualitative researchers entered the debate with vigor, supported by advocates of mixed methods designs (and others—see Phillips, 2006). But the hard-liners were not silent and struck back—with one of the responses being to hint, darkly, that use of anything except true experimental methods will identify an inquiry as being in the disreputable category of “unscientific.” (The other response was to withhold research funds from the apostates.) Although subsequent political events, especially in the USA, have had a somewhat calming effect, the issues have not been completely settled—and the intellectual, social, and educational stakes are still high.

The three chapters in Section One of this volume will, it is hoped, throw light on a number of these issues (but not, of course, on all of them), and they will offer some solace to those educational researchers who wish to see an educational research enterprise that is both rigorous and relevant to the various stakeholders who wish to draw on it for guidance (or at least for succor) that is not hamstrung by a preconceived and extremely narrow view of the nature of science, and that is informed by—and draws inspiration from—being in close contact with various realms of educational practice.

In the first chapter, D. C. Phillips (the author of this introduction) takes an unusual tack. Rather than arguing the well-known thesis that the natural sciences are a stultifying model for educational research and the related social sciences to imitate, he argues instead that the natural sciences—when properly understood—can serve as a liberating model. His case hinges, of course, on how the proviso “properly understood” is construed. The main theme of the chapter (backed up with numerous examples) is that the natural sciences are incredibly diverse in the types of inquiries that are conducted, the purpose of these inquiries, and the methods that are used. Natural scientists can be viewed as using their creativity and “doing their damndest” to construct cases that are rhetorically sound, in the sense that these are developed logically, and that marshal relevant evidence of all kinds to warrant the conclusions that are being put forward. To construe science, or even educational science, as being characterized by the use of the RFT is to ignore the history and impressive achievements of the natural sciences; and the attempt to characterize the nature of science, and the nature of rigor, in terms of the use of a narrow range of “methods” can draw no support from the history of the sciences. (It is worth noting, in passing, that on the narrow account of science recently favored in official circles and still touted by members of the educational research community, Roentgen, Galileo, Newton, Darwin, and many others must be judged to be “unscientific.”) The chapter also stresses the vital contribution that is—and inevitably must be—made by branches of educational inquiry that are nonscientific even under the most charitable interpretation (which is not to say that they are lacking in rigor).

Next, Robert Floden reminds the educational research community of the diverse audiences that our work serves—and how it serves them. Perhaps chief among the audiences for our work are our fellow researchers; it is from them that we usually draw our methods and theories—and often our problems. This is as it must be. But there are many other “stakeholders” in the educational enterprise whose problems sometimes concern us and who sometimes are concerned about our work. These stakeholders include parents; teachers; local education officials and program managers; state-level policy makers, politicians, and program managers; and federal policy makers, bureaucrats, and politicians. Floden reminds us of the contexts in which these individuals are operating, and he points out that the rhetorical form in which we report our work is not likely to resonate with their concerns or with their distaste for ingesting “social science findings.” (Reports or stories from practitioners working in parallel contexts often have more face validity—and certainly more appeal—than do our carefully crafted, data-laden papers.) If we wish our research to have impact, we must attend more carefully to the needs, interests, and problems of those with whom we wish to communicate, and we must heed their favored styles of communication.

The final chapter in this section of the volume, by David Plank, in essence starts where Floden leaves off. One of the audiences for the work of the educational researcher that Floden discusses briefly is the policy community, and this becomes the specific focus of Chapter 3. In particular, the discussion grapples with the dilemmas that need to be faced when doing policy research. Often researchers are dismayed that their work seems to be ignored by policy folk, while the latter often regard educational research as being quite irrelevant to the highly charged situations in which they are working. A large part of the cause of the mutual disregard that can exist is that researchers and policy makers, being situated in different contexts, judge different questions to be important and have a different sense of what questions are answerable or intractable; for example, ideology is officially eschewed in research but may influence which results are accepted by policy makers. This chapter, like the previous one, also remarks on the differences in rhetorical style across the research and policy contexts. Plank devotes the second half of the discussion to two illustrative case studies: the debates over teacher quality and school choice.

Overall, then, the chapters in Section One have a liberatory intent; it is hoped that they will serve to broaden the options open to researchers—the methodological options, the types of inquiry that are mounted, the audiences for whom (and sometimes with whom) the inquiry is pursued, and the styles in which the research is reported. Broadening the options for rigorous inquiry is always a good thing.

REFERENCES

---

Phillips, D. C. (2006). A guide for the perplexed: Scientific educational research, methodolatry, and the gold versus the platinum standards. Educational Research Review, 1, 15–26.

Phillips, D. C. (2009). A quixotic quest? Philosophical issues in assessing the quality and rigor of educational research. In P. Walters, A. Lareau, & S. Ranis (Eds.), Educational research on trial: Policy reform and the call for scientific rigor (pp. 163–195). New York: Routledge.

1

---

MUDDYING THE WATERS EVEN MORE

The Many Faces of Empirical Educational Inquiry

D. C. PHILLIPS

Stanford University

The progress of Science is generally regarded as a kind of clean, rational advance along a straight ascending line... [but] the manner in which some of the most important individual discoveries were arrived at reminds one more of a sleepwalker’s performance than an electronic brain’s.

—Arthur Koestler (1959, p. 11)

There are more things in heaven and earth... than are dreamt of in your philosophy.

—Shakespeare, Hamlet

Almost from time immemorial (at least for half a century, to be conservative), a great deal of ink has been spilled in debates over the nature of, and the desirability of pursuing, an important genre of educational research—namely empirical research that has scientific pretensions. Even the earlier version of this essay, in the first edition of the Handbook—an essay that bears an embarrassingly similar title (“Muddying the Waters”)—spilled (but not necessarily wasted) a great deal of ink on this broad topic.

Put crassly, in many influential quarters around the world, empirical educational research has been treated as, in essence, a straightforward enterprise both with respect to its aim and its methods—despite the fact that many of the theories and much of the background knowledge upon which it depends are complex, occasionally abstract, and (usually) require much concentrated effort to master. Empirical educational inquiry is straightforward in principle, the story implies, because its purpose is clear-cut and narrow—namely, to establish causal relationships that can be used by planners, policy makers, and practitioners. And luckily there is a “gold standard” methodology that, if properly deployed, points the way to rigorous work and eventually leads to reliable, and hence usable, findings. This methodology, of course, involves carrying out randomized controlled experiments or field trials (RFTs), and it also requires narrowing one’s focus to the issues: “Does treatment T cause an effect E?” and “What is the size of the effect?” Doing educational research, on this account, is rather like gourmet cooking—if you follow the recipe, in all likelihood you will be successful, although following the recipe requires that you have had some prior experience and that you have a number of relevant skills at your disposal. Having the right recipe does not guarantee success, but it facilitates success and, in the case of some neophyte cooks, it might even be a necessary condition for success. The favored educational version of Julia Child’s cookbook is, of course, Campbell and Stanley’s Experimental and Quasi-Experimental Designs for Research (1963; first published as a long essay and later released in the form of a monograph).

The aim of the following discussion (it was also the aim in the earlier manifestation of this essay) is to show that this is an untenable view; the empirical research that we remember and that achieves something worthwhile does not generally follow a simple, straight-line trajectory and is much more like the meanderings of Koestler’s sleepwalkers—and furthermore, sometimes it is long after the night is gone that the significance of these meanderings becomes apparent. There is much more to rigorous, “scientific” research than is dreamt of in the philosophies of those who single-mindedly advocate (or even mandate) the use of the RFT, or who narrow their focus to the question of whether some treatment or factor is causally efficacious. (For a guide through the quicksand concerning the RFT as gold standard, see Phillips, 2006.)

However, it needs to be emphasized that it is not part of the argument of this chapter that there is anything wrong or blameworthy about the rigorous use of true experimental and related designs; it is beyond dispute that they deserve to be part of the educational researcher’s armamentarium. But it also needs to be stated clearly that they are only part, and do not constitute anything like the whole. And the reason that they cannot be the whole, cannot be the entire gold standard, is that they are useful designs only for certain purposes, and though important, these purposes are quite limited in the scholarly (and even in the scientific) scheme of things. There is more to educational research than establishing that a treatment causes an effect (or that the effect is of a particular size); therefore, educational research must be acknowledged as being more complicated and as having more purposes and functions. Crucially, the purpose of the following discussion is to show that taking a proper view of science supports this conclusion; educational research often aspires to be scientific, and it is a serious mistake to think that science serves only a narrow set of purposes.

WOULD REAL SCIENCE PLEASE STEP FORWARD? TWO BRIEF EXAMPLES

---

There is a strong tendency, when research is being discussed—especially by students who are being trained for research careers—to consider as examples pieces of work that not only have been completed but that also have had a productive outcome (this is a kind of “whig” history of science). But this, of course, gives a false view of what the doing of research is really like. It would be an affront to common sense to suggest that a couple of examples can redress the balance and can stand as being “typical” or “representative” of an endeavor as vast and as rich and open-ended as scientific inquiry. So the following are presented, rather, as “existence proofs”—they show that there are indeed cases where scientific work cannot adequately be conceptualized in terms of the model provided by the RFT. One of these examples is taken from the history of physics and the other from the history of educational psychology. These examples are, I believe, quite liberating if taken as models for educational researchers to ponder.

(1) The Discovery of X-Rays. In early November 1895, Wilhelm Roentgen was working with the recently developed glass vacuum tubes, through which an electric current could be passed; his interest was the external effects of the electrical discharge. He had placed in a tube an aluminum shield with a small aperture in it so that a beam of the “cathode rays” passing down the tube could exit at the end of the tube, and he shielded the remainder of the tube with a close-fitting jacket of black cardboard. Standing nearby at the side, waiting to be used in a later experiment, there happened to be a paper screen coated on one side with a barium salt, and Roentgen noticed that it started glowing. What on earth was happening? Roentgen’s own words are interesting:

That paper... lights up with brilliant fluorescence when brought into the neighborhood of the tube, whether the painted side or the other be turned towards the tube. The fluorescence is still visible at two meters distance. It is easy to show that the origin of the fluorescence lies within the vacuum tube. It is seen, therefore, that some agent is capable of penetrating black cardboard which is quite opaque to ultra-violet light, sunlight, or arc-light. It is therefore of interest to investigate how far other bodies can be penetrated by the same agent. (Roentgen, 1896)

He found that the fluorescence would still occur if the screen was placed behind a book of a thousand pages, a sheet of tinfoil, a pine board three centimeters thick, ebonite, and a human hand. He found that covered photographic plates would register an image and that prisms that “deflected” light had no effect on this mysterious “agent,” which also could not be concentrated by the use of lenses. He found that this agent—which, because nothing was known about it, he called “X radiation”—did not “deviate” in a strong magnetic field, indicating that it was not the newly discovered “cathode rays” (now known to be beams of electrons) that were discharging down the evacuated tube (which were deflectable by magnets); and he showed that these X-rays seemed to be produced by the cathode rays striking the end of the glass tube. Are these rays a form of ultra-violet radiation? Roentgen listed four ways in whic...

Table of contents

• Cover Page
• Title
• Copyright
• Contents
• Preface
• Acknowledgments
• Introduction
• PART ONE. EXPLORING IDEAS: EMBRACING INQUIRY AND THE CRAFT OF FRAMING MEANINGFUL PROBLEMS
• PART TWO. PURSUING AND SHARING IDEAS: THE DESIGN AND CONDUCT OF INQUIRY AND THE COMMUNICATION OF RESULTS
• PART THREE. ENGAGING IDEAS: REFLECTIONS OF SCHOLARS
• Index
• About the Editors
• About the Section Editors
• About the Contributors