Chapter 1
When will we Ever Learn? Or the Elephant in the Classroom
Kevin Wheldall
Preamble
Nul nâaura de lâesprit, hors nous et nos amis
[No one shall have wit save we and our friends]
(Molière, Les Femmes Savantes, III, 2 1672)
The very powerful and the very stupid have one thing in common. Instead of altering their views to fit the facts, they alter the facts to fit their views which can be very uncomfortable if you happen to be one of the facts that needs altering.
(Doctor Who, âThe Face of Evilâ 1977)
In his play Les Femmes Savantes (âThe Wise Womenâ), the seventeenth-century French dramatist, Molière, satirised the contemporary cult of la prĂŠciositĂŠ (or âpreciousnessâ). It had become the fashion in learned society to speak in an extravagant and prolix manner as a means of demonstrating erudition. Instead of referring to oneâs teeth, for example, the pretentious, fashionable elite would instead talk of lâameublement de la bouche (literally, âthe furniture of the mouthâ).
I am reminded of this famous play, with embarrassing frequency, when I read much contemporary educational âresearchâ, especially that emanating from a supposedly postmodernist perspective. Rhetoric appears to have replaced reason and assertion appears to have replaced the need for empirical fact. It is apparent to almost everyone, except to many educational researchers themselves, that much contemporary educational research has little relevance to, or has little potential to inform, educational practice. I believe passionately that we shall only make progress in education when we base educational decision making on the findings from empirical, and where possible experimental, educational psychological research. I welcome this opportunity to provide a personal perspective on developments in educational psychology over recent years and to comment on the contemporary state of affairs.
How has the Discipline of Educational Psychology Developed?
More than 25 years ago, in 1981, Richard Riding and I, both then working in the Department of Educational Psychology at the University of Birmingham, established and became joint editors of a new journal: Educational Psychology: An International Journal of Experimental Educational Psychology. In this collection of chapters, contributed by members of the Editorial Board of Educational Psychology and others, a series of perspectives is provided on progress in key areas within the discipline over the past 20â30 years. In the first issue of Educational Psychology in 1981, Richard Riding and I (as the founding joint editors) nailed our colours to the mast in our opening editorial article entitled âEffective Educational Researchâ (Riding and Wheldall 1981). In that article, we spelled out our hopes and aspirations for educational psychology, the discipline, as well as for the journal. The aim of this introductory chapter is to reconsider some of these issues, to question the progress made in educational psychology, and to explore the interface between research and practice in education.
The journal was dedicated to the ârapid dissemination of experimental psychological research which has a direct bearing on educational topics and problemsâ (Riding and Wheldall 1981: 5). Riding and I (1981: 8) wrote:
Teachers frequently complain that educational psychology has little to offer them that is of real value in the classroom. We think that it is important to admit that, to some extent, they are right. Experimental studies, in the true sense of the term, are in the minority in educational psychology.
We argued that effective educational research necessarily entails the predominant use of traditional experimental psychological methods and designs. The main founding aim of the journal was to provide a forum for and to encourage experimental research in educational psychology. It seemed to us, working in two separate fields and from two very different theoretical perspectives (behavioural and cognitive psychology), that there was a clear need for a vehicle to promote a rigorous, experimentally based educational psychology which would directly inform educational practice. This view was based on our belief that the balance between experimental and non-experimentally based research had been seriously tilted in favour of the latter, and that far fewer truly experimental studies were being carried out, or at least reported. Has this situation changed?
Before answering this question directly, it is only fair to comment on the achievements of Educational Psychology in helping to redress the balance. A perusal of the contents over the years provides clear testimony to the variety and quality of the experimental research in educational psychology being carried out internationally. This is true for both cognitive and behavioural perspectives and also for the developmental perspective. Many important experimentally based studies have been reported, which have clear implications for practice, for immediate application in the real world of education. From this, we could begin to conclude that clear progress was being made in the right direction. Without wishing to play down these achievements, however, it must also be said that one journal does not make a movement. To take a camping analogy, it is not enough to make sure that life is warm and cosy inside the tent. It is even more important to look at what is happening outside.
It would be foolish to pretend that there has been much of a change of research climate in this direction within educational research generally. In fact, there has been an increasing preference for manifestly non-experimental research and widespread disenchantment with empirical educational research methodology altogether.
First the good news. There is evidence that single-subject and small-N methodology, pioneered by behaviour analysts, has become more generally accepted as a means by which the effectiveness of interventions in applied settings can be experimentally assessed. In some field settings it is certainly difficult, if not impossible, to run fully randomised control group designs with large sample sizes. On the other hand, small-N designs using single subjects as their own controls by means of reversal and/or multiple baseline designs allow rigorous evaluation of treatment effectiveness for a wide range of educational innovations. The manifest major disadvantage of such methods, weak generalisability, is at least partially overcome by the accumulation of clear replications of the effect over a number of subjects and over replicated studies by different research groupings. It is important, however, not to overemphasise the applicability of small-N methodology; it is neither feasible nor appropriate for all experimental studies.
The bad news is the âcrisis of confidenceâ by not a few academics and researchers in traditional, statistically based methodology. In a desperate search for immediate and manifest relevance, some have been seduced by the apparent charms of action research, ethnographic methodology, postmodernist theorising, and so-called qualitative approaches. These are, of course, by no means the same thing, but have become increasingly frequent bedfellows. One student, quizzed as to the nature of his proposed research, responded by saying: âItâs action research really. Itâs phenomenological.â When the bemused academic asked him what he meant by this, the student replied: âWell, Iâm only doing it in one school.â Amusing howlers aside, however, this points up a very real danger.
The impression has gained ground that qualitative research is necessarily more relevant, avoids the problems of carrying out research based on traditional methodologies, and is altogether easier to do. As a consequence, we are left with the curious notion that a more subjective approach is preferable to a more objective approach with known and acknowledged limitations. Without wishing to minimise the contribution that such research might conceivably make to sociological analyses of educational processes, or to deny the valuable service action research can perform in demonstrating the practical utility and applicability of new initiatives in schools and classrooms, neither of these methodologies can possibly do more than serve as informal pilot or pre-pilot studies to properly conceived psychological evaluations of essentially psychological phenomena relevant to educational practice. Teaching and learning are essentially psychological phenomena.
Again, perhaps one should stress that what has gone before is not meant to ignore the possible value of qualitative or action research in suggesting relevant testable hypotheses for future empirical research. It is whether such methods alone can unequivocally demonstrate scientific truths that is in doubt. How and why educational innovations work is and must be the legitimate concern of research in educational psychology â not because it affords the luxury of sterile model building, but because an empirically determined psychology of pedagogy and related educational processes is the only way we can achieve long-term progress. As Wheldall and Carter (1996: 133) put it:
We would support the great value of qualitative methodology, particularly in providing rich insight into the processes in naturalistic situations. However, qualitative methodology is equally unsuitable to other research tasks such as providing unambiguous evidence of causal relationships. Notwithstanding the problems of artificiality and socially valid outcome measures that can arise with quantitative methodology, many questions in education are fundamentally quantitative in nature. Rational and incremental progress should be informed by the research methods which are appropriate to the questions being forwarded. Qualitative methods are both different and complementary.
Finally, it is instructive to reflect on whether true experimental designs are any more prevalent today in educational research than they were when Educational Psychology (the journal) was founded. Seethaler and Fuchs (2005) suggest that this is unlikely on the basis of their analysis of the content of five peer-reviewed special education journals over the past five years. Of the articles published, only 5.46 per cent reported a reading or maths intervention using a group design and only 4.22 per cent employed random allocation to groups. It is, then, perhaps rather harsh to criticise teachers for not recognising the value of evidence-based practice by employing teaching methods and programmes validated by âgold-standardâ randomised control group designs when so very few articles published by educational researchers actually report findings from such studies! Moreover, special education has a far stronger tradition than regular education for hard-nosed, data-driven educational psychological approaches to both instruction and research. If this is the case for special education, what must the situation be like for regular educators keen to be guided by gold-standard evidence-based practice?
The Rise of Constructivism
Alongside and sometimes in consort with an increasing preference for qualitative educational research methodology, within educational psychology per se, as well as in educational research more generally, we have also witnessed the rise of what has come to be known as âconstructivismâ as a dominant force within the discipline (see Chapter 2 and Chapter 9). But while constructivism, in its many forms and guises, appears to be the current âflavour of the monthâ in educational psychology, perhaps the enthusiasm with which it is being advocated (if not always so readily adopted) should be tempered with a more rigorous concern with its evidential basis.
Wheldall and Carter (1996) commented that some aspects of constructivism appear to be little more than âold wine in new bottlesâ. Moreover, much of the apparent research literature promoting constructivist pedagogy appears to be more descriptive or exhortative than evidence-based. Apps and Carter (2006), for example, refer to a pilot study they conducted in which they searched the ERIC database from 1982 to 1999 for the terms âconstructivismâ and âdiscovery learningâ, and also for the termâdirect instructionâas a comparison reference point. According to Apps and Carter, the search revealed that, while discovery learning produced 1,871 hits and constructivism 1,170 hits, direct instruction produced fewer than half as many, 409 hits. More important, however, was their subsequent more detailed analysis of the abstracts of the first 50 and the last 50 articles within each category. As Apps and Carter (2006: 8) comment, their results:
illustrated the increase in constructivist literature and revealed a tendency for this literature to be primarily of a non-empirical nature. For example, 51% of articles addressing direct instruction were empirical and examined student learning outcomes, compared with 2% of articles addressing discovery learning and 4% addressing constructivism.
(In a subsequent study specifically addressing constructivist approaches to special education, they examined all 114 peer-reviewed articles up to October 2004 on this topic revealed by searches of both ERIC and PsychINFO and found that only 6 [5.3 per cent] were experimental in nature.) These findings suggest that there is considerably more empirical work to be done before the evidence can match the rhetoric advocating constructivist approaches to teaching.
This lack of reliance on empirical evidence is not confined to constructivists, however. The idea of advocating the use of programmes of proven effectiveness sounds like âteaching oneâs grandmother to suck eggsâ:who would use an ineffective programme? But teachers and educators often do. This leads to an important point regarding the way in which educational practice is advanced.
Programmes of Proven Effectiveness
It is quite commonplace in education to see methods, programmes, and techniques employed for which the data on effectiveness is minimal, non-existent, or even contrary. The late American educator Jeanne Chall (2000: 3) addressed this very issue:
What is particularly striking about educational innovations is that most were considered successes long before they were actually sufficiently tried and tested. Seldom were they presented together with a rationale based on educational theory and research. Nor had they been tried first in small pilot studies before being offered as solutions to national education problems.
She subsequently laments: â[P]ractice often went in a direction opposite from the existing research evidenceâ (2000: 180). My own experience as an educational psychological researcher bears testimony to Challâs observations. Specifically, I am referring to educational innovations, for which there was limited or no evidence of efficacy, being adopted. In New South Wales, Australia (where I have lived and worked since 1990), the state educational bureaucracy embraced and implemented at least three disparate programmes or innovations, without sufficient or sufficiently strong evidence of their effectiveness in improving student outcomes in schools. Reading Recovery, âaccelerative learningâ, and a behaviour management approach called W.I.N.S. (Working Ideas for Needs Satisfaction) were all enthusiastically adopted by the state education department and all three exemplified at least one of Challâs criticisms. I shall resist the temptation to dwell on accelerative learning, the treatment for which there was no known disease, since I was not involved with it in any way, but I will mention briefly the other two.
I was commissioned to evaluate the behaviour management package, W.I.N.S., following a wide-scale âtrain the trainerâ implementation programme across the state. We found that, in spite of the positive evaluations it received in our surveys from trainers and others, there was no empirical evidence to testify to its efficacy in terms of bringing about substantial change in either teacher or student behaviour or, indeed, on any other measure we employed. Fortunately, as it turned out, very few schools were actually using it (Wheldall and Beaman 1994).
The case of Reading Recovery is even more tragic since it is still widely employed in New South Wales schools (and in some other Australian states and many other countries). Again, we were commissioned to evaluate the programme but, even before we reported, a decision ...