Problematic Research Practices and Inertia in Scientific Psychology

History, Sources, and Recommended Solutions

James Lamiell, Kathleen Slaney, James T. Lamiell, Kathleen L. Slaney

• Publisher: Routledge
• Year: 2020
• ISBN:
  9781000283686
• Edition: 1
• Topic: Philosophy
• Subtopic: Philosophy History & Theory

This volume explores the abiding intellectual inertia in scientific psychology in relation to the discipline's engagement with problematic beliefs and assumptions underlying mainstream research practices, despite repeated critical analyses which reveal the weaknesses, and in some cases complete inappropriateness, of these methods. Such paradigmatic inertia is especially troublesome for a scholarly discipline claiming status as a science.

The book offers penetrating analyses of many (albeit not all) of the most important areas where mainstream practices require either compelling justifications for their continuation or adjustments – possibly including abandonment – toward more apposite alternatives. Specific areas of concern addressed in this book include the systemic misinterpretation of statistical knowledge; the prevalence of a conception of measurement at odds with yet purporting to mimic the natural sciences; the continuing widespread reliance on null hypothesis testing; and the continuing resistance within psychology to the explicit incorporation of qualitative methods into its methodological toolbox. Broader level chapters examine mainstream psychology's systemic disregard for critical analysis of its tenets, and the epistemic and ethical problems this has created.

This is a vital and engaging resource for researchers across psychology, and those in the wider behavioural and social sciences who have an interest in, or who use, psychological research methods.

1Introduction

James T. Lamiell and Kathleen L. Slaney

As the title of this volume indicates, our core concern here is with an abiding intellectual inertia in the discipline of scientific psychology. This inertia is reflected in the continued prevalence of beliefs and assumptions that have guided research methods and justified psychologists’ practical interventions for decades, despite repeated critical analyses revealing the weaknesses—and, in some instances, complete inappropriateness—of those beliefs and assumptions. Occasionally, but far too infrequently, advocates of the critiqued assumptions have attempted to refute the critics, but, routinely, the insufficiencies of the putative refutations have in turn been pointed out by the critics, leaving the critiques fully in force. Nevertheless, the problematic beliefs and assumptions have continued to dominate within the mainstream of the field, much as if no critiques had never even been mounted. In short, apart from the infrequent and consistently unsuccessful attempts at refutation just mentioned, the response within the mainstream to the critiques has been to simply ignore them. Paradigmatic inertia of the sort reflected by these developments would be intellectually untoward for any scholarly discipline, but it is especially problematic for one claiming status as a science.

A discussion sharing concerns along this line took place in a session of the 2018 midwinter conference of the Society for Theoretical and Philosophical Psychology (Division 24 of the American Psychological Association), held in Phoenix, Arizona. Contributions to this volume have been made by all five participants in that session: James Grice, Richard Hohn, and Jack Martin, in addition to the two individuals who have coauthored this chapter and coedited the volume. This work also contains chapters by two individuals, Donna Tafreshi and Fiona Hibberd, who were not present for the session in Phoenix but had relevant contributions to make. Finally, Lisa Osbeck’s concluding chapter offers a thoughtful critical perspective on each of the other seven contributions and on the volume as a whole.

In what follows, we provide a concise preview of each of the seven topical chapters, discussed in the order in which they have been organized in this book.

A condensed preview of the topical chapters

On the systemic misuse of statistical methods within mainstream psychology

Lamiell’s chapter reiterates his long-running argument against interpreting knowledge of statistical relationships between variables defined only for aggregates of individuals as if it constituted knowledge of the individuals within the aggregates. The chapter provides an overview of key historical developments behind the institutionalization of the field’s problematic interpretive practices, and then explains the inadequacy of recent attempts to defend those practices by invoking (a) probabilistic thinking and/or (b) considerations of practical utility. The thesis is reasserted that knowledge of statistical relationships among variables marking differences between individuals is properly regarded as knowledge of populations and not knowledge of any single one of the individuals within those populations. Such knowledge is, literally, knowledge of no one, and this is true whether the between-person differences being statistically examined are differences that have been captured by tests of one sort or another, as in correlational studies, or differences that have been created through investigator-imposed treatments, as in true experiments. Lamiell contends that through the long-prevalent indulgence of inappropriate interpretations of aggregate statistical knowledge, mainstream thinkers within scientific psychology have gradually but effectively, even if unwittingly, transformed the field into a species of demography he calls ‘psycho-demography,’ and he mourns the persistence within the field, over many decades, of a quite deliberate ‘ignore-ance’ of this conceptual reality and its full implications.

Psychology’s inertia: Epistemological and ethical implications

Hibberd’s contribution takes aim at the persistence of a positivistic ethos within psychology despite the recognition long ago in other disciplines of positivism’s failure as a philosophy of science. The enduring positivistic ethos in psychology, Hibberd argues, sustains psychology’s defective concepts of method and evidence, the false scientific claims that those concepts sanction, misguided research questions and interpretations of results, and, what is perhaps most problematic in the long run, an inadequate system of self-correction. Hibberd argues forcefully that attempts at redress have repeatedly fallen on deaf ears, while a seemingly ingrained not-wanting-to-know—more of the deliberate ‘ignore-ance’ mentioned above—prevails instead. This, Hibberd contends, is unethical and contrary to the disinterested truth-seeking which is a hallmark of genuinely scientific inquiry. Importantly, Hibberd suggests some actions that could help to remediate these serious conceptual problems within psychology and, in the process, enhance the discipline’s integrity and scientific maturity.

Intransigence in mainstream thinking about psychological measurement

Hohn’s focus is on the inertia that has existed for decades within mainstream psychology in the domain of measurement practices. In his chapter, he leads the reader through a discussion of a series of missteps that have occurred within the field, beginning in the late 19th century with the work of the psychophysicists and continuing through the 20th century and now into the 21st. As those missteps unfolded, psychology’s accepted measurement practices became widely discrepant from those prevailing in the natural sciences that, ironically, psychology has all along seen itself as emulating. Although critical expositions of that untoward development have been offered in recent decades, the critiques have largely been ignored, and, today, the questionable assumptions and practices remain firmly entrenched within the field. In his treatment of these matters, Hohn pays particular attention to the relevant efforts of the contemporary Australian scholar Joel Michell.

Persistent disregard for the inadequacies of null hypothesis significance testing and the viable alternative of observation-oriented modeling

Despite a sizable extant literature discussing the problems with null hypothesis significance testing (NHST) as a methodological vehicle for advancing the agenda of psychological science, the practice continues to prevail within the field as the investigative paradigm of choice. In their chapter, Grice, Huntjens, and Johnson draw attention once again to that literature, and to Grice’s own efforts, begun over a decade ago, to develop an alternative framework that would not only avoid the major problems with NHST, but would, in the process, enable investigators to actually accomplish much that most have mistakenly believed has been and is being accomplished with NHST. Despite Grice’s efforts to establish the framework he calls Observation-Oriented Modeling (OOM), however, mainstream inquiry remains dominated by NHST. Persisting in his efforts to overcome this inertia, Grice and his coauthors discuss in this chapter how concepts in OOM can be used to expand upon or supplant contemporary understandings of effect size, causality, measurement, and inference. These authors then bring the conceptual and analytic tools of OOM to bear in a reanalysis of the findings of a previously published paper on inter-identity amnesia in persons diagnosed with Dissociative Identity Disorder.

On the interpretive nature of quantitative methods and psychology’s resistance to qualitative methods

Tafreshi’s contribution to this volume addresses the long-standing and continuing resistance within the mainstream of psychology to the use of qualitative methods. Historically, this resistance has been grounded in the belief that the adoption of qualitative methods would compromise psychology’s status as a science. Tafreshi’s core thesis is that the roots of that resistance are to be found in inadequate education and training in quantitative methods. She advances that thesis by showing the various ways in which the exercise of quantitative methods in psychology, both in the domain of measurement and in the domain of statistical analysis, necessarily incorporates considerations of a fundamentally qualitive nature. She argues that if quantitative methodological approaches were taught to psychology students in a way that emphasizes rather than obscures the interpretative role of the investigator, the inevitable presence of qualitative considerations in all psychological research, including research that incorporates quantitative methods, would be more apparent. This, in turn, would incline coming generations of psychological researchers to be more receptive to the larger possibilities offered by qualitative methods for expanding scientific psychology’s methodological toolbox.

Is there a waning appetite for critical methodology in psychology?

Slaney begins her chapter by noting that, over the years, there has, in fact, been an appreciable amount of scrutiny by critically oriented psychologists of the assumptions and beliefs of their discipline’s prevailing research methods and practices. Unfortunately, she argues, those discussions seem to have had little impact on those working ‘in the trenches’ of psychological research and practice. She is thus led to wonder about the nature and sources of this disconnect. After briefly surveying the history of major critical efforts in psychology, Slaney moves into a discussion of the ways in which psychological science does and does not appear to be methodologically inert. Her analysis is organized around two broad types of methodological critique: restitutive and radical. Critiques of the former type are aimed at achieving correctives in certain of mainstream psychology’s methodological practices without challenging the very methodological foundation of the discipline. Radical methodological critiques, by contrast, strike at just those methodological foundations, often promoting fundamentally different approaches to psychological inquiry. Slaney offers for consideration various exemplars of each of these two types of critique, as well as her assessment of their relative (in)effectiveness to date in leading to disciplinary changes.

Psychology’s struggle with understanding persons

In what would well qualify as a radical critique in Slaney’s framework, Martin argues that the research methods that have long dominated mainstream psychology must finally be recognized as fundamentally ill-suited for a psychology that would aim toward an understanding of particular persons. Complementing Lamiell’s argument (refer above) that psycho-demographic knowledge is, effectively, knowledge of no one, and working in the intellectual patrimony of such well-known predecessors such Wilhelm Dilthey (1833–1911), Wilhelm Windelband (1848–1915), William Stern (1871–1938), Gordon Allport (1897–1967), and others, Martin contends that the sampling and statistical methods of most psychological research impede rather than facilitate an understanding of individual persons, i.e., entities that are (or once were) living, breathing some ones. Such an understanding, to the extent that it can be achieved at all, requires biographical work that illuminates the makeup of real persons’ lives as they are actually lived. Resistance within the mainstream to views of the sort expressed by Martin has prevailed within mainstream thinking at least since the time shortly before the turn of the 20th century when Hermann Ebbinghaus (1850–1909) directly rebutted Dilthey’s call for a verstehende or ‘understanding’ psychology modeled on the human sciences (Geisteswissenschaften). Ebbinghaus firmly defended an erklärende or ‘explanatory’ psychology devoted to the search for general laws on the model of the natural sciences (Naturwissenschaften). In a very real sense, therefore, Martin’s thesis represents a vision for scientific psychology that has been defended by a small minority of psychologists virtually from the outset of the time in the late 19th century when the field was formally established as an independent scientific discipline in the laboratories of Wilhelm Wundt (1832–1920) at the University of Leipzig. For just as long a time, however, mainstream resistance to this vision has been firm, even though mainstream psychology continues to have precious little to show by way of accomplishments on the order to those that have been achieved in the natural sciences.

In her clear and cogent commentary, Lisa Osbeck identifies how the individual contributors to this volume address in common what putatively are the core problems of psychology, and gives reasons and potential solutions for these problems. However, Osbeck illuminates the differences across the individual chapters in terms of which problems are emphasized and which reasons and solutions are offered. Osbeck astutely recognizes the need for the contributors to not fall prey to diagnosing what is in large part a feature of a system that sustains erroneous or ill-fitting methods and practices as an illness suffered only or primarily by individual researchers. Osbeck also appeals for the accompaniment to critique of constructive alternatives, such as Grice’s OOM, a concrete and salient application of which is featured in Chapter 5.

A call for critical responsiveness

We noted at the outset of our discussion that this work arose out of concern over the widespread and long-standing indifference within the mainstream of scientific psychology to penetrating critical analyses of certain of the disciplines most fundamental tenets and investigative practices. Our objective has been to gather into a single volume forceful statements of several different scholars documenting this indifference, and drawing attention to its untoward consequences both for the discipline of scientific psychology itself and, in turn, for the people whose lives are impacted by the knowledge claims invoked by applied psychologists to justify their interventions.

The contributors to this volume harbor no illusion that it will, by some mysterious force, immediately overcome the inertia that has dominated within the mainstream for a great many decades. We do hope, however, that the critical analyses offered here will finally receive the commensurately critical consideration that they have long deserved. Should such consideration reveal the faultiness or otherwise problematic nature of a critic’s arguments, we hope that there will soon appear carefully crafted rejoinders explaining just why this has been seen to be the case.

By the same token, we hope that where careful critical consideration forces recognition of the validity of the critics’ original arguments, acknowledgments of same will likewise soon be forthcoming, followed in due course by the implementation of the changes mandated by the critiques: in research methods, in interventions and other practical applications, and in the textbooks and other materials that will be used to train future generations of psychologists and to communicate the works of the discipline to interested others situated beyond the field’s borders but standing to benefit from the field’s scientific accomplishments.

Our broader hope for this volume is that it will help to revive within the discipline of psychology an appreciation of the inevitable need within this or any other scientific discipline for continuous critical reflection on its basic assumptions and methodological practices, and in the process expel the deliberate ‘ignore-ance’ that now so widely prevails. We firmly believe that if change of this nature can finally occur, psychology will be improved both as a basic scientific discipline and as field of professional practice.

2On the systemic misuse of statistical methods within mainstream psychology

James T. Lamiell

The concern in this chapter is with the troubling inertia among mainstream psychological investigators in the face of repeated critiques of the long-standing practice of regarding statistical knowledge of populations as if it warrants claims to scientific knowledge about the psychological functioning of individuals within those populations. The problematic nature of this common interpretive (mal)practice has been pointed out in various ways in publications dating at least to the middle of the 20th century. Scholar David Bakan (1921–2004) discussed the problem in terms of a widespread conflation by researchers of knowledge of the general with knowledge of the aggregate (cf. Bakan, 1955, 1966). As he correctly stated, “the use of methods which are appropriate to [gaining] the [former] type [of knowledge] in the establishment and confirmation of the [latter] leads to error” (Bakan, 1955, p. 211, brackets added). Bakan’s admonitions were widely ignored.

Subsequently, the well-known research methodologist, Fred N. Kerlinger (1910–1991), dis...

Table of contents

• Cover
• Half Title
• Series Information
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• List of Contributors
• 1 Introduction
• 2 On the systemic misuse of statistical methods within mainstream psychology
• 3 Psychology’s inertia: Epistemological and ethical implications
• 4 Intransigence in mainstream thinking about psychological measurement
• 5 Persistent disregard for the inadequacies of null hypothesis significance testing and the viable alternative of observation-oriented modeling
• 6 On the interpretative nature of quantitative methods and psychology’s resistance to qualitative methods
• 7 Is there a waning appetite for critical methodology in psychology?
• 8 Psychology’s struggle with understanding persons
• 9 Summary and commentary on Scientific Psychology’s Troubling Inertia
• Index