Quantitative and Qualitative Methods in Psychotherapy Research

Wolfgang Lutz, Sarah Knox

• Publisher: Routledge
• Year: 2014
• ISBN:
  9781136733857
• Edition: 1
• Topic: Psychologie
• Subtopic: Psychotherapie

In this collection, international contributors come together to discuss how qualitative and quantitative methods can be used in psychotherapy research. The book considers the advantages and disadvantages of each approach, and recognises how each method can enhance our understanding of psychotherapy.

Divided into two parts, the book begins with an examination of quantitative research and discusses how we can transfer observations into numbers and statistical findings. Chapters on quantitative methods cover the development of new findings and the improvement of existing findings, identifying and analysing change, and using meta-analysis.

The second half of the book comprises chapters considering how qualitative and mixed methods can be used in psychotherapy research. Chapters on qualitative and mixed methods identify various ways to strengthen the trustworthiness of qualitative findings via rigorous data collection and analysis techniques. Adapted from a special issue of Psychotherapy Research, this volume will be key reading for researchers, academics, and professionals who want a greater understanding of how a particular area of research methods can be used in psychotherapy.

1 Quantitative and qualitative methods for psychotherapy research Introduction

Wolfgang Lutz and Sarah Knox

We are pleased that, because the Special Issue on “Quantitative and Qualitative Methods” that appeared in 2009 in the journal Psychotherapy Research was such a success, the publisher (Taylor & Francis) has asked us to develop an update in the form of a book. Many psychotherapy researchers throughout the world have used that special issue to teach classes on psychotherapy research methods and to expose students to the most exciting developments in the area. Now with this volume, such vital content is available as a user-friendly book, thereby providing three advantages: First, the material from the Special Issue has been updated to reflect recent developments in the field since 2009; second, presenting the material in book format is ideally suited for classroom teaching; and third, additional material for some of the quantitative chapters that provides support for researchers seeking to begin their own analysis in a specific area, or that gives helpful links to additional material, can be found on the following website: www.methodsbook.uni-trier.de (User: book; Password: methods).

We are thus excited to offer this volume, and hope that it will be helpful for the field of psychotherapy research. The book can be used as a resource for planning and designing new studies, for analyzing the results of existing data, and as a basis for teaching advanced methods in psychotherapy research.

The volume can be roughly divided into two parts. In the first part, we focus on issues related to quantitative research, which relies on numbers and statistical analyses. Quantitative methods reflect the predominant paradigm used for psychotherapy research since its inception over a hundred years ago. Quantitative methods have become increasingly complex, and hence there are many issues about which the average psychotherapy researcher needs to be aware. In the second part, we focus on qualitative and mixed methods, which have arisen more recently, and offer additional ways of increasing our understanding of psychotherapy. Qualitative methods rely more on words, narratives, and clinical judgment, bringing back some of the humanistic quality to research. At this point, both quantitative and qualitative methods are regarded positively within psychotherapy research, although individual researchers often prefer one approach over the other.

Quantitative methods

The goal of psychotherapy research is to advance our knowledge about the process, as well as the course and the outcome, of psychotherapy. Researchers try to identify the best treatment options possible for patients with a given problem, disorder, or set of problems or disorders. Ideally, then, we can select optimal treatments for individual patients. Quantitative research methods are helpful tools for achieving these goals because they enable us to study the complex relations between the patient, the therapist, the process of therapy, external events in the lives of patients, and in-session progress, post-session progress, and therapy outcome at the end of treatment as well as at follow-up. Such methods also help us aggregate and integrate findings about psycho therapy (e.g., via meta-analysis).

The quality and scope of the 14 chapters on quantitative methods depict the progress in the areas just described. These articles are grouped into three major categories: (a) developing new and improving existing measures in psychotherapy research, (b) identifying and analyzing change in psychotherapy, and (c) aggregating research findings via meta-analysis.

Measure development

The clinical and scientific value of the psychotherapy research enterprise depends on the validity of our measures. One trend that can be seen in the field is the development of new research tools (e.g., IRT or multitrait–multimethod analysis) to improve the validity of our measures.

The papers on measurement issues start with Doucette and Wolf (Chapter 2), who discuss advances in latent trait and item response theory (IRT) and their advantages over classical test theory. In the next paper, Eid, Geiser, and Nussbeck (Chapter 3) discuss multitrait–multimethod procedures and their implications for test validity, multimethod assessment, and psychotherapy research in general. The chapter by Wasserman, Levy, and Loken (Chapter 4) then introduces generalizability theory as a framework within which multiple sources of error can be simultaneously evaluated; generalizability theory allows researchers to improve the accuracy of reliability estimates and provides critical information for the modification of coding procedures in psychotherapy research. Carrig, Kolden, and Strauman (Chapter 5) then discuss functional magnetic resonance imaging (fMRI), a new methodological tool to assess outcome and process in psychotherapy, one with the potential to provide new insights for psychotherapy research.

Identifying and analyzing change

A decade or so ago, researchers were pleased when they were able to demonstrate the average difference between two groups using only pre-post change as put forward by Beutler and Howard (1998) and Newman and Howard (1991). Since that time, several advancements have been made in quantitative methods that allow us to complete more sophisticated analyses, such as examining longitudinal data on the course of psychotherapy, as well as integrating several levels of change (e.g., sessions, patients, treatments) and differences across therapists. These new developments are particularly important because we may soon be fortunate enough to have data for thousands of patients, arising from the new developments in scientist-practitioner networks or patient-focused research (e.g., Castonguay, Barkham, Lutz, and McAleavey, 2013; Lambert, 2001, 2007, 2013; Lutz, 2002). To be able to analyze such vast data, we need new tools to aggregate the information without neglecting interindividual differences.

In the first paper on how to identify and to analyze change, Kazdin (Chapter 6) discusses moderators, mediators, and mechanisms of change, and shows promising lines of work to better identify these components of the change process. This paper pinpoints the conceptual and research difficulties in studying change mechanisms and presents recommendations for future research on how and why therapy works. Anecdotally, this paper was one of the most downloaded papers of the 2009 special series and addresses the pressing questions related to the identification of central change agents in psychotherapy. The set of papers by Tasca and Gallop (Chapter 7), Gallop and Tasca (Chapter 8), and Tasca, Illing, Joyce, and Ogrodniczuk (Chapter 9) introduce fundamentals and complexities of multilevel models (MLM), as well as a three-level growth MLM approach for the analysis of longitudinal data and nested data in general. These modern sophisticated statistical tools allow researchers to model individual change and group change, and provide new opportunities for handling missing data in longitudinal designs and nested designs. Kenny and Hoyt (Chapter 10) extend the multilevel approach to the analysis of group as well as rolling group data. They also give practical guidelines on how to conduct multilevel analyses within a statistical package (SPSS), and include a link to the website that provides the R-syntax on how to conduct such analyses. Tschacher and Ramseyer (Chapter 11) then introduce the methodology of aggregated time-series analysis (time-series panel analysis, TSPA), which allows for the identification of prototypical and fine-grained process patterns to approximate causal dynamic structures.

The two final papers in this section focus on the evaluation of progress. We start with a debate about a common method to assess and evaluate clinically significant change (Lambert and Ogles, Chapter 12). These authors suggest using Jacobson and Truax’s (1991) method as a standard way of defining clinical significance in psychotherapy research. The use of a definition and classification of clinically meaningful change on an individual basis is essential in reporting and communicating results in efficacy and effectiveness studies, as well as in patient-focused research.

Lutz, Stulz, Martinovich, Leon, and Saunders (Chapter 13) then present different kinds of decision rules as the basis for the evaluation of progress and the application of feedback tools. Rational as well as empirical approaches are discussed, and examples as well as material for the application of Growth Mixture Models (GMM) using Mplus are provided.

The aggregation of research findings via meta-analysis

Finally, the field now benefits from new developments in empirically aggregating information over many studies via meta-analysis. In comparison to the original meta-analysis by Smith, Glass, and Miller (1980), several advancements have been made that allow us to more precisely define weighted effect size, the statistical significance of effects, as well as tests of homogeneity (including file-drawer analysis and moderator variables). Meta-analysis is an important tool, one that not only allows researchers to aggregate information over hundreds of studies, but also allows them to demonstrate the efficacy and effectiveness of psychotherapy in comparison to other treatments (e.g., medical, psychopharmacological).

Berkeljon and Baldwin (Chapter 14) provide an introductory tutorial about conducting meta-analysis in psychotherapy outcome research. Their central topics involve identifying and collecting studies, coding effect sizes, coding substantive and methodological information, combining effect sizes, and interpreting effect sizes; moderator analyses are also introduced. Finally, Diener, Hilsenroth, Weinberger, and Monroe (Chapter 15) provide a primer on using meta-analysis for correlation coefficients. Based on an example of the relationship between patient-reported therapeutic alliance and adult-attachment style, they demonstrate aspects and calculations of the weighted average effect size, the statistical significance of effects, a test of homogeneity, confidence intervals, and file-drawer analysis.

Qualitative and mixed methods

Despite the many strengths of rigorous quantitative methods, some researchers have long expressed dissatisfaction with such empirical approaches, particularly for investigating psychotherapy process (e.g., Goldman, 1976, 1979). The standard for such research in those days was that you could only publish something if you observed it and measured it reliably and validly. Unfortunately, this demand often forced researchers to study relatively trivial things because they could easily be seen and coded (e.g., head nods), and left unexamined the vast amount of clinically rich data from psychotherapy. Kiesler (1973) aptly summarized this dilemma: “If you can’t count it, it doesn’t count; if you can count it, that ain’t it” (p. 16). This rift led many clinicians to bemoan the gap between science and practice (see Morrow-Bradley and Elliott, 1986).

As a result, some in the field of psychotherapy research became excited when learning of qualitative methods used by colleagues in education and anthropology. Indeed, many people wrote about the promise of qualitative research to enhance our understanding of psychotherapy (Borgen, 1992; Hill and Gronsky, 1984; Hoshmand, 1989; Howard, 1983; Neimeyer and Resnikoff, 1982; Polkinghorne, 1984). Although it took some time before people developed rigorous approaches suitable for psychotherapy research, we now have a number of sound methods: comprehensive process analysis (Elliott, 1989), consensual qualitative research (Hill et al., 2005; Hill, Thompson, and Williams, 1997), grounded theory (Rennie, Phillips, and Quartaro, 1988; Strauss and Corbin, 1990, 1998), and phenomenological approaches (Giorgi, 1985).

Methodological developments in qualitative research

Qualitative psychotherapy research methods continue to evolve and be refined. We highlight here new developments within task analysis (Pascual-Leone, Greenberg, and Pascual-Leone, Chapter 16), a method that “pushes the envelope” in combining aspects of qualitative and quantitative approaches to explain therapy processes. Task analysis procedures have existed for some time (see Greenberg, 2007), and these authors describe how this approach can be used in a programmatic way to study client change processes. In addition, Elliott et al. (Chapter 17) present a mixed-method approach that involves hermeneutic case studies, borrowing from legal processes to use arguments to determine the weight of the evidence about whether change has occurred in psychotherapy. Next, Schielke, Fishman, Osatuke, and Stiles (Chapter 18) present an intriguing new approach borrowed from architecture (called the Ward method) to help researchers more effectively integrate the multiple voices inherent on a research team as they seek to understand phenomena.

A major complaint about ...

Table of contents

• Cover Page
• Half-Title Page
• Title Page
• Copyright Page
• Table of Contents
• List of illustrations
• Notes on contributors
• Preface
• List of abbreviations
• 1 Quantitative and qualitative methods for psychotherapy research: introduction
• Part I Quantitative methods
• Part II Qualitative methods
• Part III Epilloge
• References
• Supplementary online material
• Author index
• Subject index