A Tale of Two Cultures

Qualitative and Quantitative Research in the Social Sciences

Gary Goertz, James Mahoney

• Publisher: Princeton University Press
• Year: 2012
• ISBN:
  9781400845446
• Topic: Social Sciences
• Subtopic: Sociology
• Index: Social Sciences

Some in the social sciences argue that the same logic applies to both qualitative and quantitative methods. In A Tale of Two Cultures, Gary Goertz and James Mahoney demonstrate that these two paradigms constitute different cultures, each internally coherent yet marked by contrasting norms, practices, and toolkits. They identify and discuss major differences between these two traditions that touch nearly every aspect of social science research, including design, goals, causal effects and models, concepts and measurement, data analysis, and case selection. Although focused on the differences between qualitative and quantitative research, Goertz and Mahoney also seek to promote toleration, exchange, and learning by enabling scholars to think beyond their own culture and see an alternative scientific worldview. This book is written in an easily accessible style and features a host of real-world examples to illustrate methodological points.

Chapter 1

Introduction

In this book, we explore the relationship between the quantitative and qualitative research traditions in the social sciences, with particular emphasis on political science and sociology. We do so by identifying various ways in which the traditions differ. They contrast across numerous areas of methodology, ranging from type of research question, to mode of data analysis, to method of inference. We suggest that these differences are systematically and coherently related to one another such that it is meaningful to speak of distinct quantitative and qualitative research paradigms.

We treat the quantitative and qualitative traditions as alternative cultures. Each has its own values, beliefs, and norms. Each is associated with distinctive research procedures and practices. Communication within a given culture tends to be fluid and productive. Communication across cultures, however, tends to be difficult and marked by misunderstanding. When scholars from one tradition offer their insights to members of the other tradition, the advice is often viewed as unhelpful and inappropriate. The dissonance between the alternative cultures is seen with the miscommunication, skepticism, and frustration that sometimes mark encounters between quantitative and qualitative researchers. At its core, we suggest, the quantitative–qualitative disputation in the social sciences is really a clash of cultures.

Like all cultures, the quantitative and qualitative ones are not monolithic blocks (see Sewell (2005) for a good discussion of the concept of “culture”). They are loosely integrated traditions, and they contain internal contradictions and contestation. The particular orientations and practices that compose these cultures have changed over time, and they continue to evolve today. The two cultures are not hermetically sealed from one another but rather are permeable and permit boundary crossing. Nevertheless, they are relatively coherent systems of meaning and practice. They feature many readily identifiable values, beliefs, norms, and procedures.

By emphasizing differences between qualitative and quantitative research, this book stands in contrast to King, Keohane, and Verba's work, Designing Social Inquiry: Scientific Inference in Qualitative Research. They famously argue that “the differences between the quantitative and qualitative traditions are only stylistic and are methodologically and substantively unimportant” (1994, 4). They believe that the two traditions share a single logic of inference, one that can be largely summarized in terms of the norms of statistical analysis. The differences between the two traditions that they identify concern surface traits, especially the use of numbers versus words.

We reject the assumption that a single logic of inference founded on statistical norms guides both quantitative and qualitative research. Nor do we believe that the quantitative-qualitative distinction revolves around the use of numbers versus words. Instead, we see differences in basic orientations to research, such as whether one mainly uses within-case analysis to make inferences about individual cases (as qualitative researchers do) or whether one mainly uses cross-case analysis to make inferences about populations (as quantitative researchers do). We even suggest that the two traditions are best understood as drawing on alternative mathematical foundations: quantitative research is grounded in inferential statistics (i.e., probability and statistical theory), whereas qualitative research is (often implicitly) rooted in logic and set theory. Viewing the traditions in light of these contrasting mathematical foundations helps to make sense of many differences that we discuss in this book.

In pointing out basic divergences, our goal is not to drive a wedge between the quantitative and qualitative research paradigms. To the contrary, we seek to facilitate communication and cooperation between scholars associated with the different paradigms. We believe that mutual understanding must be founded upon recognition and appreciation of differences, including an understanding of contrasting strengths and weaknesses. We advocate boundary crossing and mixed-method research when questions require analysts to pursue goals characteristic to both the qualitative and quantitative paradigms. At the same time, we respect and do not view as inherently inferior research that stays within its own paradigm. There is a place for quantitative, qualitative, and mixed-method research in the social sciences.

One lesson that grows out of this book is that asking whether quantitative or qualitative research is superior to the other is not a useful question. King, Keohane, and Verba (1994, 5–6) also state that “neither quantitative nor qualitative research is superior to the other.” However, they arrive at this conclusion only because they believe qualitative methods must be used as a last resort when statistical analysis is not possible.¹ By contrast, we believe that quantitative and qualitative techniques are appropriate for different research tasks and are designed to achieve different research goals. The selection of quantitative versus qualitative techniques is not a matter of the data that happen to be available. Rather, for some research goals, quantitative methods are more appropriate than qualitative techniques, and qualitative methods are more appropriate than quantitative methods for other research questions. Depending on the task, of course, it may well be the case that the analyst must draw on both kinds of techniques to achieve his or her goal. Mixed-method research that combines quantitative and qualitative techniques is essential for many complex research projects whose goals require analysts to draw on the orientations and characteristic strengths of both traditions.

Like some anthropologists who study other cultures, we seek to make sense of research practices while maintaining a kind of neutrality about them. Our goals are mainly descriptive, not primarily normative or prescriptive. Certainly, the methods of the two traditions are not beyond criticism. However, we believe that the critique and reformulation of methods works best within a given tradition. Thus, statistical methodologists are the scholars most qualified to improve statistical methods, whereas qualitative methodologists are the scholars best positioned to improve qualitative methods. We find that many existing “cross-cultural” criticisms, such as critiques of quantitative research by qualitative scholars, are not appropriate because they ignore the basic goals and purposes of research in that tradition. What appears to be problematic through one set of glasses may make good sense through the lenses of the other tradition.

In telling a tale of these two cultures, we often end up considering how lesser-known and implicit qualitative assumptions and practices differ from well-known and carefully codified quantitative ones. This approach is a by-product of the fact that quantitative methods, when compared to qualitative methods, are more explicitly and systematically developed in the social sciences. Quantitative methods are better known, and the quantitative culture is, no doubt, the more dominant of the two cultures within most social science fields. As such we devote more space to a discussion of qualitative methods. Yet the approach throughout remains clarifying what is distinctive about both traditions while avoiding invidious comparisons.

Why Two Cultures?

King, Keohane, and Verba suggest that there is a single logic of inference—one basic culture—that characterizes all social science, both quantitative and qualitative. An alternative, “many cultures” view might hold that the quantitative and qualitative traditions are heterogeneous groups with many variants and subcultures within each. Indeed, each paradigm—like any culture—features big divisions as well as smaller ones. For example, historically within the statistical paradigm, one big division was between the classical, frequentist school and the Bayesian approach to statistical analysis (e.g., see Freedman 2010 and Jackman 2009). Other smaller divisions—over issues such as the utility of fixed effect models or the number of independent variables that should be included in a statistical model—exist among scholars who may agree on larger issues such as the frequentist versus Bayesian debate.

Likewise, the qualitative paradigm includes many divisions. Perhaps the biggest split concerns the differences between scholars who work broadly within the behavioral tradition and who are centrally concerned with causal inference versus scholars associated with various interpretive approaches. These two big tents each have their own subdivisions. For example, qualitative scholars who embrace the goal of causal inference may disagree on the relative importance of specific tools, such as counterfactual analysis or Qualitative Comparative Analysis (QCA). Likewise, within the interpretive camp, there are differences between scholars who embrace interpretive analysis à la Clifford Geertz (1973) and scholars who advocate critical theory and poststructural approaches.

Our two cultures approach shares certain similarities with King, Keohane, and Verba's one culture approach, especially in that we focus on research that is centrally oriented toward causal inference and generalization. The methods and techniques that we discuss are all intended to be used to make valid scientific inferences. The employment of scientific methods for the generation of valid causal inferences, above all else, unites the two research traditions discussed in this book.

One consequence of our focus on causal inference is that important currents within the qualitative paradigm drop out of the analysis. In particular, interpretive approaches are not featured in our two cultures argument. These approaches are usually less centrally concerned with causal analysis; they focus more heavily on other research goals, such as elucidating the meaning of behavior or critiquing the use of power. The interpretive tradition has its own leading norms and practices, which differ in basic ways from the quantitative and qualitative paradigms that we study in this book. One could certainly write another book focusing on the ways in which the interpretive culture contrasts with the “causal inference” cultures that we discuss. Such a book would bring to light fundamental clashes over epistemology and ontology that exist within parts of the social sciences. In this book, however, we focus on scholars who agree on many basic issues of epistemology and ontology, including the centrality of causal analysis for understanding the social world.²

There are various reasons why it makes sense to focus on these two traditions of research. For one thing, the qualitative–quantitative distinction is built into nearly everyone's vocabulary in the social sciences, and it serves as a common point of reference for distinguishing different kinds of work. Nearly all scholars speak of qualitative versus quantitative research, though they may not understand that contrast in the same way. Even scholars, such as ourselves, who feel that the labels “quantitative” and “qualitative” are quite inadequate for capturing the most salient differences between the two traditions still feel compelled to use this terminology.

Furthermore, social scientists have organized themselves—formally and informally—into quantitative and qualitative research communities. In political science, there are two methodology sections, the Section on Political Methodology, which represents quantitative methodology, and the newer Section on Qualitative and Multi-Method Research. In sociology, the Section on Methodology stands for mainly quantitative methods, whereas the kinds of qualitative methods that we discuss are associated with the Section on Comparative and Historical Sociology. Leading training institutes reflect the two culture division as well: the Interuniversity Consortium for Political and Social Research (ICPSR) provides almost exclusively quantitative training, whereas the Institute for Qualitative and Multi-Method Research (IQMR) focuses on qualitative and mixed-method research.

Our goal in this book is not to turn quantitative researchers into qualitative researchers, or vice versa. However, we do seek to increase the number of scholars who understand the norms and practices—and their rationales—of both cultures of research. We believe that overcoming the quantitative-qualitative division in the social sciences is significantly a matter of better understanding the methodological differences between these two traditions along with the reasons why those differences exist.

Characterizing and Comparing the Two Cultures

In discussing the quantitative and qualitative traditions, we draw on various data sources and focus on certain kinds of practices and not others. In this section, we briefly describe our approach to characterizing and comparing the two cultures.

Types of Data

Our characterizations of research practices derive from three kinds of data. First, we rely on the literature concerning quantitative and qualitative methodology. Methodologists often do an excellent job of making explicit the research techniques used in a given tradition and the rationale behind these techniques. For the quantitative paradigm, we make much use of text-books written by prominent scholars in the fields of statistics, econometrics, and quantitative social science. Our presentation draws heavily on literature concerning the Neyman-Rubin-Holland model and the associated “potential outcomes” framework (e.g., Angrist and Pischke 2009, Berk 2004, Freedman 2010, and Morgan and Winship 2007). We also reference the literature on experimental research in the social sciences when relevant. For the qualitative paradigm, our discussion is grounded in the “classic cannon” of work associated with scholars such as Giovanni Sartori, Alexander George, and David Collier. In addition, we utilize many insights from the work of Charles Ragin. At the end of each individual chapter, we recommend books and articles that one might read to explore further the differences discussed in the chapter.

Second, we use exemplary quantitative and qualitative studies to illustrate the distinctions that we discuss in the individual chapters. These studies are not only useful as examples, but also as sources of insight about characteristic practices in the two cultures. Some of these exemplars engage topics that are important to both research cultures, such as the study of democracy. Looking at the same topic as treated in exemplary studies from each culture allows us to illustrate more vividly the different kinds of questions and methods that animate the two cultures. At the same time, however, one of our key points is that some topics are more easily addressed in one culture than the other. Hence, some of our examples do not extend across both cultures.

Third, we also sampled and coded a large number of research articles from leading journals in political science and sociology. The items coded and the results are summarized in the appendix. This large-N sample was intended to be representative of good work—as defined by appearance in major journals in political science and sociology. The sample provides a further basis for generalizing about leading research practices. For example, when we make assertions such as the claim that quantitative researchers often include several control variables in their statistical models, it is based on results from our survey.

Explicit and Implicit Practices

Our discussion focuses on the dominant methodological practices in the quantitative and qualitative paradigms. In general, when discussing quantitative research, we focus on explicit practices that follow well-established advice from the methodological literature. Quantitative research methods and procedures are often clearly specified, and quantitative researchers often quite explicitly follow these well-formulated methodological ideas.

At many points, nevertheless, we discuss assumptions and procedures in the quantitative tradition that are usually implicit. The comparison of quantitative research to qualitative research calls attention to underlying norms and practices in both traditions that otherwise might go unnoticed. For example, by considering the asymmetry assumptions of many qualitative methods, the extent to which most quantitative methods implicitly assume symmetric relationships becomes more visible. Systematic comparison of the paradigms helps bring to light research practices that are often taken for granted.

Our treatment of qualitative research focuses more heavily on a set of implicit procedures and techniques. In general, qualitative methods are used far less explicitly when compared to quantitative methods. At this stage, in fact, the implicit use of methods could be seen as a cultural characteristic of qualitative research. To describe this research tradition, we must reconstruct the procedures that qualitative researchers use when doing their work. Our reconstruction draws on a broad reading of qualitative studies, including an effort at systematically coding qualitative research articles. In addition, the practices that we describe are consistent with other methodological texts that have worked to make explicit and codify qualitative research practices (e.g., Brady and Collier 2010; George and Bennett 2005; Ragin 1987). Nevertheless, because qualitative methods are often used unsystematically, certain characterizations of this tradition will inevitably be controversial. In the text, we try to indicate areas where our description of dominant practices in qualitative research mi...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• Preface
• 1. Introduction
• 2. Mathematical Prelude: A Selective Introduction to Logic and Set Theory for Social Scientists
• I. Causal Models and Inference
• II. Within-Case Analysis
• III. Concepts and Measurement
• IV. Research Design and Generalization
• Appendix
• Name Index
• Subject Index