Experiencing Social Research

A Reader

Kerry Strand, Gregory Weiss, Kerry J. Strand, Gregory L. Weiss

• Publisher: Routledge
• Year: 2020
• ISBN:
  9781000150230
• Edition: 1
• Topic: Social Sciences
• Subtopic: Sociology
• Index: Social Sciences

This reader introduces students to the social research process by pairing 16 published research articles with candid interviews with the lead researcher on each study.

chapter 4
Experimental Research

Experimentation is the research technique that is at the core of scientific investigation because it offers the most powerful way to test causal relationships—that is, to determine that one variable (the treatment or independent variable) is truly affecting another (the outcome or dependent variable). The reason is the amount of control that researchers have in experimental research—control over the nature and timing of the independent variable and control for outside factors that might also have an impact on the dependent variable and thus confound interpretation.

Despite its obvious advantages, the experimental method does have some limitations in social science research. The three most important limitations are that it is only appropriate for micro-level studies (it is difficult to study large social forces with this method), that it allows one to examine only a very limited number of variables at a time, and that it can be difficult to study a large or representative sample (making generalizability difficult).

Despite these limitations, there are research questions for which the experimental method is very appropriate and useful. Two are illustrated in the articles that we include here, each of which also exemplifies one of the major kinds of experimental design: the laboratory experiment and the field experiment. Laboratory experiments are conducted in very controlled conditions that are established within a laboratory or some other specially created location. Unlike field experiments, which take place in natural settings, lab experiments maximize the control of the researcher in assigning subjects to groups, administering the pretest and posttest (measures of the outcome variable), and determining exactly what the treatment variable will be and how and when it will be administered. On the other hand, to the extent that laboratory experiments are conducted in artificial environments in which subjects know that they are being studied, the issue of reactivity (that is, respondents behaving in an abnormal way when they know they are being studied) is a concern. Thus, laboratory experiments may leave us wondering if the results of the experiment—say, the way people behaved when they were subject to the treatment—would be the same if the situation occurred naturally in their everyday environment.

In the first article, John Zipp’s classroom provides a good laboratory in which to examine the effectiveness of a teaching technique, and the experimental method is the most logical research technique to use for this study. Zipp wants to determine if one specific technique (his use of group discussion) does, in fact, help to achieve a clearly identifiable and measurable learning outcome: students’ understanding of the impact of social structure on decisions about marriage. Specifically, he wants to know if it leads to greater learning than traditional lecture, which is what the control group experiences. Like all conscientious professors, Zipp wants to teach effectively. Although instructors may find out something about their teaching effectiveness in a variety of ways—course evaluations, student performance on examinations and papers, and personal observations and impressions—none of these sources offers the same ability to determine the specific effects of a particular teaching strategy on student learning. In his interview, Zipp not only talks about the process of designing and implementing this study, but he also emphasizes how important it is for instructors to engage in the “scholarship of teaching and learning”—conducting research on curriculum and pedagogy and on how they influence student learning.

The study described in the second article is an example of a field experiment in which researchers use a real-world setting. Field experiments involve either inserting some manipulation (the treatment variable) into the everyday course of events and then observing any influences on people’s behavior, or simply measuring the outcomes of a naturally occurring change in the social world. The study by Richard Scribner and Deborah Cohen is an example of the latter. Their goal is to assess the effectiveness of strategies to curb underage drinking. They note that most efforts have focused on young people themselves (e.g., arresting them for underage drinking), and they wonder if a strategy that focuses more on discouraging retailers from selling alcohol to underage youth might be more effective. They determined that a field experiment would be best, in part because this topic can really only be studied in real-world settings.

The state of Louisiana offered a naturally occurring quasi-experimental situation. From 1986 to 1995, Louisiana’s underage drinking deterrence policy only targeted the young people purchasing or using alcohol. When this strategy was changed in 1995 to permit both the citation of alcohol retailers caught selling to minors and publicity about these citations, Scribner and Cohen created an experiment around this situation. Their objective was to determine if the routine issuing of citations to retailers who violated the law (and publicity about the citations) influenced the likelihood of compliance with the law. As you will see, Scribner and Cohen conclude that this deterrence strategy of issuing citations to merchants who do not comply with the law works. Their study design relied on analysis of the effects of a naturally occurring intervention in a real-world setting. As is always the case with field experiments, they lost some of the control available in laboratory experiments, which somewhat undermined the internal validity of their study. However, they felt confident that the experiment produced genuine results because the alcohol merchants did not know they were being studied, so they behaved in ways that we can assume were normal.

Finally, note how in both articles the authors explicitly identify possible limitations of their studies and encourage the use of caution in interpreting their results. These are signs of a good scientific researcher: a willingness to be self-critical, to be honest, and to be very cautious in accepting research results. Scribner also tells us much about the politics of social research—in this case, how political pressures can be brought to bear on researchers whose investigation may challenge political office holders, corporations, and even prevailing paradigms within certain academic fields.

The Impact of Social Structure on Mate Selection

An Empirical Evaluation of an Active-Learning Exercise

John F. Zipp

The individualistic orientation of most U.S. college students presents a persistent problem for teaching sociology, especially at the introductory level where many students find it hard to understand social structure and how it shapes their lives. This paper provides an empirical evaluation of whether an active-learning exercise focused on mate selection increases student understanding of the impact that social structure has on marital choice. I fielded the experiment in an introductory sociology course—half the class participated in the active-learning exercise while the other half attended a traditional lecture. Results indicate that those who participated in the exercise were significantly more likely to learn the role that social structure plays in mate selection.

The individualistic orientation of most U.S. college students presents a persistent problem for teaching sociology, especially at the introductory level. Put simply, many students find it hard to understand social structure and how it shapes their lives (e.g., Bohmer and Briggs 1991; McCammon 1999). As Lemert (1997:138–39) remarked, social structure tends to be “invisible.” Although there are no hard data on this, discussions with colleagues and reactions to an earlier version of this paper suggest that many sociologists use dating or mate selection as one way to make social structure more visible to students in introductory courses. O’Brien and Foley’s (1999) recent Teaching Sociology article describes a dating game aimed at helping students understand the impact of social structure on their lives. I have used an exercise called “choosing a mate” (described in detail below) for seven years to do the same.

The premise underlying these and other similar exercises is that they increase student learning over and above a more traditional lecture format. Although this is eminently testable, there do not appear to be any such tests in the literature. This is a sizable shortcoming, for at least two reasons. First, without such models, more traditionally-oriented colleagues can dismiss innovative classroom practices as providing too much entertainment and too little scholarly rigor. Indeed, after I described my own exercise to one such colleague, gently implying that he might consider using it, he dismissively informed me that he “had too much material to cover” to waste his time on these “sorts of ‘games.’” Second, and more generally, evidence of this sort is at the heart of the recent emphasis on the scholarship of teaching and learning (SoTL), an effort that seeks to fundamentally re-shape the academy (e.g., Atkinson 2001; Boyer 1990; Hutchings and Shulman 1999). Although there are almost as many definitions of SoTL as there are practitioners, common to most approaches is the call for scholarly investigation about how various teaching methods and approaches impact student learning.

Thus, the purpose of this paper is to provide such a test. Using my own “choosing a mate” exercise; I assess whether it can improve student learning, at least in the short run, over the delivery of the same material in a traditional lecture format. To accomplish this I split my mass lecture class, with one-half the class participating in the “choosing the mate” exercise and the other half receiving a lecture on the topic. At the end of each session, students completed a short “minute paper” (Angelo and Cross 1993) about (among other topics) the most important thing that they learned in class that day. The question at hand, then, is whether students who participate in the exercise are more likely to report learning the role that social structural factors play in marital choice than are students who attend the lecture. Since exercises like this may be relatively common, it is essential that sociologists have some evidence as to their impact on student learning.

“Choosing a Mate:” Fielding the Study

Before describing the experiment, it is worthwhile to note that, for a number of reasons, the choice of whom to date and whom to marry provide good ways to illustrate the effects of social structure. First, dating and marriage are relatively salient to a large number of college students; some students are married, others are engaged, while many of the rest are actively looking for life partners. Second, as the literature on the family makes clear, social structural factors—especially race, age, religion, ethnicity, socioeconomic status, and geographical propinquity—play a key role in how one chooses a mate. This is usually discussed under the rubric of “marital homogamy,” “marital similarity,” or “assortative mating” (e.g., Cherlin 1999; Coltrane and Collins 2001; Kalmijn 1998). Third, even with increases in cohabitation and other family forms, projections indicate that approximately 90 percent of whites and 70 to 75 percent of African Americans will eventually marry (Cherlin 1999:248).

The main goal of this paper is to determine if this active-learning experiment enhanced students’ learning about the role that social structural factors play in mate selection. There are good reasons to expect that active learning will have such an impact. Considerable literature exists affirming that active learning increases student understanding of course material, enhances critical thinking skills, and improves attitudes toward the class and relationships with classmates (e.g., Bransford, Brown, a...

Table of contents

• Cover
• Title Page
• Copyright Page
• Contents
• Preface
• About the Authors
• Overview of Social Research
• Quantitative and Qualitative Approaches