Laboratory Experiments in the Social Sciences

Murray Webster, Jane Sell

• Publisher: Academic Press
• Year: 2014
• ISBN:
  9780124051867
• Edition: 2
• Topic: Education
• Subtopic: Education General

While there are many books available on statistical analysis of data from experiments, there is significantly less available on the design, development, and actual conduct of the experiments. Laboratory Experiments in the Social Sciences summarizes how to design and conduct scientifically sound experiments, be they from surveys, interviews, observations, or experimental methods. The book encompasses how to collect reliable data, the appropriate uses of different methods, and how to avoid or resolve common problems in experimental research. Case study examples illustrate how multiple methods can be used to answer the same research questions and what kinds of outcome would result from each methodology. Sound data begins with effective data collection. This book will assist students and professionals alike in sociology, marketing, political science, anthropology, economics, and psychology.

• Provides a comprehensive summary of issues in social science experimentation, from ethics to design, management, and financing
• Offers "how-to" explanations of the problems and challenges faced by everyone involved in social science experiments
• Pays attention to both practical problems and to theoretical and philosophical arguments
• Defines commonalities and distinctions within and among experimental situations across the social sciences

Part I

Designing and Conducting Experiments

Introduction

This revised and expanded edition of Laboratory Experiments in the Social Sciences reflects thorough rewriting of nearly all the chapters. Authors have included new material and examples from recent experiences, and they have rewritten several passages to improve accessibility without sacrificing technical rigor where that is needed. These chapters present the most useful current information about how to conduct high-quality experiments in the social sciences.

The book is divided into three parts or sections. This first Part I contains seven chapters introducing the methods, the philosophical and ethical foundations and some controversies associated with experiments, and the practical concerns that investigators face in developing and implementing experimental designs.

Chapter 1, by the editors, tells what experiments are and what they are not, how the method came into the social sciences, compares experiments to other common methods for collecting and interpreting data, and considers strengths and weaknesses of the method. This chapter describes our understanding of the value of experiments for developing sociological knowledge.

Chapter 2, by Karen A. Hegtvedt, presents information on protecting human participants in experimental research. This chapter distills knowledge from her career conducting experimental studies of justice and emotions, from advising faculty members as Chair of Emory University’s Department of Sociology, as editor of the leading sociological journal reporting experimental research, and, most directly relevant, her years of service as a member and Chair of Emory University’s institutional review board (IRB) that reviews all procedures and oversees research to ensure the welfare of participants. Every institution conducting research has an IRB, and experimenters must learn to successfully interact with them and to ensure protection of experimental participants. Hegtvedt describes ethical requirements and principles as they apply to this type of work. She reviews recent work analyzing sources of potential problems in research. She also advises on ways to promote sensitive, ethical, and methodologically sound experimental research and to work collaboratively with an IRB.

Chapter 3, by Shane R. Thye, addresses the skeptical or cynical views some social scientists still have of experimentation and shows that many of those are based on outdated views of experimentation. He describes the most important threats to good experimental design and ways to avoid them or to compensate for uncertainty they can produce. This chapter explains the importance of theoretical foundations for experiments (rather than simply trying to “seek what will happen”), explores conditions for inferring causality from different kinds of evidence, and shows why properly designed experiments offer a strong likelihood of producing an increase in understanding.

Chapter 4, by Robert K. Shelly, describes how he and other skilled experimenters select, train, and manage members of research teams who conduct experiments. The research team is not only the face of the research organization for participants but also the source of information an experimenter uses in deciding success or failure of an experiment and of its underlying theoretical ideas. An experiment is a simplified situation, and so details, including dress and speech habits of experimenters, can have great importance. Shelly likens experiments to theatrical performances in which an alternate reality is created and sustained for a particular purpose. Experiments go beyond theater, however, in that a researcher must consider establishing authority relations and mentoring among the research team, among other considerations.

Chapter 5, by Will Kalkhoff, Reef Youngreen, Leda Nath, and Michael J. Lovaglia, addresses some neglected topics in experimental design: who the participants will be, how to recruit them, and what happens after they volunteer for research. These authors summarize a considerable body of recent research on effects of seemingly small differences in procedures associated with recruiting and working with participants. They consider the benefits and costs to participants, and they offer suggestions for maximizing the former as part of good research design. The authors describe the many steps in finding and contacting reliable sources of participants, deciding which of them are suitable for an experimental project, record keeping, and interpersonal relations with participants.

Chapter 6, by Lisa Slattery Walker, is designed to help a research program move from theoretical ideas to experimental research. In other words, once you have some good research questions, how do you develop an experimental design to answer them? This chapter includes a wide range of suggestions and topics that a researcher must address in developing experiments. These include deriving the most useful empirical hypotheses, thinking about independent and dependent variables, pretesting designs and procedures, and analyzing experimental data. Power analyses to decide needed sample size and experimenter effects on the data receive extended treatments.

Chapter 7, by Kathy J. Kuipers and Stuart J. Hysom, describes a number of practical problems that can occur in setting up experimental research and provides solutions for them. Challenges faced include everything from issues of experimental design to dealing with others in the organizational environment, what to do when scheduled participants fail to show up, and managing the funds to pay for their participation. This chapter includes discussions of ways to assess whether a participant was actually in the social situation that the experimenter wished to create or whether disbelief or misunderstanding seriously affected his or her interpretations and behavior. It also offers suggestions for ways to enlist participants in the research enterprise so that they do not, for instance, tell future participants details about the project that would skew the outcomes.

Chapter 1

Why Do Experiments?

Murray Webster, Jr. University of North Carolina–Charlotte, Charlotte, North Carolina

Jane Sell Texas A&M University, College Station, Texas

Abstract

Laboratory experiments in social science developed most rapidly in the years since the end of World War II, fostered by the growth of technology for observation and recording. Experiments offer powerful advantages for testing predictions, although their advantages and proper uses, and relations to other kinds of research designs, still are not well understood by many social scientists. Experiments are most useful when investigating predictions derived from explicit theories, and it is theories, rather than experimental results, that are properly used to explain features of natural settings. Theoretical foundations, concrete and theoretical concepts, abstract design, operations, and interpretation of outcomes all are parts of experimental research programs. Although not every ­research question lends itself well to experimental research, when questions are formulated abstractly, the range of experimental usefulness is much broader than many people appreciate.

Key Words

Experimental design

operations

artificiality

generalization

I A brief history of experiments

Many social scientists, and most physicists, chemists, and biologists, see experimental methods as one of the defining characteristics of scientific inquiry. Although the experiment is far from being the only research method available to the social sciences, its usage has grown remarkably in the years since World War II. Many historical changes are associated with the growth in experimental methods, of which two kinds of changes are especially important: new topics and new technology.

In early decades of the 20th century, sociologists were largely occupied with classifying types and growth of societies or with development of different parts of cities. Following that war, many social scientists became interested in phenomena that can be studied experimentally. In sociology and social psychology, for instance, topics such as interpersonal influence, distortions of judgment, and conformity processes seemed more pressing than they had seemed before authoritarian and repressive societies were common topics. With the new topics came new theories, many of them amenable to experimentation. Economists began to conceptualize strategic game-playing and became interested in behavioral economics; political scientists developed rational choice theories of voting choice; communications scientists began to understand influence processes; sociologists had new theories of social exchanges; and psychologists, whose discipline had used experiments from its beginnings, expanded its study of effects of social factors that appear in the presence of one or more other individuals. New topics and new theory were both congenial to the development of experimental methods in social science.

The second factor was new technology. Starting at a few universities, experimental laboratories were built, followed by laboratories in government and at private research firms. New laboratories required and facilitated development of many kinds of technological advance: coding schemes to record discussion groups, one-way mirrors and, later, television and computers to observe and control communication among researchers and experimental participants, sound and video recorders, and many other elements of contemporary experiments began developing in the years following World War II.

Although experiments are a recognized part of today’s social science research techniques, for many social scientists they are still not well understood. Training in laboratory experimentation is still not part of the graduate training of the majority of social scientists (psychology may be the exception). That is unfortunate for many reasons. Those who might wish to conduct experimental research may not feel confident enough in their skills to approach this method. Social scientists who use other methods, such as survey researchers in sociology, may misunderstand the goals and uses of experiments. Because every science relies on peer review of research, misunderstandings can slow the accumulation of knowledge; good experiments may be criticized on inappropriate grounds, and real flaws in an experiment may be overlooked.

With the continuing growth and development of experimental methods in social science, it will not be long before understanding experiments is an important part of every social scientist’s professional skills. We and the other authors in this book hope to contribute to that understanding. For new experimenters, we offer suggestions to improve the quality of their work; for those who read and wish to assess experimental research, we describe techniques and offer guidelines. In these ways, we hope to contribute to the growing quality of experimental research in social science. A poorly designed experiment will either produce no results or, worse, will produce results that are not what an experimenter thinks they are. This book brings together “best practices” by several of today’s outstanding experimental researchers. The chapters can be read as “how to” manuals for developing one’s own experiments or as sources of criteria to judge and improve the quality of experimental research by practitioners and by the professional audience. All of the chapters contain background to their individual topics that explicitly address common and some uncommon points crucial for understanding this method.

Experimental research is one kind of intellectual activity. A good way to approach experiments, either those one plans to conduct or those conducted by others, is to ask what they contribute to knowledge. What do we know as a result of an experiment or what do we hope to learn from a contemplated experiment? As will be clear in several of the chapters, the central issue in experimental research, as well as in other kinds of research, is how the research can contribute to knowledge of social processes and social structures.

We begin with some terminology on research design. All research is about how things are related. In describing a research design, often it is convenient to distinguish independent and dependent variables, and we use those terms in describing what we mean by experiments. A variable is anything that takes on different values, something that can be measured. In research, variables are tied to measurement operations; for instance, the variable socioeconomic status or SES may be measured by a person’s or a family’s income in dollars. Beginning students must become accustomed to the idea that so-called independent variables are usually controlled in some way by investigators, whereas dependent variables are left free to vary; they are controlled only by nature. Thus, a study might control educational level statistically by partitioning a sample of individuals into those whose education ended after eighth grade, after some high school, with a high school diploma, etc., to determine education’s effect on SES. Education is the independent variable in this study, and SES is the dependent variable. The design described uses survey methods.

As we use the term, a study is an experiment only when a particular ordering occurs: when an investigator controls the level of an independent variable(s) before measuring the level of a dependent variable(s). In the preceding hypothetical survey design, presumably data were collected all at the same time on the respondent’s schooling and income. The independent variable was partitioned afterwards, and the interest was in how SES divided after edu...

Table of contents

• Cover image
• Title page
• Table of Contents
• Dedication
• Copyright
• Preface
• Contributors
• Part I: Designing and Conducting Experiments
• Part II: Experiments across the Social Sciences
• Part III: Applied Research and Proposals
• Index