Understanding and Applying Research Design

Martin Lee Abbott, Jennifer McKinney

• Publisher: Wiley
• Year: 2013
• ISBN:
  9781118605295
• Edition: 1
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

A fresh approach to bridging research design with statistical analysis

While good social science requires both research design and statistical analysis, most books treat these two areas separately. Understanding and Applying Research Design introduces an accessible approach to integrating design and statistics, focusing on the processes of posing, testing, and interpreting research questions in the social sciences.

The authors analyze real-world data using SPSS software, guiding readers on the overall process of science, focusing on premises, procedures, and designs of social scientific research. Three clearly organized sections move seamlessly from theoretical topics to statistical techniques at the heart of research procedures, and finally, to practical application of research design:

• Premises of Research introduces the research process and the capabilities of SPSS, with coverage of ethics, Empirical Generalization, and Chi Square and Contingency Table Analysis
• Procedures of Research explores key quantitative methods in research design including measurement, correlation, regression, and causation
• Designs of Research outlines various design frameworks, with discussion of survey research, aggregate research, and experiments

Throughout the book, SPSS software is used to showcase the discussed techniques, and detailed appendices provide guidance on key statistical procedures and tips for data management. Numerous exercises allow readers to test their comprehension of the presented material, and a related website features additional data sets and SPSS code.

Understanding and Applying Research Design is an excellent book for social sciences and education courses on research methods at the upper-undergraduate level. The book is also an insightful reference for professionals who would like to learn how to pose, test, and interpret research questions with confidence.

PART I

WHEEL OF SCIENCE: PREMISES OF RESEARCH

1

“DUH” SCIENCE VERSUS “HUH” SCIENCE

HOW DO WE KNOW WHAT WE KNOW?

When we go through the education process, we each take several categories of classes, especially if we know we’re headed to college. Often one of these categories is “science” and includes classes in biology, chemistry, or physics. Because of this we come to think of science as particular substantive areas rather than as a particular process. The process of science allows us to follow systematic steps to better understand the world around us. Whether using amino acids, elements along the periodic chart, sound waves, or people’s attitudes, following the process of science allows us to see patterns in our materials. Granted, it’s often harder to think of people as “materials” than it is to think of saltwater solutions as materials. Regardless of what we are looking for, following the scientific process allows us to gauge what is going on in the world.

The process of social science differs from other sciences only in that the social sciences use people to find patterns. While most of us think of people as individuals, each individual lives in a particular social context that has a surprising amount of order to it. For example, Americans drive on the right side of the road; Britons drive on the left. Even though both countries are made up of individuals, they each tend to transfer their cultural order to walking on the same side of the sidewalk. Even though each individual may walk in a unique way (perhaps like Monty Python’s “lumberjack walk”), each tends to gravitate toward the right or left side of a sidewalk depending on country—or cultural order—of origin.

Keeping with a roadway example, have you ever thought about the only thing keeping one vehicle from hitting another in a head-on collision is a measly 6 inches of yellow paint? Think about the 6 inches of white paint that keeps cars traveling in your direction from driving into you. If you consider a large urban area with millions of people trying to travel by car into and out of the area every day, isn’t it amazing how few car accidents there are? In Seattle (even with our perpetually wet weather), there are roughly four million people trying to get into and out of the metropolitan area each weekday. But there are less than a hundred vehicular accidents in a given 24-hour period, illustrating just how effective 6 inches of paint can be in regulating the behavior of millions of people. That people and social patterns have such a high degree of order allows us to study just where these patterns originate and predict when they are going to show up.

Knowing there are social rules and boundaries in place that create a high degree of social order, the task for the social scientist is to measure people’s attitudes, behaviors, and experiences to find common patterns. The question becomes, however, why should you need social science when you live in the same world or social context and experience these things for yourself? Why rely on social science to generalize to a population or group of people or things? How do you know what social science says is true? How do you know what is good information? The only way to truly know about the social patterns around us is to understand the process of science.

Say, for example, your professor distributes a class exercise asking you to evaluate some research finding. You are first asked if the finding is surprising or not, and then you are asked to write down a reason or two why you believe that finding is or is not true. Let’s say that you are given the finding, “Social scientists have found that opposites attract.” Is this finding surprising? How do you evaluate this statement? What evidence do you have that opposites attract? Go ahead and think of or jot down why you believe that opposites attract.

What if your professor is being a bit cagey and secretly handed out two contradictory research findings? Whereas you received “opposites attract,” the other half of the class received the reverse finding that “Birds of a feather flock together.” As the class comes together to discuss the research finding, an interesting thing will happen. When asked how many in the class found “this” finding to be not surprising, most of the class will raise their hands to show how unsurprised they were. That a majority of the class reports their research result is true and not surprising is interesting considering the class had two very different findings. This predicament illustrates the hindsight bias. In hindsight, research results seem like common sense; we take for granted that research findings must be true—after they are given.

As you thought about the finding you were given, you probably searched your experience for one case (person) where “opposites attract” was true. Generally when we hear about research findings after the fact, we think of at least one case of confirming evidence. This means we look to our own experience and try to find one person or situation that fits the finding given. In this case, you probably thought of at least one friend or acquaintance who was in a relationship where opposites attract. Your classmates with the contradictory finding were doing the same thing, trying to find an example of someone they knew in a relationship where birds of a feather flock together. But trying to explain research findings using our own experiences and already being biased by what the result appears to be hurts our ability to see the world as a whole. If you thought of one person who served as an example of each finding, that’s two people. Can two (or even 10 people you may have thought of) represent the whole social spectrum? Even in just an American context, there are well over 310 million people to consider. Do we really want to base our understanding of which adage is more true simply by finding two examples that confirm the finding and conform to our limited experience? It’s highly unlikely that diametrically opposed research findings like opposites attract or birds of a feather flock together happen exactly randomly and at the same rate in a given social context. So how do we know which is more descriptive of everyday attraction?

Social scientists recognize that, while everyone’s personal experience of the world is unique, there are social patterns that transcend our own experiences. Social scientists look for both the confirming evidence and the disconfirming evidence—examples where a finding would not be true—to give us direction as to how to generalize to the whole population which would be true. Can we find instances where B does not follow A? If so, that leads us into asking new questions and testing data to give us a more comprehensive picture and stop us from making a hasty conclusion based on very little evidence.

Every day we evaluate information based on our own experience. This is usually helpful for us. What if you and your friends are trying to decide which movie to see on a weekend? Do you choose whichever blockbuster is showing? Do you go to the film that may not be in the theater next week? Do you choose a movie based on what genre you tend to prefer? Do you pick a movie based on a friend’s recommendation? Do you choose the film based on the critical reviews? Do you choose a film based on your schedule—whichever is showing at the closest theater at a particular time? You probably use a combination of these methods to figure out which is the best movie to see at any given time. Have you found that even when using your own judgment (based on your preferences and friends’ reviews) that the movie was a dud? In effect, social scientists are trying to tease out all of the ways we can think about a particular topic. That helps us to test topics to try to find consistent answers to research questions.

Science is needed because we do not experience the world randomly. How we experience and view the world is highly influenced by our social location—where we fit into the social order (our social class perspective, our gender perspective, our educational perspective, our political perspective, our religious perspective, etc.). Two people viewing the exact same event could interpret it very differently, depending on their personal context (or biases). For example, bringing a homeless encampment to a local college campus could elicit a hearty “well done” from students and faculty who want to address the issues of homelessness and poverty. At the same time, parents and local residents may protest bringing a group of homeless people to stay on campus as dangerous—for their personal safety and the safety of their property. The same event is viewed very differently by people living within the same neighborhood because they have different social locations (students, faculty, parents, and homeowners have different interests and expectations of events). Wouldn’t it be helpful to have some social science research that can explain and predict what really happens when a homeless encampment is brought to campus, as well as why people from different social locations respond differently to the same event?

Common Sense versus Science

Because we view the world from particular social locations, or biases, we need science to provide a baseline; what effects does one thing have on another, regardless of your perspective? Like trying to explain why opposites attract or birds of a feather flock together, social scientists are often accused of pointing out what is only common sense or what everyone already knows to be true. Of course, the hindsight bias hurts our ability to think novelly or clearly about particular relationships or facts, and it leads many to conclude that social science is just “duh” science—senseless science that points out the obvious.

“DUH” SCIENCE

Eryn Brown (2011) from the Los Angeles Times writes about this seemingly pointless research, enumerating studies that seem silly at best, wasteful at worst. For example, she writes of studies confirming that nose-picking is common among teens, or that college drinking is as bad as researchers believe, or that making exercise more fun may improve the fitness of teens, or that driving ability is compromised with people who have Alzheimer’s disease. “Well, duh, you might think—and you wouldn’t be the first,” she writes. The perception that social science simply tests the obvious is widespread, and yet there is more to “duh” science than meets the eye.

Many studies have to test the so-called obvious because until there are widely established links between behaviors or attitudes and some effect, we simply cannot be sure that real links between them exist. Even when clear and reliable links are found, it may take oodles of evidence to convince others that the links are real—often because people don’t understand the nature of science or they dismiss commonsense findings as “duh” science. Look at how many studies had to be done linking smoking to various cancers and lung disease before people began to believe these results were real (Brown 2011). Because of research we now understand the link between smoking and cancer, but we didn’t at first (and of course, with hindsight bias it seems silly to think there isn’t a link between smoking and a variety of cancers).

---

An Example of “Duh” Science: The 2011 Ig Nobel Prize Winners¹

Chemistry Prize: For determining the ideal density of airborne wasabi (pungent horseradish) to awaken sleeping people in case of a fire or other emergency, and for applying this knowledge to invent the wasabi alarm. Makoto Imai, Naoki Urushihata, Hideki Tanemura, Yukinobu Tajima, Hideaki Goto, Koichiro Mizoguchi, and Junichi Murakami of Japan.

Reference: U.S. patent application 2010/0308995 A1. Filing date: February 5, 2009.

Medicine Prize: For demonstrating that people make better decisions about some kinds of things but worse decisions about other kinds of things when they have a strong urge to urinate. Mirjam Tuk, Debra Trampe, and Luk Warlop and jointly to Matthew Lewis, Peter Snyder, and Robert Feldman, Robert Pietrzak, David Darby, and Paul Maruff.

Reference: Tuk MA, Trampe D, Warlop L. Inhibitory spillover: increased urination urgency facilitates impu...

Table of contents

• COVER
• TITLE PAGE
• COPYRIGHT PAGE
• DEDICATION
• PREFACE
• ACKNOWLEDGMENTS
• PART I: WHEEL OF SCIENCE: PREMISES OF RESEARCH
• PART II: WHEEL OF SCIENCE: PROCEDURES OF RESEARCH
• PART III: WHEEL OF SCIENCE: DESIGNS OF RESEARCH
• PART IV: STATISTICS AND DATA MANAGEMENT
• GLOSSARY
• BIBLIOGRAPHY
• INDEX