Remember the time you asked one of your parents to buy a particular pair of expensive running shoes, let you go to a party, or do something just because âall the kidsâ were? Well, the astute parent would question this and wonder if it were really true that all your friends were actually involved. âWhy not collect some data first and then report back to me what percentage of your friends own those clothes or are going to that concert?â Thatâs what parents trained to do survey research might say!
Too often we make conclusions about entire groups of people on the basis of observations of only those we know or see around us. Or to put it more scientifically: Too often we generalize about an entire population on the basis of a nonsystematic method of collecting data from a biased sample. This is one of the major differences between everyday experience, pseudoscience, and scientific thinking. What follows is an argument for doing things more systematically and scientifically than usual, and a justification for doing quantitative survey research as one of many approaches to understanding how and why humans think and act the way we do.
Everyday Thinking
How we see the world around us is shaped by a variety of forces that include the books we read, the television and movies we see, the cultureâs rules and guidelines for good behavior we repeatedly hear, and the teachings from the religious organizations and schools we attend. It is also highly influenced by our friendsâthe peer groups we spend a good deal of time in and from whom we learn. But so many of these social forces are linked to each other in ways that tend to reinforce our already held values and beliefs. Conclusions based on networks of friends and family members are hardly useful, then, in trying to understand how most other people think or behave.
The trick of socialization, to paraphrase writer Carlos Castaneda, is to convince us that the way we see the world is the only reality, one supported by social consensus. By experiencing more diverse cultures and meeting a wider range of people, we come to understand that to make conclusions based simply on the way our friends and family live, what they believe in, or how we were socialized, is limiting. Too often those with whom we associate are similar to ourselves in values and beliefs; we end up selectively sampling like-minded people and erroneously concluding that âeveryoneâ experiences reality in the same way we do.
Although we seem to manage fairly well on a daily basis, our everyday experiences are often based on methods that can lead to problematic decisions with outcomes that can seriously affect our lives. Imagine if some friends told us that they were able to stop flu symptoms and prevent a recurrence of a cold simply by eating only the white filling in a chocolate cookie every day for a month. Would we rush out to buy bags of these wonderful cookies and use them to ward off the flu instead of getting a vaccine? It certainly would be more fun (and fattening) but not likely to keep us from getting the flu. What if they told us to take vitamin C or echinacea? Would we run to the store to get some, or would we critically inquire about the research that tests this general statement?
Every day we make conclusions and act on them with similarly limited information. Recall how sometimes you get a message from someone you suspected would be getting in touch: âI had a feeling you were going to text me.â Never mind that you donât remember the number of times you did not guess accurately or that you normally contact that person around the same time every day.
Or recall when you ran to the nearest movie theater to catch a film your friends exclaimed to be the best ever and some critic called âthe best film of the year,â even though it was only March. Although it would be impossible and crazy to survey a large number of people every time we need to make a decision to do something, be aware of how the process of making a choice is often not much more systematic than believing the person with the secret cure for the common cold. Yet we make such decisions because we know from experience (âempirical dataâ) that this particular friendâs recommendations have almost always been reliable.
What we do in everyday life is typically the result of some less scientific thinking than the procedures we would expect policy makers, neurosurgeons, or airline pilots to follow when they are in control of our well-being. In other words, what we need to consider is how ordinary thinking differs from the systematic methods needed to understand complex social behavior and attitudes. By doing so, we begin our research journey on the correct foot.
The characteristics of ordinary, everyday thinking and inquiry include
- Biased questions
- Limited sampling
- Selective attention, perception, and retention
- Pseudoscience.
Biased Questions
Did anyone ever ask, âDo you really want to go to that Justin Bieber concert?â and say it with an air of disdain? It would be difficult to answer, âYes, I really do want to hear him singâ since the questioner was implying that the performer was so bad no one in his or her right mind could really want to go. And remember the time someone wanted to know the reasons why everyone was so dissatisfied with the workplace cafeteria, thereby assuming that âeveryoneâ was unhappy and all we needed to do now was find those reasons? Consider the research question usually worded this way: âWhy is it that I study more than the other students yet they get better grades?â and the implications already formed by the phrasing. Rhetorical questions are posed to make a point rather than to seek an answer and are certainly not suitable for scientific data gathering.
In each of these three cases, everyday nonscientific thinking leads us to ask questions that are already biased or slanted in a particular direction. The first is already providing the answer in the way it is worded. It does not allow for a full range of possible answers, so the only conclusion would be that everyone who was asked the question in this way in our survey does not want to hear Bieber sing. Weâd be making an accurate statement of what was uncovered, but the answers are only as good as the way the question was framed and applicable just to the limited sample. The tendency, of course, is to generalize to all people, that is, to make conclusions about people not surveyed using the information obtained from those that were.
Similarly, the second question proceeds from a selective assumption that may not be accurate, so the only answers we get from a survey of employeesâ experiences are negatively biased ones about the organization. How many times do we really ask, âWhy is this workplace such a happy one?â with the same enthusiasm and inquisitiveness as we asked why people are dissatisfied with work? Or, more accurately, how often do we ask what they think about their workplace, without qualifying it as good or awful when we inquire?
With the third question about study habits, we are talking ourselves into believing something that may not be true, but asking it in this form just the same. Rarely do we, in everyday talk or thought, put it the following way: âI wonder if there is a relationship between the amount of time people study and their grades.â We often begin with a particular viewpoint and proceed to ask questions from that selective position, sometimes not even aware that we are doing so. Thus, the answers we get tend to verify what we already believe (sometimes referred to as âconfirmation biasâ) and give us a false sense of having found some objective and honest answers (see Box 1.1). We now have the reasons why our limited sample of employees hate the cafeteria food and falsely generalize that everyone hates the food because we never found out how many liked it in the first place.
Box 1.1 Everyday Biases
A great source for studying how people selectively see the world, make use of limited samples, generalize to people not studied, and ask questions in biased ways is to read the Letters to the Editor page in newspapers or the Comments section of online blogs and responses to controversial Facebook posts. Soon after a major shooting occurred in a suburban high school, a series of comments appeared in an online story, each with its own explanation for what happened and each likely representing the writerâs own personal values and biases. These opinions included the following possible reasons:
- Life in white monotonous suburbia and the alienation that results
- Republican congressmen who refuse to pass legislation controlling guns
- Parents who neglect their kidsâ depression and isolation
- Schools that allow hate-filled speech and all sorts of taunting and bullying of kids who are different
- Legal abortion and the theory of evolution, both of which lead to a devaluation of life.
Take a look at your local paperâs letters to the editor or the comments for an online column or Facebook post and see how many examples you can find of everyday, nonscientific reasoning.
Our daily interactions with friends, family members, and the people we encounter on a routine basis rarely require anything more scientific. Ordinary discourse contains many such poorly worded questions and biased assumptions, and the world manages to keep going. However, imagine the consequences of asking questions in these ways when trying to understand more complex and important behaviors and opinions or when attempting to develop public policies that can seriously affect peopleâs lives. No one should think that an educational institution, for example, can create rules to require a certain amount of study hours per week simply based on the findings of a survey that asks only why those who study less get better grades. Should the Human Resources office modify an entire organization on the basis of the results of a survey focused solely on the dissatisfaction of the employees who took the time to fill it out?
Everyday thinking, in other words, typically employs biased phrases and other nonscientific styles of framing questions when making sense of the world we live in. The results of these queries may help us decide which movie to watch, but they are not useful when it comes to making choices that affect social policies or arriving at conclusions fairly and consistently. How we can improve the way we ask questions and minimize the biases that derive from faulty wording is discussed in more detail in Chapter 4.
Limited Sampling
Most of the time, we make sense of reality by reflecting on the experiences weâve had. It is not too difficult to figure out how our friends feel about various rules, political leaders, music, and television programs. We are generally good at assessing the climate of opinion about controversial topics among our peers and those whom we encounter regularly in our living spaces. Unfortunately, there is also the tendency to take these limited experiences and then assign them to larger groups of people.
âEveryone I know hates the food in the cafeteriaââso it must be bad. âI talked with others in the class who have good grades and theyâre not reading the assignmentsââso it doesnât matter how much one studies. âNo one I know liked that movieââso it must be failing at the box office. While it may sound reasonable to make such concluding statements, the problem, of course, is that we have simply taken the opinions or measured the behaviors of those we already know and then made the assumption that they are somehow representative of âallâ people: âBut, Mom, everyone is going to be driving to school this year.â But if they are our friends, they are likely to be people who share our tastes and values.
Confirmation bias is this tendency to seek information and sources that support our already held opinions. We tend to avoid sampling websites, opinion articles, and news sources that contradict or challenge our values and views.
Selective Perception
People forget that most of the time our lives are constrained and limited by the social spaces we inhabit. Ethnicity, social class, gender, sexual orientation, and religion are just some of the many characteristics that provide different experiences and push us int...