part 1
thinking about research
You are already thinking about research
Right at this moment you are reading the first sentence of a book on how to do research with people. You canāt be doing this by accident. We definitely remember asking our publishers to write something on the cover to say that this book is all about doing research. You chose to ignore this warning, so you must already be thinking about research ā well done!
Okay, so weāre being a bit silly here: itās the first book weāve written together and weāre a little giddy with the excitement. Bear with us, weāll have calmed down in a few paragraphs.
Actually, though, in amongst all this nonsense we are trying to make quite a serious point: you might well think that reading this book is the first step you have ever taken towards learning how to do research with people, but the truth is you have been thinking about how to study people all of your life. Pretty much from the moment you were born, you instinctively experimented with your caregivers to learn how they would react to various things you did ā how else would you have learnt what is and is not acceptable in your culture? Today, every time you ask a question like āI wonder why some people are more popular than othersā or āI wonder why people seem to get more aggressive when they drive a car,ā you are setting up questions about people which could be addressed through systematic research. Indeed, if you put your mind to it, we are sure you would get at least part-way towards answering those questions right now.
And that is the point we wish to make: we are not about to teach you a completely new skill here. Instead, we are going to make you better at doing something you already know about, at least in part. Obviously we hope to teach you a lot of new information in this book. But we will also spend time taking rules and concepts that you already know in a fairly subconscious, intuitive sort of way, and making that knowledge more explicit. When you are properly aware of the ideas you already have about how research should be carried out, you can more easily use this knowledge when you conduct research yourself. You will also be better equipped to use other peopleās research: you will be able to spot mistakes in the data you come across, to know what questions you need to ask when you hear about research in the news, and to know which studies you can trust and which you cannot trust when making decisions about your own life.
Let us prove to you that you already know quite a lot about doing research with people, even if you have never thought about it before. As it happens we are both amateur pharmacologists and have worked away in our basements to produce a new drug called Holtodol. Holtodol not only makes you a great deal healthier and improves your memory, but also makes you more attractive to whichever gender it is you like to attract. Sounds good, doesnāt it? So if we were to offer you some Holtodol, would you take it?
How about this: would you take the drug if we had tested it on two people, both 21-year-old men, for a week, and neither experienced any side-effects?
Would you take the drug if we had tested it on 10,000 people, with a mixture of men and women of various ages, for 10 years, and none of them experienced any side-effects?
We strongly suspect you would feel happier taking Holtodol in the second situation. The reason for this is that you already know, at some level, that although the finding is exactly the same ā no side-effects ā this is much more convincing when it comes from a large, mixed sample of people over a long period of time than when it comes from just two people over a few days.
Let us give you another example, which illustrates some of the issues that arise when we deal with peopleās feelings. There are two hospitals near where we live. Each claims its employees have the best working conditions in the area. Obviously they canāt both be right, so we decide to put it to the test.
We take a member of staff from each hospital. Specifically, from Hospital A we take Professor Margaret Pritt, the 56-year-old Head of Ophthalmology, and from hospital B we take David Fish, a 19-year-old car park attendant. We ask each of them how happy they are at work. David Fish says he is āreally happyā whereas Margaret Pritt says she is āfairly happyā. We therefore conclude that Hospital B treats its employees better, and suggest that Hospital A should change its working practices to be more like Hospital B.
Is this a fair conclusion? Should we really use this research to change policies throughout Hospital A? We suspect that right now you are violently shaking your head, and quite possibly wondering if you still have the receipt for this book so you can return it.
The thing is, we spoke to only one person in each hospital ā thereās every possibility we have chosen completely inappropriate people for our comparison. You can clearly see that the people we interviewed had very different jobs: one was the head of a clinical unit with decades of employment history and lots of responsibilities; her experience of work is likely to be totally different to that of a worker not long out of school. And it isnāt just the differences in their jobs that might be a problem: David and Margaret are different genders, and are at completely different stages in their lives ā middle-aged workers like Margaret could well be juggling their job with family responsibilities, which might affect their enjoyment of work. On the other hand, a younger worker like David might be affected by having less money and job security.
And all this is just the tip of the iceberg. There are a huge number of other problems with the study we have just described. For example, simply asking people āhow happyā they are is too vague and subjective: it would have been better to ask several questions, each using more objective terms, such as āHow often do you find you simply cannot face going to work?ā. And is āhappinessā even the right thing to be measuring in a study of working conditions anyway?
So we are confident that when you read our description of the hospital study, you immediately felt there was something wrong with it, even if you could not put your finger on what exactly it was. This shows that, even when addressing a fairly complex question, you already have some knowledge of what is good and bad research. In the rest of this book we will be developing this knowledge and adding to it. What we will not be doing is teaching you something completely new.
What we are going to cover
The rest of Part 1 is going to lay some foundations before we get to the serious business of guiding you through how to plan and conduct research projects with people. We will start out by explaining the logic of how research works in Chapter 1, then Chapter 2 will look at the question of who will take part in your studies. Chapters 3 and 4 are an introduction to what you might do with the data you collect when carrying out research: Chapter 3 looks at how you can describe data and Chapter 4 introduces the tests that are used to see whether what you see with the people you test can be generalized to other people.
chapter 1
the problem with people: variation and hypothesis testing
From time to time, everybody who carries out research with people for a living wishes they worked with something a little simpler and more predictable. Sleep-deprived exploding panthers, for example. The problem is that, whilst people can be fascinating and amazing things, they are also maddeningly inconsistent. A physicist studying pieces of copper doesnāt come in to work one day to find their blocks of metal are suddenly behaving wildly because it is windy outside (this happens all the time when studying small children). A biologist working with sheep doesnāt have to worry about them having hangovers if they are tested early on Monday morning (as happens with some students). An engineer hanging weights on a model bridge to see how it flexes doesnāt need to make allowances for the bridge getting better at its task with practice (as you would).
The point is that when we are dealing with people, with their memories and social lives and moods and beliefs, we have to be aware that every observation we make is the end product of a million different influences. Go and ask somebody what is their favourite colour. Done that? Okay, now think about the answer you received. Of course, the person you spoke to might simply have told you their favourite colour. But what if you then learnt they had the colour vision deficit known as protanopia, and so were not be able to tell red and green apart? This gives their answer a rather different complexion, doesnāt it? And anyway, how can you be sure the answer you received is really that personās favourite colour, and not just a colour other people have told them they should like (as when five-year-old girls say āpinkā)? Perhaps the personās favourite colour was orange but they told you āblackā because they wanted to appear moody and interesting. They might have said the colour of your shirt, because they thought your question was stupid and just wanted to get rid of you. If they know you, they might have said your favourite colour because they wanted to reinforce your friendship. They might have said āmauveā, simply because saying the word āmauveā makes you smile (try it). Or they might have been tediously predictable and said āblueā.
The point is, all you have from that person is the name of a colour. Is it their favourite colour, as you requested? It might be, but there are so many possible influences on their response that you just canāt know that with any certainty. This means you canāt reach safe conclusions.
Ian: | Iāve just spoken to Anna and I now know her favourite colour is sea-green. |
Nigel: | Weird, because Iāve just spoken to Anna and I very much think youāll find her favourite colour is lilac. |
Ian: | Sea-green. |
Nigel: | Lilac! |
Ian: | SEA-GREEN! |
And so on, until the fisticuffs begin. When this sort of thing happens, how can we ever find the correct answer?
One approach might be to anticipate all the various influences that could affect a personās behaviour and deal with them in advance. So you could try to discover somebodyās real favourite colour by asking them a question like: āDisregarding your recent experiences, and what you think I might be expecting you to say, and what your mother told you was her favourite colour, and what you feel Society thinks you ought to say, and any colour you might be looking at right now (unless it actually is your favourite colour, in which case donāt disregard it), and the colour of your bedroom walls, and the colours of your toys when you were a child, and the magnificent mouthfeel of the word āmauveā1 ā¦ disregarding all that, what is your favourite colour?ā You could try that, but youād still fail to get a reliable answer because you didnāt realize that the person you are speaking to is a member of the Himba tribe from Namibia and so uses the same word to mean both ādark redā and ādark blueā.2
This is all by way of saying that tremendous amounts of variation and complexity are the norm when studying people. People differ from one another, and individuals differ over time, and so you cannot just measure things simply as you would with blocks of copper or model bridges. Nor can you hope to deal with this issue by anticipating all the factors which might affect peopleās responses, as we just attempted to, because, no matter how much you try to anticipate the influences on peopleās behaviour, people will always find a way to surprise you. Luckily, you do not need to do this, as procedures and techniques have evolved over the years to cope with people and their tricksy ways. These procedures and techniques...