Psychology in Crisis

Brian Hughes

• Editorial: Bloomsbury Academic
• Año: 2018
• ISBN:
  9781350312524
• Edición: 1
• Categoría: Psychologie
• Categoría: Geschichte & Theorie in der Psychologie

Throughout the history of psychology, attempting to objectively measure the highly dynamic phenomenon of human behaviour has given rise to an underappreciated margin of error. Today, as the discipline experiences increasing difficulty in reproducing the results of its own studies, such error not only threatens to undermine psychology's credibility but also leaves an indelible question: Is psychology actually a field of irreproducible science? In this thought-provoking new book, author Brian Hughes seeks to answer this very question. In his incisive examination of the various pitfalls that determine 'good' or 'bad' psychological science – from poor use of statistics to systematic exaggeration of findings – Hughes shows readers how to critique psychology research, enhance its validity and reliability, and understand the strengths and weaknesses of the way psychology research is produced, published, and promulgated in the 21st century. This book is essential reading for students wanting to understand how to better scrutinise psychological research methods and results, as well as practitioners and those concerned with the replication debate.

CHAPTER1

‘The Same Again, But Different’: Psychology’s Replication Crisis

Hurricane in a teacup

Stop the presses! We bring you some breaking news:

Scientists Say Female Hurricanes are DEADLIER than Male Hurricanes!

Could there be a more twenty-first century headline? Its synergy of human-interest soundbites – gender politics, life and death, even the weather– might just represent the epitome of clickbait.

And the news conveyed is so seductively counterintuitive. Or is it intuitive? I suppose it depends on your prejudices. So let us contemplate its main point: according to research, if a hurricane has a woman’s name, it becomes more deadly. It literally kills more people. Hurricane Eve is a greater threat than Hurricane Steve.

But it’s a hurricane, you might say. How could this be true? How could the name human beings choose for it cause a weather system to wreak a different calibre of havoc? The idea seems, well, implausible.

Nonetheless, the smoke of this news, which made global headlines in 2014, came from the fire of science. Professional researchers reported the claim to be true. More specifically, psychology researchers reported it. And psychology, lest we forget, is a science. Psychologists use scientific methods. Psychologists are committed to a search for scientific truth. They wouldn’t seek to publish such a finding unless it was reliable, would they?

Well – spoiler alert – as we will go on to see throughout this book, when it comes to psychology, it is never quite safe to assume anything.

The hurricanes-are-deadlier-than-himmicanes study originally app­eared in the prestigious American academic journal Proceedings of the National Academy of Sciences(Jung et al., 2014). Its authors presented datashowing that more people are killed by female hurricanes than by male ones. A lot more. According to them:

changing a severe hurricane’s name from Charley … to Eloise … could nearly triple its death toll. (p. 8783)

To be fair, they did not claim that the wind blows differently when it has a girly name. They argued that humans react differently depending on the gender they perceive a hurricane to be. Humans take female hurricanes less seriously than male ones. It isn’t a gender effect so much as it is a gender bias. So addled are people by sexist prejudice, they even stereotype the weather. In the words of one CNN columnist (Cupp, 2014):

Girl hurricanes get no respect.

The news spread widely through traditional and social media. And why not? The finding was not just intriguing, it appeared in a highly reputable journal, maybe even one of the most reputable journals in all of science.

But there was, of course, a problem. The finding – while earnestly presented, widely reported, and credulously considered – was not in fact reliable. The claim being made was limited in one crucial respect. It was simply not true.

It did not take long for critics to highlight the many restrictive definitions the authors had used in their analyses (Smith, 2016). They had focused on hurricanes, yes, but this meant they had opted to ignore tropical storms, which are similar and just as deadly. They considered Atlantic hurricanes but not Pacific ones, without an obvious reason for geo-restriction. They confined themselves to hurricanes that made landfall, discounting those that remained offshore, even though offshore hurricanes are regularly just as lethal. (In 2009 the offshore Hurricane Bill became so famous that ten thousand sightseers gathered along the coast of Maine in order to look at it, despite coastguard warnings telling them to stay away. A single wave washed twenty of them into the ocean. Clearly, people didn’t give Hurricane Bill much respect, even though it had a male name.)

Most strikingly of all, the authors only considered fatalities within the United States, even though many of the hurricanes they studied affectedneighbouring nations too. In 1980 Hurricane Allen killed 269 people in several Caribbean countries, but only two people in the USA. The authors duly recorded Hurricane Allen as one of the safest in their dataset. After all, it killed only two Americans.

The authors had made a series of peculiar choices. They could have included Hurricane Bill, but they opted not to. They could have considered all the fatalities resulting from Hurricane Allen, but they didn’t. They could have looked at tropical storms, and events in the Pacific, but they decided to exclude them. Instead they shaved and slimmed their dataset in a way that produced a profound effect on their finding. Their choices made female hurricanes seem more lethal and male ones appear safer.

Statisticians refer to this type of thing as the ‘torturing’ of a dataset. Such mistreatment of data is as nasty as it sounds. Moreover, as is generally well known nowadays, there are two significant problems with ­torture. Firstly, it is unethical: it is not morally correct to do it. And secondly, it is ineffective: what gets blurted out under duress might sound good, but, more often than not, it is just not true.

Many researchers have attempted to reproduce the finding that female hurricanes are deadlier than male ones. None have been able to do so. In fact, every single subsequent study has shown that male hurricanes are just as deadly as their female equivalents (Smith, 2016). It turns out that people don’t attribute gender stereotypes to the weather after all. They treat girl hurricanes with just as much respect as any other.

Despite making global headlines and remaining on the record as a result of peer-reviewed published research, the finding that female hurricanes are deadlier than male ones is demonstrably specious. Despite many attempts, the result cannot be replicated.

‘Fundamentally squishy psychobabble’

It never looks good for psychology when its research findings turn out to be false. It stains the field’s image. This is unfortunate, because psychologists are normally quite image-conscious. Professional bodies devote enormous energy to outreach and media work. They develop detailedstrategies to optimize psychology’s ‘footprint’ and ‘impact’. They produce packed portfolios of press releases, breathlessly pitching even very vague research summaries as potential news stories. University psychology departments often operate like press offices, feeding stories directly to news outlets, sometimes on a weekly basis. The total collective effort expended in enhancing the field’s media profile is little short of monumental.

Presumably, these endeavours are intended to avoid headlines like the following:

Study Reveals That a Lot of Psychology Research Really Is Just ‘Psycho-babble’.

Psychobabble is certainly not a term that psychologists would choose to describe their work. But it was the word chosen for them by the UK’s Independent newspaper in August 2015, when covering a just-­published critique of the quality of psychology research (Connor, 2015). The Independent succinctly summarized the critique’s main finding:

Psychology has long been the butt of jokes about its deep insight into the human mind … and now a study has revealed that much of its published research really is psycho-babble.

For ‘psychobabble’ to be mentioned once in a national newspaper is bad enough. For the term to be used repeatedly is certainly not in any psychologist’s media strategy.

This psychobabble conclusion was widely reported. The Guardian described the finding as ‘a bleak verdict on the validity of psychology experiment results’ (Sample, 2015). The New York Times felt it ‘confirmed the worst fears of scientists’ regarding the state of psychology (Carey, 2015). According to the Washington Post, it confirmed that much of what is published in psychology journals is ‘fundamentally squishy’ (Achenbach, 2015). Only a modicum of French-language proficiency is required to interpret the verdict of Le Monde: ‘La psychologie est-elle en crise?’ (Larousserie, 2016). By not mentioning psychology in its headline the Huffington Post(Reid, 2016) managed to hide the field’s blushes (but only just):

Scientific Study Proves Scientific Studies Can’t Prove Anything.

Psychologists had been whispering about such a crisis for decades. But now the idea that psychology research is not replicable was about to go viral.

The excitement was stimulated by work produced by the Open Science Collaboration, an international group of over two hundred researchers led by University of Virginia psychology professor Brian Nosek. The group published an extensive and damning report in the elite academic journal Science(Open Science Collaboration, 2015). In their programme, they attempted to re-conduct one hundred published psychology experiments. Over 60 per cent of the attempted replications ‘failed’.

In other words, the results of the re-conducted studies bore little or no similarity to those of the originals they were based on. In psychology, as in any other science, this is a real problem. When the same study done twice produces two different outcomes, then one thing is for sure: both cannot be right. When 60 per cent of studies are like this, the accuracy of the entire field is called into question. As a column in Nature(Baker, 2015), the world’s leading scientific journal, bluntly put the point:

Don’t trust everything you read in the psychology literature. In fact, two thirds of it should probably be distrusted.

The problem of course is this: how do you know that the study you are interested in – or indeed any particular psychology research study that you happen to encounter – is one of the trustworthy few and not one of the untrustable majority?

It is little wonder that journalists were interested. Psychology research affects our lives in so many ways. It is consulted in the design of everything from road safety campaigns, to anti-litter initiatives, to health ­promotion programmes, to educational curricula, to psychometric tests, to household products, to advertising. Politicians cite psychology research when debating policies about childcare, or poverty, or social exclusion, or environmental protection. Even disputes about major constitutional issues, such as whether to extend access to abortion or to lower the voting age, will be influenced by what psychology research has revealed about these issues.

Psychology relates to so much of daily life that its research has universal resonance. If it is actually true that 60 per cent of psychologicalscience is not replicable – if psychology research is indeed ‘fundamentally squishy psychobabble’ – then of course the world’s media will report this. They have a moral obligation to do so.

According to a famous quotation often attributed to Albert Einstein, insanity can be defined as doing the same thing over and over again and expecting different results. In psychology, such a scenario is the standard way of things. When psychologists conduct the same study over and over again, it would be reasonable to expect different results each time, given the historical pattern to date. In Einsteinian terms, it is the confident expectation that a replication might succeed that should rightly be considered madness.

The recurrence of non-replication

It would be wrong to think that psychology’s replication crisis is new news. In fact, it is very much old news. The Open Science Collaboration itself was the culmination of decades of concern, the outpouring of extreme frustration among an increasingly disaffected subgroup of the psychology research community.

Psychology is often called the ‘science of behaviour’. These days, we take behaviour to include people’s actions, thoughts, and feelings. Unlike other sciences, psychology is not just a method of inquiry, it is also a field of human activity that itself can be inquired into. Not only can psychologists examine how people in general act, think, and feel, they can also examine how they themselves act, think, and feel. Psychology is itself a behaviour. Therefore, psychology can – and possibly should – study itself.

In historical terms, the use of logic and evidence to uncover knowledge is a relatively modern thing. Before science, and for many centuries, ­people had little choice but to draw knowledge from authority figures, superstition, or popular consensus. Any claim, no matter how weak, stood a good chance of being believed. People thought that diseases were transmitted through foul-smelling vapours, that all living beings were animated by a spirit-like force, and that California was an island. Science changed all this. It elevated the status of logic and evidence in public discourse. It attached value to validated accuracy. Intellectually, more than technologically, science altered the course of humanity.

Throughout the world, a scientific culture developed during the sixteenth, seven...

Índice

• Cover
• Half Title
• Series Page
• Title Page
• Copyright
• Contents
• Acknowledgements
• 1 ‘The Same Again, But Different’: Psychology’s Replication Crisis
• 2 ‘Black Is White’: Psychology’s Paradigmatic Crisis
• 3 ‘Never Mind The Quality, Feel The Width’: Psychology’s Measurement Crisis
• 4 ‘That Which Can Be Measured’: Psychology’s Statistical Crisis
• 5 ‘We Are The World’: Psychology’s Sampling Crisis
• 6 ‘Fitter, Happier, More Productive …’: Psychology’s Exaggeration Crisis
• 7 From Crisis To Confidence: Dealing With Psychology’s Self-inflicted Crises
• References
• Index