Navigating Challenges in Qualitative Educational Research

Research, Interrupted

Todd Ruecker, Vanessa Svihla, Todd Ruecker, Vanessa Svihla

• Publisher: Routledge
• Year: 2019
• ISBN:
  9780429509247
• Edition: 1
• Topic: Éducation
• Subtopic: Recherche dans l'éducation

How do education researchers navigate the qualitative research process? How do they manage and negotiate myriad decision points at which things can take an unexpected – and sometimes problematic – turn? Whilst these questions are relevant for any research process, the specific issues qualitative researchers face can have impactful repercussions, that if managed adeptly, can lead to successful and even new research opportunities.

Navigating Challenges in Qualitative Educational Research includes narratives that provide real world experiences and accounts of how researchers navigated problematic situations, as well as their considerations in doing so. These contributions give students and researchers a chance to understand the possibilities of research challenges and better prepare for these eventualities and how to deal with them.

Providing educative windows into the challenges and missteps even seasoned researchers face along the way, this book is an invaluable resource for graduate students and early career qualitative researchers, particularly those who are interested in education.

1

INTRODUCTION: SANITIZED RESEARCH DESCRIPTIONS AND MESSY REALITIES

A story of contrasts

Todd Ruecker and Vanessa Svihla

Introduction

Read the literature, develop research questions or hypotheses, design a study to answer them, collect data, analyze data, write up the results. The research process discussed in many articles, methodology textbooks, and methodology courses often comes across as linear and tidy. For instance, Johnson & Christensen (2016) said little about the nuances and challenges of participant recruitment and instead discussed the importance of sampling in a way that overlooks the agency and humanity of research participants: “Once these inclusion boundaries are set, the researcher knows whom he or she wishes to study and can then attempt to locate and observe the sample” (p. 273). They urged researchers to pick a sample that can be used to answer research questions while “meeting cost and other constraints,” noting that “Trade-offs will always be present” (p. 273). To be fair, broad survey textbooks like Johnson & Christensen’s (2016) have to cover a lot of material in a short amount of time. Methods books focused specifically on qualitative or ethnographic research tend to cover issues such as rapport (Marshall & Rossman, 2016) and how open and honest one can be with participants (Hammersley & Atkinson, 2007). Yet, even with this advice, understanding what this looks and feels like in practice can be unclear. Paris & Winn (2013) drew attention to researcher emotionality, writing that methodology courses “ignore the obvious; some of the dehumanizing processes, conditions, and experiences that our participants/students/friends have encountered will remind us of our own lived experiences. No one tells emerging scholars that, yes, sometimes ‘we cry’” (p. 1). Unfortunately, broad methods courses may not always include books and topics like these and early researchers may feel deficient when they fail to meet the neat plans described in their textbooks and the articles they read.

As a number of scholars have noted (e.g., Cook, 2009; Lerum, 2001; Mellor, 2001; Thomas, 1998), challenges and disruptions underlying the research process tend to be left out of depictions of the research process for various reasons: our fields’ preoccupation with objectivity in research; space limitations in a typical journal article; and perhaps the fear of looking naive as a researcher. These issues may often be connected back to what Lerum (2001) referred to as “academic armor,” the “linguistic, physical, and ideological” moves that academics make to “protect their expert positions and jurisdictions” (p. 470). Lerum (2001) argued that academic researchers “have clearly marked their turf as the intellectual, the detached, the objective, and hence, because of the cultural privileging of these qualities, the superior realm” (p. 471). The dominance of this “academic armor” is evident in the experience that inspired this collection, which Ruecker details in Chapter 8. Here, he emotionally raises concerns about how immigrants were treated in a school where he was conducting research, resulting in him being disinvited from future visits. As he unsuccessfully tried to publish an article detailing this in two different venues, he faced a mix of enthusiasm and skepticism, ultimately resulting in rejections without a chance to revise. Reviewer comments at one venue noted appreciation for a provocative discussion on the real challenges faced by researchers while rejecting it, in part because of a sense of bias towards participants and its focus on one experience. In response, Ruecker toned down some of the comments revealing his true feelings about the situation and submitted it elsewhere. There, one reviewer was generally enthusiastic, noting that we needed to do more of this type of reflection and discussion of the challenges we face as researchers. The other reviewer strongly advised not to publish this piece in part because it would make Ruecker look emotionally and politically naive and hurt his reputation as an early career scholar—the risk of dropping his “academic armor” was deemed too great.

As is evident from our work on this collection, we share concerns that these reviewers’ comments are symptomatic of a research culture in which discussions of interruptions and researcher emotionality are suppressed—or at least not shared in ways that support growth and learning—even though our discussions with others indicate they are quite common. As early career researchers and while working with graduate students developing their research abilities, we have been concerned that the sanitized realities so common in methodology descriptions may lead emergent researchers to question themselves and their work when something does not go as neatly as planned. This led us to craft a call for proposals, which posed questions such as the following:

• What are some challenges you have faced in recruiting participants?
• Have you put yourself at risk physically, mentally, financially or otherwise, in conducting your research?
• In what ways have language or cultural differences created tensions in or otherwise interrupted your work?
• What are some of the ethical dilemmas you’ve faced while researching and how did they impact your work?

Our drive in working on this collection stemmed from a desire to hear stories of interruption at various stages of research processes, as we believed these stories needed to be heard and that both new and established researchers would learn through reading how other researchers navigated challenges.

Challenge and mess in research: what we know

By and large, the clean portrayals of methodological processes presented in published research have gone unquestioned. There have been a few exceptions to this trend in the various fields we inhabit—education, learning sciences, applied linguistics, engineering education, and writing studies—and we review some of these here. For instance, action researchers, perhaps because of their increased focus on researcher positionality, have been more open than others about some of the methodological challenges they have faced, and we found several examples in the journal Educational Action Research. We argue that positivist traditions have, in some cases covertly and in others overtly, structured expectations about the research process, and this has encouraged researchers to avoid transparency in reporting on their work. We share consequences of this, including misleading novice researchers about the research process, rendering some findings suspect, and limiting the potential of our work.

Thomas (1998) provided one early critique of the positivist tradition, in which faults in research approaches and designs are typically masked or suppressed, leading to an approach in which “fertility is sacrificed to orderliness” (p. 143). He problematized the idea that the researcher is expected to consistently make order out of chaos, in his words a modern day Midas who is expected to turn everything into theory: “In the modern circumstances of the education academic this predisposition provides a predilection to make shape and theory out of chaos. And having once constructed the shape we of course have to start being analytic and rational about it” (p. 146). Researchers in the field of writing studies—a field that has an established tradition of questioning the objectivity of traditional research modes—have openly discussed such challenges. Describing her ethnographic dissertation project, Cook (1998) explained how “ethnography can seem spectacularly out of control” (p. 107).

Some qualitative researchers have reported challenges getting study approval from institutional review boards (IRBs), as—in their regulatory role—these bodies tend to encourage researchers to coerce their studies into traditional positivist designs (Babb, Birk, & Carfagna, 2017; Patel, 2015). While all IRBs are guided by federal mandates, it is relevant to note that they make determinations based on the opinions of local reviewers and administrators; when such boards are primarily staffed by those from positivist traditions, their guidance and decisions can diminish study design. For instance, Lunsford & Lunsford’s (2008) national study of error and feedback in writing classes—a replication of an earlier study (Connors & Lunsford, 1988)—included a reflection titled “A new study of student writing: those IRB blues.” The authors discussed how the process of soliciting research approvals had changed dramatically in the intervening 20 years. In the first study, the protocol “easily gained approval from their home institution IRBs—and that approval covered all requirements” (Lunsford & Lunsford, p. 787), allowing them to study classrooms at many other institutions. In the second study, the authors’ IRB required them to seek approval from all institutions where their participants were enrolled, a process that extended what had been three months in the first study to eighteen months in the second, while also reducing the number of participating institutions. Furthermore, whereas in the first study they were able to use entire, intact classrooms, in the second they were typically restricted from using data unless students consented, resulting in a much smaller dataset. They described this process in some detail:

To our surprise, instead of exempting or expediting local approval in light of the Stanford and UCSB approvals, many officials then asked us to go through their own full review process. Thus began the tedious, the time-consuming, the mind-numbing task of filling out dozens upon dozens of IRB forms, each with slightly different emphases and questions, and then waiting, sometimes for months, for a response.

(Lunsford & Lunsford, p. 787)

What is notable about this instance is how institutional requirements, while well intentioned, dramatically reduced the number of participants in the study.

Researchers who work outside the positivist paradigm may also face rejection at the hands of journal editors and other gatekeepers. Work that does not fit into particular molds of what reviewers see as new, rigorous, and objective can be deemed unsuitable for publication and that work may never be published. We are not disputing the value of the peer review process but note that it can present additional challenges to researchers whose work does not fit in particular molds. And, as Svihla explains in Chapter 14, these challenges can be time consuming, but also discouraging to the point of abandoning an initial commitment of sharing findings with the people most positioned to act upon them. As Ruecker (2017) explained elsewhere and Svihla discusses in Chapter 14, publishing challenges can be especially acute for interdisciplinary researchers, as many in education are. For instance, Ruecker reported that applied linguistics reviewers would critique his writing style as too subjective, delaying the publication process as he strove for a more impersonal, objective, and sanitized voice.

These sanitized representations of the research process provide an impoverished view of research for novice researchers. For instance, Strauss (1995) revealed how his research led him into multiple periods of despair and self-doubt about his ability as a teacher, writing at one point in his journal: “Maybe I’m just not a very good teacher. Maybe teaching is, knowing what the children are learning and responding to that knowledge. Maybe there’s nothing else to it, and I’m not very good at it” (p. 33). Another action researcher, Cook (2009), described feelings of doubt in light of the published descriptions of research: “We saw a gap between our more convoluted practice and published models of neat research. This led to doubts as to whether we were doing ‘proper research’ or whether we were doing ‘research properly’” (p. 278). A different Cook (1998) wrote that only after completing her ethnographic dissertation did she feel comfortable talking about some of the challenges she faced, including feeling “deviant” throughout the process. These comments highlight the sense of risk that novice researchers encounter when things don’t go quite as planned.

McKinley & Rose (2017) noted that sanitized versions of research are “depriving researchers of valuable insights from which we can grow and allow novice and experienced researchers alike the chance to learn from our pitfalls, mistakes, and follies” (p. 14). Their edited volume invited applied linguistics researchers to shift their focus away from the findings of an already published study and more closely examine the methodological implications of their work. By not being more transparent about the realities of our research, we make it harder for newer researchers to learn the trade, particularly as they might struggle with feelings of self-doubt as they are unsure how to proceed when facing challenges.

Researchers who are experienced with the complexities of doing longer term, larger scale, or more critical research have also raised concerns about the limitations posed by typical final form representation. In light of her experiences with mess in research, Cook (2009) argued that it is not necessarily a sign of sloppy, poorly done research but in fact “an indicator of serious critique taking place” (p. 285). Upon recognizing that the research process as reported in journal articles is not always the whole story, some researchers react with suspicion about published findings. Writing about the “messiness” of action research, Mellor (2001) explained how he initially came to research with the impression that it was a neat, linear process involving three steps: “collect data, analyze data, write up” (p. 467). He described how he came to be “suspicious of ‘hygienic’ accounts” and that “messy method” was the norm: “if we are honest, we all work this way” (pp. 474–475). In talking with other researchers, Mellor found that, while some said their work aligned to the neat approaches described in journal articles (whether they were honest or not is another question), others admitted to messiness in their own research process. Cook (2009) argued that hiding this mess from view in published research makes “it difficult for future researchers to understand how outcomes were achieved and how they might build on those outcomes” (p. 289). In that sense, detailing researcher challenges and interruptions can be vital if other researchers are to replicate the process and really understand how the researcher came to the conclusions they did.

An increasing number of researchers in education and related disciplines have challenged the traditional research paradigm in recent decades, recognizing that rigid paradigms fail to serve researchers well in a complex world. In the oft-cited book After Method, sociologist John Law (2004) made an argument somewhat similar to Cook’s (2009): “hegemonic and dominatory pretensions” (p. 4) within the traditional, restrictive culture of research methods limit researchers in their quest to capture a complex, messy reality. He wrote, “if we want to think about the messes of reality at all then we’re going to have to teach ourselves to think, to practise, to relate, and to know in new ways” (p. 2). Part of this new way of thinking and practicing involves room for emotionality in research as well as an understanding that the researcher plays an integral role in shaping a reality that we can never have a complete, firm grip on. Prior (2017) exposed the emotional side of analyzing participant narratives, writing: “I was not just exposed to emotionality by vicariously ‘reliving’ participants’ narratives through repeated data analysis; I was also (re)exposed to emotionality by reliving the intensity of the interviews and the fieldwork” (p. 176). The emotional aspect of researching is often avoided and even discouraged under the guise of objectivity. Prior’s (2017) suggestions of self-care and taking breaks from data are welcome. Appleby (2017) discussed how to deal with controversial findings in which participants express racist views or detail illegal activities or emotionally charged experiences. He suggested thinking through issues that might arise in advance, so one can think of how to respond or not. For instance, should one be silent or speak out if interviewees express ideas that are objectionable to the researcher? As Ruecker details in Chapter 8, speaking out carries risks that can interrupt the research, especially if relations with participants are not fully established.

Law (2004) argued that we need to dismiss our notion of “hygiene” (p. 9) in research because our obsession with clean, neat research methods limits researchers’ abilities to uncover new findings in their work. It can also limit researchers from tackling ambitious and challenging critical work. For instance, Kubota (2017) explained that she has generally avoided studying those less powerful than herself, such as migrants, because of the discomfort and risks of ignoring the institutions and policies that oppress less powerful groups. Cook (1998) detailed the c...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• List of illustrations
• List of contributors
• 1. Introduction: sanitized research descriptions and messy realities: A story of contrasts
• PART I: Getting started: navigating bureaucracies and recruiting participants
• Part II: Relationships across contexts: geographical, cultural, political, institutional
• PART III: Interruptions during data collection: making it work
• PART IV: The disruptive forces of scholarly peer review
• Index