Looking for Insight, Transformation, and Learning in Online Talk

Trena M. Paulus, Alyssa Friend Wise

• Publisher: Routledge
• Year: 2019
• ISBN:
  9781315283234
• Edition: 1
• Topic: Education
• Subtopic: Inclusive Education

Looking for Insight, Transformation, and Learning in Online Talk is a comprehensive guide to analyzing digital interaction in formal and informal online spaces. The book establishes a new research framework for addressing major challenges that have arisen as social exchanges, meaning-making, and knowledge-building increasingly take place in social media, discussion forums, and online communities.

With a focus on methodological alignment to support valid and trustworthy knowledge claims, the authors present a series of design decisions to help researchers:

• 
  
  

• frame their object of interest and unpack underlying assumptions

• 
  
  

• understand key differences between researcher-influenced and pre-existing online talk

• 
  
  

• ethically extract and organize data for analysis

• 
  
  

• apply rigorous qualitative, quantitative, and computational methods to answer their research questions

Written for scholars in education, business, communication, media studies, health sciences, political sciences, and beyond, this is a thorough approach to the research methods and concerns essential to the study of talk in online contexts.

1

WHY INVESTIGATE ONLINE TALK?

Introducing the Research Design Framework

Scope of the Chapter

• Establishment of the purpose and goals of this book

• Introduction to the authors and their experience with investigating online talk

• A brief history of online talk research

• Definitions of key terms

• A preview of the research design framework

• An outline of the structure of each chapter

Introduction

In late 2018 as we were finishing this book, the Pew Research Center reported that the American population had reached near-saturation levels of computer, cell phone, Internet, and social media adoption. In February of that year, roughly seven out of ten Americans reported being active on social media—engaging with news and entertainment, sharing information, and connecting with each other (Smith & Anderson, 2018). As social science researchers, we want to understand human life—what happens in social worlds and how people experience them. Our research methods for doing so have been honed over time—surveys, observations, interventions, interviews. Studying how people talk together online—or designing interventions that invite people to talk online—can provide invaluable insights into what is happening in a social world increasingly connected through its devices. However, online talk (also referred to as computer-mediated communication, online discussions, Internet-based communication, and so on) is complex—it is varied in form, ambiguous in origin, persistent, and often without a natural endpoint. It involves multitudes of prospective participants competing for the floor, overlapping their turns of talk, and conversing on virtually limitless topics of interest. As such, online talk as a research focus can be challenging to define and analyze. Researchers have, however, been studying computer-mediated communication for over four decades now. We wrote this book to help researchers learn from this previous work and better design studies to investigate the phenomenon that is online talk.

People talk together online in a multitude of ways and in a variety of spaces. While in 2018 social media refers to interactions that take place through the latest generation of mobile-enabled apps, these platforms are not the only spaces in which online talk occurs. Online support and hobby groups engage using threaded discussion forums, people share their stories in blog posts, and educators teach in entirely online learning management systems. These spaces can emerge spontaneously or be intentionally designed, can occur as part of formal or grassroots endeavors, and are of research interest because they are spaces in which insights, transformation, and learning among people are made visible. Understanding what happens in these spaces can have value across a wide range of disciplines from public health to consumer marketing and from international politics to environmental education.

Yet too little attention has been paid to the methodological design of studies into online talk, bringing into question the validity of the conclusions that can be drawn. In this book we overview the particular set of analytic issues and challenges that arise in identifying and analyzing the insights, transformations, and moments of learning that occur (what we call the “objects of interest”) in online talk—whether they arise by intentional design, incidentally, or sometimes not at all.

About the Authors

I (Trena M. Paulus) was trained as an applied linguist, language teacher, instructional designer, and qualitative research methodologist. My language teaching experiences in the early 1990s coincided with the development of the Internet, and I was instantly intrigued by how virtually connecting native and novice speakers of a language might facilitate faster fluency. This grew into a more general interest in online teaching and learning, and my first study of online talk was an analysis of the asynchronous (delayed-time) threaded discussions that took place in an online university teacher education course in 2002—long before Facebook, Twitter, or anything called social media was mainstream. Our research team was fascinated by how these discussions among future teachers were, in essence, visible displays of what they knew and believed about the course topics. Even more revelatory was that it seemed almost possible to trace a change in these beliefs—online talk seemed to be, in essence, a space that displayed visible evidence of learning.

In that first study we used Garrison, Anderson, and Archer’s (2001) practical inquiry model as a theoretical framework to understand the online talk by coding, counting, and tallying up the segments that fit into each phase. I clearly remember how difficult it was to operationalize the theoretical framework into coding categories and the struggle to come to agreement with other analysts on which segment fit into which category. It seemed like it should be completely straightforward, but it was not, and this sparked my interest in methodological innovation around ways to make sense of online talk.

Since then, I’ve worked as a methodologist with dozens of researchers interested in tackling this ubiquitous “source of data” to answer important questions about what happens in online spaces—not only in university courses, but also in support groups, hiking communities, crowdfunding campaigns, library chat reference interactions, and on Twitter. Each study poses unique challenges around identifying the precise object of interest, how an enormous corpus of online talk should be bounded for analysis, and how best to approach that analysis.

∗∗∗

I (Alyssa Friend Wise) come from a background in the sciences with subsequent training in instructional technology, learning analytics, and the learning sciences. Despite this apparent focus on quantitatively oriented methodologies, I also have a longstanding interest in understanding the ways people come together to build up systems of meaning, which has kept me engaged with and open to the different kinds of knowledge qualitative approaches offer. More recently, I have become fascinated with the possibilities for data mining and interpretive approaches to inform each other, and the possibility that such combination might be the start of bridging the quantitative-qualitative divide.

I initially became interested in online talk through the lens of assumptions—I wanted to investigate many of the untested ideas that people held about how online talk occurred, in order to validate, refine, or refute them. My first study focused on the idea of “social presence” (Wise, Chang, Duffy, & Del Valle, 2004), that is, the degree to which people feel like they know the other people they are talking to online. Along with notions of community, social presence had been commonly considered important for online learning; however, all empirical support for this claim was correlational. So we conducted a study that directly manipulated the level of instructor social presence in a one-on-one online mentoring environment. The results were surprising; students both recognized and reciprocated the social presence exhibited by the instructors, but this didn’t appear to impact the learning. Since then, I’ve conducted numerous studies to explore other assumptions about online talk: That people necessarily explore disagreements before moving past them, if and how people actually “listen” to each other before “speaking” online, and that meaningful community cannot be found within large online crowds of conversants. Time after time, I’ve seen how the decisions we make around how to transform online talk into data for a study and how that data is analyzed have critical implications for the quality of a study’s outcomes.

When we were approached about writing a handbook chapter on analyzing online learning discussions for the second edition of The SAGE Handbook of E-Learning Research (Haythornthwaite, Andrews, Fransman, & Kazmer, 2016), we jumped at the chance to combine our years of experience studying online talk from different methodological stances in an effort to create a congruent framework for identifying moments of insight, transformation, and learning. In doing so, we realized that this framework was the first of its kind. While information on how to conduct some individual parts of the process was available, a comprehensive (and coherent) guide for studying online talk did not exist. This book extends that original handbook chapter to offer such a guide, the kind of comprehensive resource we wished we had available when we were learning how to conduct research in these spaces. We hope it is useful to researchers embarking on studies of online talk for the first time.

Definitions and Scope of the Book

Because not all communication in online environments can necessarily be characterized as multiparty conversations, we adopt the phrase “online talk” to describe what we are studying. While somewhat awkward, we use the phrase “author of the talk” when referring to those who participate in these spaces, because they are talking through writing (or typing, or voice-texting). We limit our scope in this book to the analysis of online talk in which the communication occurs in a written (text-based) form, both because this is still one of the most frequently occurring modalities through which such discussions take place and because this single channel allows for a clear focus on how the talk can be analyzed as evidence for insights, transformation, and learning. We acknowledge that image- and video-based communication platforms are increasing at a rapid rate and, alongside this growth, research methods are evolving that can best capture these exchanges for meaningful analysis. Certainly, much of the guidance presented will also be applicable to the analysis of talk which occurs via voice or video technologies; however, the addition of these channels adds analytic concerns not dealt with here (for example, see Goldman, Pea, Barron, & Derry, 2014 and O’Halloran, Tan, Smith, & Podlasov, 2011).

We consider online talk to include both discussions that are designed in the hopes that particular insights, transformation, and/or learning will occur—such as online university courses or social media campaigns—as well as those that emerge informally among people with a common interest, such as online support or hobby groups. Researchers may enter the online talk space as part of the design team, in which case the online talk is researcher-influenced. In contrast, researchers can also work with pre-existing talk as a source of data. Online talk includes exchanges that occur synchronously in real time, such as via text messaging, and those that occur asynchronously across “delayed time” (Christie & de Alberdi, 1985), through platforms such as blogs and discussion forums. It includes conversations that grow as a single linear string of comments, those that branch out into multiple threads, and those distributed across multiple communication tools.

Online talk can take place between an individual and the “invisible audience” of anyone who may be listening out there on the Internet, two individuals talking directly to each other, members of multiparty groups, or across large aggregations of people. It may take place as an event unto itself or be coordinated with actions in a system (e.g., a chat that is linked to editing a Google Doc). Thus, situations as diverse as an email exchange, a text chat among a project team, or the multi-participant comment stream following an online news story would all be considered spaces of interest to researchers of online talk. What all of these different situations have in common is the...

Table of contents

• Cover Page
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• List of Figures
• List of Tables
• Preface
• Acknowledgments
• 1 Why Investigate Online Talk? Introducing the Research Design Framework
• 2 What Are We Looking For? Identifying the Object of Interest
• 3 What Does Online Talk Represent? Philosophical Assumptions and Methodological Alignment
• 4 What is the Research Question and Approach? Informing the Study Design from Prior Work
• 5 When Can Online Talk be Treated as Data? Ethical Practices
• 6 How Will the Data be Acquired? Characterizing, Bounding, and Extracting Online Talk as a Data Source
• 7 How Will the Data be Analyzed? Part One: Quantitative Approaches including Content Analysis, Statistical Modeling, and Computational Methods (with Robyn Singleton)
• 8 How Will the Data be Analyzed? Part Two: Qualitative Approaches including Thematic, Narrative, Conversation, and Discourse Analysis
• 9 Have We Found What We Were Looking For? Revisiting the Research Design Framework
• Index