Building Writing Center Assessments That Matter

Ellen Schendel,William J. Macauley

• Publisher: Utah State University Press
• Year: 2012
• ISBN:
  9780874218343
• Topic: Sprachen & Linguistik
• Subtopic: Sprachwissenschaft

No less than other divisions of the college or university, contemporary writing centers find themselves within a galaxy of competing questions and demands that relate to assessment—questions and demands that usually embed priorities from outside the purview of the writing center itself. Writing centers are used to certain kinds of assessment, both quantitative and qualitative, but are often unprepared to address larger institutional or societal issues. In Building Writing Center Assessments that Matter, Schendel and Macauley start from the kinds of assessment strengths already in place in writing centers, and they build a framework that can help writing centers satisfy local needs and put them in useful dialogue with the larger needs of their institutions, while staying rooted in writing assessment theory.

The authors begin from the position that tutoring writers is already an assessment activity, and that good assessment practice (rooted in the work of Adler-Kassner, O'Neill, Moore, and Huot) already reflects the values of writing center theory and practice. They offer examples of assessments developed in local contexts, and of how assessment data built within those contexts can powerfully inform decisions and shape the futures of local writing centers. With additional contributions by Neal Lerner, Brian Huot and Nicole Caswell, and with a strong commitment to honoring on-site local needs, the volume does not advocate a one-size-fits-all answer. But, like the modeling often used in a writing consultation, examples here illustrate how important assessment principles have been applied in a range of local contexts. Ultimately, Building Writing Assessments that Matter describes a theory stance toward assessment for writing centers that honors the uniqueness of the writing center context, and examples of assessment in action that are concrete, manageable, portable, and adaptable.

1 THE DEVELOPMENT OF SCHOLARSHIP ABOUT WRITING CENTER ASSESSMENT^1 William J. Macauley, Jr.

After most of the writing center assessment workshops, sessions, and talks Ellen and I have done together, participants have shared their high levels of frustration with not finding scholarship to support assessing their writing centers. Coupled with the increasing assessment pressure that so many writing center directors (WCDs) are feeling, these worries have only escalated. Workshop participants have often found little scholarship on writing center assessment in the usual library databases. Another concern is that the scholarship on writing center assessment is interesting but doesn’t really answer the right questions. These frustrations make writing center assessment increasingly problematic, even as the pressure mounts to develop, conduct, and complete meaningful writing center assessment.

If we step back from these particular conversations, though, this has been a field-wide issue for some time. In fact, the frustration and confusion surrounding writing center assessment has been a concern for WCDs for several decades. Back in 1982, Janice Neulieb pointed out that the first problem in evaluating a writing center is that:

there is no established method for going about the evaluation… . The director is faced with the prospect of creating a new research design that somehow anticipates all the possible questions that will be asked by those who read the finished report. (227)

This is apparently still true for many WCDs. Assuming that the concerns Ellen and I have heard directors voice are at least minimally representative, it seems as though the field has not yet overcome this problem.

In fact, because WCDs are now moving out into their campuses and participating in institutional discussions and decision making, this limitation is more frustrating and becomes a higher stakes issue for the center and its director. Inexperience with assessment also becomes “a weakness” in the director that is easier for others to see. Making the right assessment choices seems even more important now that assessment is more than a campus conversation but also a public and political one.

However, there actually is quite a bit of writing center assessment information that directors can use to educate themselves and think through their own assessment plans and procedures. Thirty years ago, Mary Lamb (1981) surveyed all of the writing centers she could find (120 at that point) in order to find out what assessment practices were most frequently in use. She identified basic counting, questionnaires, pre-/post-tests, external evaluation, and professional staff publication/activity as the most frequent writing center assessment methods, findings that share a great deal in common with the bulk of practices we hear and read about today. A year later, Joyce S. Steward and Mary K. Croft, in The Writing Laboratory: Organization, Management, and Methods (1982), wrote:

A lab director can choose from several kinds of evaluation: internal (reactions of tutors and tutees), school or campus-wide (reactions of referring faculty and departments), and external (use of a professional consultant); and can collect data through questionnaires, surveys, interviews, discussions, and case studies. (92)

Stephen North (1984), only two years later, argued that “writing center research has not, for the most part, been formal inquiry by which we might test our assumptions. It has tended to fall, instead, into one of three categories,” which North identified as “reflections on experience,” “speculation,” and “survey” or questionnaire-based methods (24–5). James H. Bell (2000) reiterated this critique more than a decade later: “Writing centers should conduct more sophisticated evaluations” (7, emphasis original). While North named three specific methods that seemed to dominate writing center research in the 1980s, Bell’s comments in 2000 suggest that the sophistication of writing center research methods had not progressed. Reflections, speculations, and surveys may have become so familiar, so commonplace, that their appropriateness or limitations aren’t even questioned anymore. And these common understandings among professionals in the field may have removed the need to explain why these methods figure so prominently in the way we track the successes of our centers and push our centers to grow and change. For the WCD trying to understand and choose assessment methods, the absence of such discussions only complicates an already significant set of challenges. Likewise, there may be little scholarship that helps even a seasoned WCD articulate the “why” behind assessment choices. This omission is especially problematic when the audience is people outside of the writing center who receive, read, or act on writing center assessment reportage.

But the field’s blind spot may not be borne out of simple familiarity. A number of scholars have also pointed to quantitative reluctance and inexperience as limitations on writing center assessment (J. Bell 2000; Donnelli and Garrison 2003; Field-Pickering 1993; Henson and Stephenson 2009). Frequently cited among these scholars is Cindy Johanek’s 2000 book, Composing Research: A Contextualist Paradigm for Composition and Rhetoric, where she argues that researchers in composition studies tend to appreciate narrative and literary types of research methods over quantitative ones. In addition, Johanek writes, numbers rouse math anxiety and are too frequently accompanied by dry writing that argues from a position of objectivity that few in our field would support or accept. In Peter Carino and Doug Enders’s “Does Frequency of Visits to the Writing Center Increase Student Satisfaction? A Statistical Correlation Study——or Story” (2001), the authors claim that their statistical research enabled them to “stop fearing numbers and love the interpretation of them,” and celebrate that change as a significant shift in their thinking and work (83). Even so, within that same article, they note some concern that their writing could become dry or boring because of the introduction of numbers and statistical analyses. Who wants to produce that kind of writing?

Luke Niiler (2005) comments, “Our field needs to complement its abundantly rich qualitative research with work that can be transferred from one site to another,” proposing the kind of formal inquiry that North discussed in Gary Olson’s Writing Centers: Theory and Administration (1984) (14). Niiler goes on to suggest, “Perhaps statistically grounded research will begin to move us in that direction” (14). Given that North also called for more sophisticated methods of inquiry (formal, transferable) more than twenty-five years ago, it is somewhat disheartening that current researchers are still hoping to see that development. But it’s not hopeless. Writing center scholars have at least come to acknowledge that quantitative methods can support and/or complement qualitative assessment practices. Johanek writes:

Numbers alone won’t reveal everything we need to know. Stories alone can’t do it, either. But, when researchers stop defining their work by method only … then the full power of any data, be it story or number, will truly blossom into the knowledge our field seeks and the discipline we hope to become. (209)

For the new-to-assessment director, these circumstances create a real problem. As thinkers and researchers who largely came up through the academy within composition studies, literary studies, or allied fields, our field’s preferences for literary, narrative, and qualitative methods may be very strong. However, it seems as though the primary qualitative methods available for writing center assessment may be so fossilized within the field that the scholarship no longer explicitly articulates their foundations. We don’t see detailed discussions of why firsthand experience should be such an integral part of a scholarly argument in writing center research. Scholars in our field seem to feel quite comfortable with speculation as an outcome of research, and there are only a handful of published articles that demonstrate continued inquiry that moves beyond speculation. There is no denying the prominence of survey- or questionnaire-based inquiry in writing centers, even though the limitations of these methods are clear. In short, informed choices among qualitative methods for writing center assessment may be hard to come by. Quantitative methods may be no better an option. Our collective resistance to them is undeniable, and many WCDs are not trained in these particular data-gathering methods or analyses.

FINDING RESOURCES USEFUL TO BUILDING WRITING CENTER ASSESSMENTS

And yet, despite the present conversation about writing center assessment focusing on what we may not know, a simple Google search for the phrase “writing center assessment” reveals a wide range of writing center assessments being done at a variety of institutions. Our lack of expertise or familiarity with numerical research has not stopped us from generating assessment data. Of the seventy relevant hits generated by that Google search, the first thing that becomes apparent is the variety of venues sponsoring conversations about writing center assessment. The Web provides information on institutes/conferences focusing on writing center assessment (“East Central”; “2011 Summer Institute”; “MAWCA 2011”) as well as special interest groups at other conferences (Ballard), descriptions of individual workshops (Caswell and Werner), and materials supporting those presentations (Law).

One might also be surprised by the variety of writing center assessment documents available online, which WCDs can adapt to their own contexts and purposes:

• Entire writing center assessment plans (“Caldwell Community College”; Paoli, Silver, and Koster; “The University Writing Center (UWC)”; “University Writing Center Assessment Plan”),

• Examples of writing center assessment reports (Andrews and Kelly-Riley; Modey; Smith and Talavera; “Assessment Report”; “University Writing Center, University of Wisconsin-Platteville”),

• Combined plans/reports (“Institutional Support Area Assessment Report”; Copas; “Salt Lake Community College”).

• A number of other less-familiar writing center assessment documents include a tutor self-assessment form (“Tutor Self-Assessment Form”), a program outcomes assessment worksheet (“Program Outcomes Assessment Worksheet”), a writing center assignment sheet that invites professors to articulate their goals and objectives for assignments that students will bring to the writing center (“Writing Center Assignment Sheet”), and a heuristic for developing writing center assessment plans (Lerner and Kail, “Heuristic”).

There are also a variety of materials intended only as digital texts that can help directors shape their thinking about assessment approaches and the instruments and methods used to collect assessment data. One of the most prominent of these digital resources is The Peer Writing Tutor Alumni Research Project website (see appendix to this volume), which assesses the long-term impacts of working as a peer writing tutor as well as invites writing centers and writing center alums from all over the world to contribute to the growing data available through the project (Kail, Gillespie, and Hughes). Another valuable digital resource is the Writing Centers Research Project, housed at the University of Louisville’s website. It includes results from several international surveys of writing centers as well as data about director salaries, tutor wages, number of sessions conducted per year, and other basic information about writing centers that can help directors with benchmarking data. The journals Praxis, Writing Lab Newsletter, and Writing Center Journal all have web presences, and simple searches at those websites will reveal a number of scholarly articles about writing center theory and practice. The University of Wisconsin–Madison Writing Center provides a series of podcasts on writing center assessment (three including Harry Denny and Lori Salem and one including Jill Pennington, Neal Lerner, and Jason Mayland).

And there are a host of useful bibliographic resources, as well. A wiki on writing center assessment was started through Wikia (“Writing Center Assessment Wiki”), and the Northern California Writing Centers Association provides a discussion forum on writing center assessment that includes a useful bibliography (Griffin and Dennen). A particularly thorough resource is Neal Lerner and Harvey Kail’s “Writing Center Assessment Bibliography,” which is both annotated and available online. Their annotations provide insight into not just the content of the included sources but also the historical significance and unique qualities of pieces that can be useful in connecting writing center assessment to other fields. “A Selected Bibliography on Empirical Writing Center Research,” available at the International Writing Centers Association (IWCA) website, although not focused specifically on assessment, does bring together in one place a wide variety of research methods and discussions that are immediately relevant to writing center assessment. Lerner’s “Dissertations and Theses on Writing Centers” and the “IWCA Bibliography of Resources for Writing Center Professionals” could also be mined for useful and relevant scholarship. Finally, a thorough (maybe the most thorough) annotated bibliography on writing centers is Christina Murphy, Joe Law, and Steve Sherwood’s Writing Centers: An Annotated Bibliography (1996), a collection of more than 1,400 entries, which includes a section focused specifically on writing centers research. Although these bibliographies may not be entirely up to date, they provide discussions of evaluation, research, and scholarship in writing centers that can be readily adapted to current writing center assessment purposes. The connections these resources make to other fields and areas of study can be very informative for writing centers and their assessment, as well.

However, while all of these online sources and bibliographies include materials that can be very useful to WCDs, the concerns raised earlier still hold. Without some access to foundational ideas informing writing center assessment, much of this literature can seem only remotely relevant or useful. How to build a coherent, workable assessment plan that draws on these various methods is what seems to be lacking. Without a strong foundation in the goals, options, and processes for planning, implementing, and reporting on assessment, making sense of the rich resources already available is a challenge. There is no doubt that writing program assessment, educational assessment, and writing assessment more broadly defined have a lot to offer WCDs, but without deliberate guidance that is grounded in writing center theory, pedagogy, and practice, these adjacent fields remain provocative but remote for the new writing center assessor.

A TAXONOMY OF WRITING CENTER ASSESSMENT RESOURCES

As my search for writing center assessment resources unfolded, three general types of sources emerged. The first category includes “context” pieces. Scholarship in this group works to provide some sort of larger idea related to composition studies (loosely defined) that can support and inform writing center assessment. The second category consists of “connections” pieces; articles and chapters in this group focus on what other fields and disciplines have to offer writing center assessment. The third category is made up of “methods” pieces that focus on the nuts and bolts of writing center assessment—research articles that demonstrate methods that would be useful for collecting information as part of a writing center assessment.

As I read everything I could about writing center assessment, I put each piece into one of these categories. When each of the categories was fully populated—in other words, when each article and chapter had been distributed to a scholarship category—an interesting phenomenon emerged. About half of the pieces included in this review were methods pieces. These pieces were often research articles whose relevance to this review were the processes and methods they described for gathering or analyzing data, rather than an explicit focus on writing center assessment. The next largest grouping was context pieces. Finally, the pieces “connecting” with other fields formed the smallest group.

My strategy for taxonomizing the literature on writing center assessment is by no means the only way of characterizing the themes of this scholarship. However, because I defined the context and connections categories much more broadly than the methods category, I would have expected comparatively fewer methods pieces when compared with context, connections, or context and connections combined. Be that as it may, the distribution I found does roughly support the larger argument being made here: that the literature relevant to writing center assessment does offer a great deal of information on methods while there is much less on context or connections. In other words, the literature available to writing center assessment does not seem as focused on “global” discussions as it does on the “local.” An overemphasis on the local and anecdotal is not a new criticism of writing centers research and scholarship, but it does make clearer why the new-to-assessment WCD might find the literature disembodied, decontextualized, not readily transferrable to his or her own writing center.

Scholarship That Works to Contextualize Writing Center Assessment

Context scholarship can be very helpful in any kind of research and literature review; writing center assessment is no exception. What may be unusual is the combined questions of how far outside of the field one wishes to go and which “outside” holds the greatest promise. This review limits contextual discussions to that scholarship found within writing or composition studies because these are the most readily comparable contexts for any writing center work.

If one were looking for a detailed history of writing assessment as an industry, there would be no finer choice than Norbert Elliott’s On a Scale: A Social History of Writing Assessment in America (2005). This book chronicles the rise of psychometrics and its intersection with writing assessment in the United States. Most interesting may be the discussion of aptitude testing for military service around World War I. Elliott’s book allows a potential writing center assessor to make much more informed choices, especially when she feels pushed to use quantitative methods or standardized tests. The knowledge made available here can be very useful in making methodological choices in the richest possible way. Patricia Lynne’s Coming to Terms: A Theory of Writing...

Table of contents

• Front Cover
• Half Title
• Title Page
• Copyright
• Dedication
• Contents
• Acknowledgments
• Introduction—Yours, Mine, and Ours: Changing the Dynamics of Writing Center Assessment
• 1 The Development of Scholarship about Writing Center Assessment
• 2 Getting from Values to Assessable Outcomes
• 3 Connecting Writing Center Assessment to Your Institution’s Mission
• 4 Moving from Others’ Values to Our Own: Adapting Assessable Outcomes from Professional Organizations and Other Programs on Your Campus
• 5 Integrating Assessment into Your Center’s Other Work: Not Your Typical Methods Chapter
• 6 Writing It Up and Using It
• Afterword: Translating Assessment
• Coda
• Appendix: Annotated Bibliography for Writing Center Assessment
• Index
• About the Authors