How Teachers Can Turn Data into Action

Daniel R. Venables

• Publisher: ASCD
• Year: 2013
• ISBN:
  9781416618805
• Topic: Education
• Subtopic: Education General

From state and Common Core tests to formative and summative assessments in the classroom, teachers are awash in data. Reviewing the data can be time-consuming, and the work of translating data into real change can seem overwhelming.

Tapping more than 30 years' experience as an award-winning teacher and a trainer of PLC coaches, Daniel R. Venables, author of The Practice of Authentic PLCs: A Guide to Effective Teacher Teams, soothes the trepidation of even the biggest "dataphobes" in this essential resource. Field-tested and fine-tuned with professional learning communities around the United States, the Data Action Model is a teacher-friendly, systematic process for reviewing and responding to data in cycles of two to nine weeks. This powerful tool enables you and your teacher team to

* Identify critical gaps in learning and corresponding instructional gaps;
* Collaborate on solutions and develop a goal-driven action plan; and
* Evaluate the plan's effectiveness after implementation and determine the next course of action.

With easy-to-use templates and protocols to focus and deepen data conversations, this indispensable guide delineates exactly what should be accomplished in each team meeting to translate data into practice. In the modern sea of data, this book is your life preserver!

---

Data Meeting 1: Reviewing Existing Data and Asking Questions

---

. . . . . . . . . . . . . . . . . . . .

A Word About Protocols

Protocols are tools. Tools make the job at hand easier to do. Protocols help teachers keep their discussions focused, their time used efficiently, and their actions purposeful and impactful. They keep us on track so that we talk about the stuff that matters and do so in a structured, safe way that promotes depth and honesty of discourse. They are useful in a wide range of educational settings, from looking at student work, to setting group norms, to troubleshooting obstacles. To learn more about using protocols in these and other settings, see The Practice of Authentic PLCs: A Guide to Effective Teacher Teams (Venables, 2011); The Power of Protocols: An Educator's Guide to Better Practice (McDonald, Mohr, Dichter, & McDonald, 2007); and Protocols for Professional Learning (Easton, 2009).

For our purposes here, protocols are useful for looking at data and discussing how best to respond to the data. In addition to providing structure and keeping us focused, they also take the stinger out of the oft-threatening task of looking at data—particularly for the scores of teachers who, by their own admission, would prefer to have nothing to do with data. I affectionately refer to these folks as "dataphobes," and I spend a fair amount of time with teacher teams helping them get past the initial dread of reviewing charts, tables, and graphs. The fact that teachers most often view data publicly, as a team, doesn't help dataphobes' feelings of numerical inadequacy. Protocols can come to the rescue.

One protocol we use in the Data Action Model, the Notice and Wonder Protocol, helps take the angst out of the prospect of facing data. It also helps teachers and teacher teams who are examining data in three ways:

1. It makes it easier for teachers to internalize and make sense of the data;
2. It forces teachers to have a conversation about what the data actually say; and
3. It assuages the temptation on the parts of teachers to rush to conclusions prematurely.

Let's take a closer look, beginning with a cautionary comment on the dangers of jumping to (often erroneous) conclusions.

Do First: Review Existing Data

After teacher teams have inventoried their existing data as described in the "Before Meeting" chapter, it's time for the teams to choose one or two data reports from the usual mass of reports and to begin the cycle of data inquiry. In most cases, this will be macrodata and, most often, state or CCSS assessment data or district-administered quarterly assessment data. Regardless of the data's specific source and composition, the information chosen in this first step typically paints in broad brushstrokes where students are in their mastery, and here is where we rightfully begin. As we'll see as we continue through each step of the Data Action Model, there will be sufficient time later to paint in very fine brush lines.

The Problem That Isn't

The Data Action Model is designed so that teachers and teacher teams engage thoroughly with their data before deciding what the problems of learning and practice are. Indeed, the first phase, Gathering and Reviewing Data, makes up a full third of the model, yet it includes nary a mention of teams attempting to elucidate problems of student learning that are present in the data. This, of course, is an eventual goal of the model, but it will not be addressed in significant detail until Data Meeting 3: Determining Gaps and Goals.

There is a good reason for this delay in identifying problem areas. In my consulting practice, prior to developing the Data Action Model, I kept seeing a consistent and problematic phenomenon crop up with even the most well-intentioned data teams and PLCs: in their zeal to improve student achievement, these teams rushed to find "the problem" from the data. They would come to the meeting room with a hunch as to what the problems were in their students' learning and thus reviewed the data only cursorily—if at all—and used it chiefly to corroborate what they already believed to be true about their students' weaknesses. By viewing the data only through the lens of a preconceived problem, they saw only the numbers that supported their preconceived expectations and often overlooked other areas in need of attention—even dire attention. Sometimes, their hunches were dead-on, and the data did indeed support their hunches about their students' learning gaps. Other times, however—very often, in fact—teams focused on an area that was relatively small in comparison to what I started calling their other "bleeding arteries." In other words, their area of focus was not the central problem but instead symptomatic of a different, often broader problem. At best, this misaligned focus led to a Band-Aid approach to solving their real problem; at worst, it left them spending untold energy and time solving "the problem that isn't." If the problem is misdiagnosed, after all, the solution cannot be effective.

A few years back, I was assisting a 9th grade Algebra I PLC whose members, after briefly reviewing the results of a district-administered quarterly assessment, decided that their students couldn't successfully solve word problems that required them to apply their knowledge of two-step equations. These teachers' knee-jerk assessment of the problem was that they weren't giving the students enough practice in solving word problems with two-step equations and that the solution was, therefore, to provide the students with more opportunities to practice such problems. Only after delving into the data in more detail and with additional sources did the team realize that the students' deficiencies did not result from practicing too few word problems involving two-step equations; rather, the word problems they did practice were all essentially clones of only three different word problems. The students had become proficient at solving those three problems, but they never understood the underlying concepts behind the application of two-step equations. Their lack of understanding showed up on the district assessment when they had to solve problems that didn't resemble any of the three problems they had "learned."

Time and again, I witnessed this same phenomenon as I worked with various teams in a host of different schools. What was needed, I decided, was a protocol that forced both a thorough look at the data and a delay in concluding where the problems were, why they existed, and how they could be solved. The result was the creation of the Notice and Wonder Protocol.

Notice and Wonder Protocol

This protocol is remarkably popular wherever I share it. Teachers appreciate that the Notice and Wonder Protocol is a simple and effective tool that offers a nonthreatening way for them to view data and share observations about what they see. Anyone can notice and anyone can wonder, right? The details of the protocol appear in the Appendix.

Here's an overview of how the protocol works. Teachers are given a data report (or two) of some kind and quietly and individually review it. In Round 1, on an index card or in their notebooks, they record several factual things they notice in the data. These observations are free of inference and shared without discussion; teams often post their statements in a shared Google document. In Round 2, on the back of the index card, they note "wonders": wonder whys, wonder ifs, wonder whethers, and wonder hows. These observations may not be outright inferences but usually contain a speculative implication. Teachers share these within the group while a volunteer scribes them on chart paper. No discussion occurs at this time, except for follow-up questions the team facilitator may ask of the teacher or the team. These, too, are posted for the team to see.

To look at the Notice and Wonder Protocol in action, consider the table in Figure 4. This report reflects how each subgroup of 7th grade English language arts (ELA) students performed across five strands on a midyear standardized assessment. The subgroups are based on gender, race, economic status, students with disabilities, and English language learners. The percentages listed in the table represent the mean percentage of test items the student subgroup answered correctly in each particular strand. For example, the first subgroup, representing the 65 female students tested, had a mean scaled score of 760 and, on average, correctly answered 73 percent (11 of 15) of the questions in the Gathering Information Skills strand, 71 percent (5 of 7) of the questions in the Organizing Information Skills strand, 78 percent (7 of 9) of the questions in the Analyzing Information Skills strand, and 80 percent (4 of 5) of the questions in the Linking Information strand. (Note that the percentages are based on the average number of questions answered correctly and have been rounded up if their actual value is a decimal greater than or equal to 0.5. For example, a mean percentage listed as 100 percent may actually have been a mean of 4.5 questions answered correctly and then rounded to 5 of 5.)

---

Figure 4. 7th Grade ELA Midyear Assessment Report

Note: ED = economically disadvantaged; SWD = students with disabilities; ELL = English language learners.

---

The teacher team reviewing these data, the ELA 7 PLC, includes five ELA classroom teachers and one special education teacher who works with each of the classroom teachers on a rotating basis, assisting the 22 students with disabilities who are in general education classes. The ELA 7 PLC meets twice weekly during their hour-long common planning period, and one of those meetings is strictly dedicated to looking at and responding to data. Their PLC coach (facilitator) begins the meeting by projecting on the interactive whiteboard the data table in Figure 4.

For this data set, teachers in the ELA 7 PLC might offer the following Notice Statements in Round 1:

• I notice that more female students were tested than male students.
• I notice that, overall, students answered at least 70 percent of the test items correctly in three of the four strands.
• I notice that students, on average, scored least well in the Organizing/Interpretive strand.
• I notice that Hispanic students did least well in comparison with other subgroups in the Gathering/Literal strand.

The Notice Statements that are shared may be superficial in nature, as with the first one listed here, or they may pivot on some real and significant facts about the performance of a student subgroup, such as the third and fourth statements listed. All of the statements made are factual, as the protocol instructs, and it does not matter at this point that some of the Notice Statements are meatier than others. What's more important at this juncture is that all members of the team—even the dataphobes—have offered Notice Statements.

By having to make only factual observations, according to the protocol, the team is freed up from the common, self-imposed responsibility to draw conclusions and rush to try to fix problems. Instead, the team can look at simply what the data are saying. Their focus is entirely on the what and not on the why or how to fix it. This step is liberating for teachers who are uncomfortable with the notion of looking at data in the first place and who are used to dealing with the tacit expectation that they must quickly determine solutions. Because the protocol does not allow proposed solutions or even mention of underlying problems and asks teachers only to discuss what is revealed by the data in a factual, objective way, teachers who usually prefer to hang back during data conversations tend to feel more comfortable contributing.

Afterward, in Round 2, teacher teams are given the green light to make speculations and deeper observations in the context of Wonder Statements. For the ELA data in Figure 4, the ELA 7 PLC might offer the following Wonder Statements:

• I wonder why students did better in higher-order strands (Analytical and Critical Levels) than on the Literal and Interpretive Levels.
• I wonder why our female students are doing so much better on the Organizing/Interpretive level than their male counterparts.
• I wonder what percentage of our students of color (black, Asian, Hispanic, multiracial) are eligible for free or reduced lunch.
• I wonder if our expectations of our students in the Interpretive Level are too high or too low during instruction.

Without making a conscious effort to push the conversation to a deeper level, the transition from Round 1 (notice) to Round 2 (wonder) naturally does just that. There is good reason for this transition. Notice Statements focus on what happened, whereas Wonder Statements congregate around why it happened, how it happened, and what might be possible.

A couple of points are in order on Notice and Wonder Statements. First, the Notice Statements shared in Round 1 often resurface in Round 2 as Wonder Statements. For example, referring back to our ELA 7 data, a teacher might offer the Notice Statement I notice that African American students had greatest difficulty with the Analytical items, while Hispanic students had greatest difficulty at the Literal Level. In Round 2, a different teacher (or possibly the same teacher) might share the Wonder Statement I wonder how we're teaching analyzing skills that our black students do so poorly on this strand. Although Wonder Statements might reflect earlier Notice Statements, it is important for teams to realize that this isn't necessary; all Wonder Statements are fair game whether or not they are connected to previous Notice Statements.

Second, by the end of Round 2, the team has probably listed 15 or 20 Wonder Statements. As the team keeps its focus on its eventual goal of improving instruction, members inevitably realize that not all Wonder Statements are of equal value in terms of advancing that goal. Some of the Wonder Statements may even seem relatively insignificant compared with others. For example, I wonder how much higher these kids would score if they read more at home is less valuable to the team than I wonder if we're spending too much time with the vocabulary quizzes. In other cases, some Wonder Statements on the list can be answered with just a bit of checking, such as I wonder how many days we spent on improper fractions or I wonder if the students with special needs are permitted to use a calculator on the decimals module. Although these are all legitimate Wonder Statements, they do not...

Table of contents

• Cover
• Praise for How Teachers Can Turn Data into Action
• Title Page
• Table of Contents
• Dedication
• Acknowledgments
• Preface
• Introduction
• Before Meeting: Developing Data Literacy
• Data Meeting 1. Reviewing Existing Data and Asking Questions
• Data Meeting 2. Triangulating the Data
• Data Meeting 3. Determining Gaps and Goals
• Data Meeting 4. Planning for Action
• Implementation Period: In the Classroom
• Data Meeting 5. Evaluating Success and Determining Next Steps
• Putting It All Together
• Appendix: Protocols, Guides, and Tools
• Bibliography
• About the Author
• Study Guide
• Related ASCD Resources
• Copyright