Improving Reading Comprehension of Self-Chosen Books Through Computer Assessment and Feedback

Best Practices from Research

Keith James Topping

• Publisher: Routledge
• Year: 2022
• ISBN:
  9781000598704
• Edition: 1
• Topic: Education
• Subtopic: Education General

Schools around the world use online programs like Accelerated Reader and Reading Counts to improve students' reading comprehension of real books, but how can such software be used most effectively? In this unique resource, researcher Keith Topping analyzes independent research studies and brings you best practices on quality implementation to enhance effectiveness. He explains the evidence base for the programs in a comprehensible way and addresses many common questions, such as "Does it work?, " "How should it be implemented to make it work?, " and "Is it cheaper and more efficient in teacher time than what we were doing before?" He also discusses best practices for using the assessment data, for tailoring implementation in elementary vs. high schools, and for working with disadvantaged students. Appropriate for teachers, literacy coaches, curriculum leaders, and other stakeholders, the book will provide you with a strong research foundation and easily accessible information to help you fine-tune your understanding of the reading programs and implement them more successfully in your schools and classrooms.

1 Introduction

DOI: 10.4324/9781003215882-1

This Introduction first talks about the aim of the book – what it is and what it is not. Then it gives a brief description of the programs described in the book. The main ones are Accelerated Reader (AR) and Reading Counts (RC) (and description of them is enlarged in Chapters 2 and 5). A comment on other, somewhat similar programs follows. Then the research methods used to gather information are broadly outlined, followed by a more specific section on search methods. Then a section on statistical analysis seeks to explain in simple language some of the methods that were used (but don’t worry, it’s easy to understand). The issue of who might read the book is discussed. Finally, an expanded version of this paragraph (structure of the book) gives more details about what readers might expect, to help them decide which chapters to read first.

The Aim of This Book

This book is about a single kind of software, made in different versions by different companies. It might seem extraordinary to propose a book about a single kind of software, but this is no ordinary software. It is used in about 100 countries worldwide and is the most widely used educational software in the world. Additionally, some versions have attracted a large number of evaluation studies – more than any other piece of software. Especially in the USA and other English-speaking countries, there is great emphasis on evidence-based education, and schools are looking for a single source of evidence to justify their use of such software. This book is such a single source of evidence for improving reading comprehension of real books through computer assessment and feedback.

The book is thus not a “how to do it” book for teachers, since the precise method of delivery is well covered in manuals for the software and in the training and coaching programs provided by the manufacturers (both of which, of course, change somewhat from time to time). However, the book does make some comment on the broad pattern of implementation desired and expected by the manufacturers (see Chapter 5), the variations in how the programs are actually implemented and what the effects of these variations might be (see Chapter 6). Teachers will glean important messages about implementation from these chapters, and an implementation checklist is provided to help this.

The book also offers a summary of the independent, peer-reviewed research evidence already completed and of new research freshly done. This addresses many of the major concerns of schools, which will have questions like “Does it work?”, “How should it be implemented to make it work?” and “Is it cheaper and more efficient in teacher time than what we were doing before?” All of these questions are answered in this book.

This review of evidence leads to some suggestions for how the use of these programs can be improved. It will thus reassure schools that much if not all of their practice is evidence-based and well substantiated, lead them to improve their implementation of such programs, and enable them to state and defend their case in that regard to school district managers, local and national government, parents and other stakeholders. It will also be of interest to researchers, especially in areas where data are sparser or more difficult to track.

The book is thus intended to give the evidence base for these programs in an easy and comprehensible way. Within this, it focuses on special subtopics which may be of interest to particular readers.

What Are These Programs?

Of course, schools have long had programs to encourage independent reading of self-chosen books, such as Sustained Silent Reading (SSR), Uninterrupted Sustained Silent Reading (USSR), Drop Everything and Read (DEAR), Free Uninterrupted Reading (FUR), Free Voluntary Reading (FVR), SQUIRT (Sustained Quiet Uninterrupted Reading Time), Independent Reading Time (IRT) and Writing and Reading Time (WART) (to name a few). These programs could have an element of accountability, such as book discussion groups, book reports and so on, but this becomes a problem when students are choosing their own very diverse books and is, in any event, very consuming of time.

By contrast, computer-based systems for the management of independent reading encourage students to choose a real book of their own choice (within the boundaries of what would be considered adequately challenging for their age and ability), read it and then self-test their comprehension of it with a quiz on a computer. Their quiz performance is automatically uploaded for scoring and analysis and then the results are swiftly fed back to the student, indicating the level of success. The student can then consider whether they need to try an easier or harder book or just read with more care for better comprehension. The results are also fed back to the teacher, who can then guide the student towards more effective book choices. Teachers and librarians receive feedback on individuals or whole classes of students, which enables them to manage the reading experiences of students with far greater efficiency than before. One of the advantages is that the programs enable more efficient tracking of students’ reading habits and progress while reducing the time commitment of the teacher or librarian to this task. It is also possible for results to be fed back to parents.

Thus, the programs are about practice in reading – specifically, successful practice on self-chosen books of an appropriate level of challenge. The program does not directly teach phonological skills, important as these are – but it does give students the opportunity to practice their phonological skills in the context of real books. It might be argued that this program could lead to students reading books only in order to take the quiz, but this seems implausible when reading the book takes so long, taking the quiz is so quick and there are so many thousands of quizzes. In fact, the intrinsic desire to read a book which seems especially interesting seems to be the motivating factor.

There are two main programs which deliver computer assessment and feedback on the reading of real books: AR (published by Renaissance Learning in the USA and the UK – https://www.renaissance.com and https://www.renlearn.co.uk) and RC (published by Houghton Mifflin Harcourt [HMH] in the USA – https://www.hmhco.com/programs/reading-counts and https://readingcountsbookexpert.tgds.hmhco.com/bookexpert/default.asp).

Accelerated Reader

AR was developed around 1984 by Judi and Terry Paul and has always been owned by Renaissance Learning (although Renaissance Learning has been taken over by a larger company). Since then, very large numbers of students have become involved, especially in the USA and the UK but also in many other countries. Now there are 50,000 book quizzes for it in the UK and 205,000 in the USA (2021 numbers), and these numbers grow every year. A very large number of elementary and high schools are involved globally – representing a very large number of students.

AR is also one of the best-researched methods in the field of education. There are many independent peer-reviewed studies (including randomized controlled trials and control group studies). This research has been debated and sometimes criticized, but the alternative methods offered have often not been researched at all. AR has also met the highest review standards of reputable organizations, such as the National Center on Intensive Intervention, the National Center on Response to Intervention, the National Dropout Prevention Center, and the Promising Practices Network. In the UK, AR is now marketed together with Star Reading, a computer-based adaptive norm-referenced multiple-choice reading comprehension test. In the USA, these two products can be purchased separately.

Reading Counts

The first reading management program, called the Electronic Bookshelf, was developed in 1981 by a school librarian named Rosalie Carter along with her husband Jerry Carter. This was taken over by Scholastic in 1998 and renamed RC (and switched to being based on Lexile measures of book difficulty). RC was then taken over by HMH in 2015.

In many ways, RC is similar to AR, but in some additional ways, it is different. RC has a default Pass Rate of 70%, but teacher users can adjust this to suit their requirements. However, only about 10% of teachers actually do this (see Chapter 4). Teachers can also allow retake of tests which are failed, as their book tests have a total of 30 questions by default and are adaptive, presenting a different, set of questions on each retake. Again, the number of teachers who actually do this is fairly limited (see Chapter 4). Teachers can also change the number of questions in a book test, making it longer for more able readers or shorter for less able readers. All of these features are quite different from AR.

RC is available as a stand-alone program but is also embedded in Read 180, a program that includes the same computer assessment and feedback but also small group teacher instruction and independent reading. Read 180 is mainly targeted on the second quartile of readers (i.e., 25th–50th percentile, below average but not the worst readers) (System 44 is for even lower-ability readers). Thus, Read 180 is not for average- to high-ability readers. It has an equal amount of fiction and non-fiction, although the tendency is to increase the non-fiction element.

In fact, only about 35% of RC operates as a stand-alone program (RC Only), but obviously this is the section that is of most interest to this book. Equally obviously, the cost of RC Only is much less than that of RC embedded in Read 180, but we will come to that in Chapter 13.

Given their rather different orientation, it seems that AR and RC are not in competition, but overall target rather different students in somewhat different ways. In addition, RC is about to be wholly integrated in much more complex and expensive programs, so it will then be impossible to disentangle its separate effect.

Similar Programs

There are three other programs which appear to do somewhat similar things to AR and RC.

One is Book Adventure by Sylvan Learning (https://bookadventure.com). This is said to have 40,000 quizzes and gives rewards for reading in the form of virtual “coins”. It used to be free but now costs US $50 (2021 prices) per student per year – clearly much more expensive than AR or RC (see Chapter 13).

The other is ReadnQuiz (https://readnquiz.com), which is said to have 50,000 quizzes and allows retakes of tests (similar to RC). The quizzes are “written by librarians and teachers” (i.e., many different people who may or may not be experienced in test writing). There may be up to 40 questions per quiz, so quizzes may be unlike each other in length. ReadnQuiz seems to see itself more as a vehicle for home-schooling than for use in schools. It costs $30 per year for one student, $95 for one class for one year, and $250 per school plus $1.50 per student using the system (2021 prices). Thus, it is more expensive than AR and RC for small numbers of students but less expensive for larger numbers.

While Book Adventure is more expensive and ReadnQuiz cheaper for large numbers of students than AR and RC (unless it is used for home-schooling) (see Chapter 13), the third program seems to be free. Book It is operated for pre-K through sixth grade by Pizza Hut (https://www.bookitprogram.com), is targeted on home-schoolers as well as schools, and essentially offers extrinsic rewards of pizza for completing reading tasks. It “recommends” books and offers complementary activities. Book It seems to be the most easily criticized program of them all, but paradoxically it does not seem to have been criticized.

However, when these three companies were contacted to request their cooperation in producing this book, they declined to participate. Actually, they just didn’t reply. There is no published literature on any of the programs. Consequently, I was unable to include them any further in this book.

In addition, there were two somewhat similar programs which have since disappeared: BookSharp and That’s a Fact, Jack!

Research in the Book

A large number of research reports have been produced by the manufacturers of these programs, but more importantly, there have been many independent research studies, particularly of AR. The present book gathers together all the independent research studies in peer-reviewed journals on all versions of these programs while only listing in an Appendix and not otherwise mentioning studies done by the manufacturers, which might be considered biased. Books, chapters and doctoral dissertations and master’s theses were also excluded, as they had not been subject to peer review and were very diverse in quality. The book also adds new analyses of many of the areas under consideration, using fresh data supplied by the relevant companies but analyzed independently.

Clearly (given the numbers of schools involved), these programs are very popular with teachers, but there is a fair amount of misinformation in circulation about them, often promoted by those who have no experience of using them. This book should do much to make information and evidence, as opposed to opinions, more readily available. Its purpose is to give teachers easily accessible information that should enable them to fine-tune their understanding and implement the program more successfully in their classrooms. It also aims to provide a coherent research foundation not only for teachers but also for school managers and educational researchers.

Search Methods

The literature search began by asking all product manufacturers for their descriptions of their programs and how they worked. They were also asked to supply lists of both externally authored research reports of which they were aware and in-house research reports. Only Renaissance Learning and HMH replied. Five relevant research databases (ERIC, JSTOR, Scopus, Web of Science and Google Scholar) were then searched using the terms “Accelerated Reader” or “Reading Counts”. The first term generated many hits, the second very few. However, Read 180 (in which RC is now embedded) generated many hits.

There were over a hundred master’s theses and doctoral dissertations on AR and a few on RC. However, the quality of these was so various that it was felt safer to exclude them from the literature review. Of course, peer-reviewed journal articles are also various in quality but less so. Chapters in books and books were also excluded on the grounds that they had not been peer-reviewed, as were unpublished research reports (although research reports were included if they had appeared in peer-reviewed journals or were from other authoritative source such as the US government or a university). Studies of either AR or RC which combined the programs with some other intervention and did not disentangle the effects of the various aspects were also excluded. The full text of each retrieved item was then read to ensure that it met the criteria f...

Table of contents

• Cover Page
• Half Title page
• Title Page
• Copyright Page
• Dedication
• Contents
• Acknowledgements
• Executive Summary
• 1 Introduction
• 2 What is Computer Assessment of Reading Comprehension of Real Books?
• 3 Disentangling Opinion and Fact
• 4 So, Does It Work?
• 5 How to Implement for Increased Student Success
• 6 How Have Schools Done with Implementation?
• 7 Increasing Student Achievement in Elementary and High School
• 8 Improving Outcomes for Low- and High-Ability Readers
• 9 Using Favorite Books as a Major Motivator
• 10 Encouraging Careful Reading of Non-Fiction Books
• 11 Working Effectively with Spanish-Speaking, Disadvantaged, Disabled or EFL Students
• 12 Electronic Reading and Out-of-School Reading
• 13 What Bang for Your Buck? The Importance of Cost-Effectiveness
• 14 Summary and Conclusions
• Appendix A: Implementation Quality Checklist
• Appendix B: Reports Authored by the Manufacturers
• Appendix C: How to Use This Book in a Professional Development or Study Group
• Subject Index
• Author Index