1
Introduction and Methodology
Diane August and Timothy Shanahan
CHARGE TO THE PANEL
In 2002, the Institute for Education Sciences of the U.S. Department of Education formed the National Literacy Panel on Language-Minority Children and Youth. The formal charge to the Panel was to identify, assess, and synthesize research on the education of language-minority children and youth with respect to their attainment of literacy, and to produce a comprehensive report evaluating and synthesizing this literature. The Panelās review represents the most comprehensive review of the research to date on the development of literacy in second-language learners. This report provides a summary of the work of the Panel.
Key terms germane to the Panelās charge include literacy skills, oral language proficiency, societal/national language, language minority, English language learners, limited English proficient, and bilingual students and programs. Literacy skills are defined in this review as including pre-reading skills, such as concepts of print and alphabetic knowledge; word-level skills, including decoding, word reading, pseudoword reading, and spelling; and text-level skills, including fluency, reading comprehension, and writing skills. For purposes of this review, oral language proficiency denotes knowledge or use of specific aspects of oral language, including phonology, vocabulary, morphology, grammar, and discourse domains; it encompasses skills in both comprehension and expression. We also include studies of phonological processes (phonological recoding, phonological memory, and phonological awareness) because it has been hypothesized that these processes mediate the development of written forms of language (Adams, 1990; Ehri, 1998; Metsala & Walley, 1998; Scarborough, 2001). A societal language is one, often one of several, of the languages used in a country for public discourse, whether official (that is, recognized by law) or not, while a national language is considered to be the chief language of a country.
There are many labels for the students and programs under consideration in this report. The most commonly used term, language minority, refers to individuals from homes where a language other than a national language is actively used, who therefore have had an opportunity to develop some level of proficiency in a language other than a national language. A language-minority student may be of limited second-language proficiency, bilingual, or essentially monolingual in the second language (August and Hakuta, 1997). Second-language learners come from language backgrounds other than a national language and whose second language proficiency is not yet developed to the point where they can profit fully from instruction solely in the second language. In instances where the students are acquiring English as a second language, they are referred to as English-language learners. The term limited English proficient (LEP) may be used, however, when we are quoting another source or citing legal requirements. Appendix 1.A includes a list of standard terms used in the report.
Two other terms appear frequently in this volume. The first is bilingual students or bilingual education programs. Some education programs for second-language learners use the studentsā native language as the students acquire the second language. Thus, the term bilingual is often used to refer to programs that use studentsā first languages as well as a national language for instructional purposes. We also use the term bilingual to refer to individuals who have developed proficiency in more than one language.
PROCEDURES USED TO CONDUCT THE REVIEW
Panel Staff
The Panel included scholars with deep expertise in critical components of literacy, language learning, or research methodology, and an effort was made to include language-minority researchers. In addition, five panelists have important cross-cutting expertise: two are methodologists, two are experts in learning disabilities, and one is an expert in the assessment of students from culturally and linguistically diverse backgrounds.
To address the research questions, the Panel was divided into five subcommittees, each of which oversaw syntheses of research relevant to particular issues. The Panel was served by a principal investigator, Diane August, who managed the project; a chairperson, Timothy Shanahan, who helped guide the Panelās work; and two methodologistsā David Francis, who provided expertise in quantitative methodology, and Frederick Erickson, who provided guidance in qualitative methodology. Catherine Snow and Donna Christian served as senior advisors to the Panel. In addition, the Panel was served by two senior research associates who were instrumental in preparing several of the chapters: Nonie Lesaux (Chapters 3 and 6) and Cheryl Dressler (Chapter 4). A list of the subcommittees can be found in Appendix 1.B. Biographical sketches of the Panel members and other contributors can be found at the end of this volume. In addition to the reviews by Christian, Snow, Jill Fitzgerald, and Russell Gersten solicited by the Panel, this volume also reflects the input of anonymous reviewers solicited by the funder. These reviewers provided detailed commentary on multiple drafts, and their contributions were an instrumental part of the process leading to this volume.
Identification of Research Questions
The Panel identified five domains to investigate: the development of literacy in language-minority children and youth (Chapter 3), cross-linguistic and cross-modal relationships (Chapter 4), sociocultural contexts and literacy development (Chapter 5), instruction and professional development (Chapter 6), and student assessment (Chapter 7).
Source of Publications and Current Database
On the basis of the research questions, we set review parameters to ensure as complete and unbiased a search of the research literature as possible. For the most part, the review focused on language-minority children ages 3 to 18 acquiring literacy in a national language. However, to answer some questions, we also reviewed research on the acquisition of literacy in a foreign language, if the foreign language were English, and studied the acquisition of French by English speakers in Canada. The review incorporated only research published in peer-reviewed journals dating back to 1980. However, to be consistent with prior reviews on one topic (Chapter 6), we accepted studies of language of instruction that pre-dated 1980. For some chapters, dissertations and technical reports were used if the research in peer-reviewed journals was not sufficient to answer the research questions. Book chapters and literature reviews were used to provide context for the findings presented; these are used in the discussions, but not in establishing the research findings. Appendix 1.C lists the acceptance criteria used across the chapters in this volume.
Rigorous methodological standards were applied in both the selection and analysis of research studies. Studies had to analyze data; no thought pieces or articles detailing personal experiences were included. If language-minority students did not make up at least 50% of the sample, outcome data had to be disaggregated for those students. For experiments or quasi-experiments, the study had to include a control or comparison group and had to use either random assignment to conditions or pre-testing or other matching criteria to ensure initial similarity of the groups prior to treatment; groups had to have more than four subjects to allow for the appropriate statistical analyses (Maxwell and Delaney, 2003). Each subcommittee adopted additional standards to ensure sound findings and these are reported in the individual chapters that follow.
Approximately 1,800 articles were initially identified in the literature search, but these were reduced as panelists examined the studies more carefully. The database now consists of 970 studies (293 of which were used for this report because they are relevant to the research questions and meet our methodological criteria). Most studies were conducted in the United States, followed by the United Kingdom, Canada, and Australia. However, studies from the Netherlands, Finland, and Israel appear as well. Some studies are relevant to more than one question and, thus, are cited in multiple locations in the report.
Search Procedures
To identify studies for use in this review, we conducted seven literature searches. These entailed extensive searches of various electronic databases and hand searches of particular journals. The purpose of these carefully documented searches (for more details see the longer version of this report, August and Shanahan, 2006) was to ensure the most comprehensive, unbiased search possible for relevant studies. The intent was to use search procedures that could be replicated in future reviews.
Coding Instrument
A coding instrument was developed for use by the Panel. The use of this instrument ensured that each study in the database met the selection criteria, and that no studies that met the criteria were excluded from the report. The use of a common coding instrument ensured that the various articles were summarized in the same ways for all studies. A compact disk containing the database is included as part of the full Panel report published in 2006.
Public Advice and Input
To gain public advice and input from educators, community members, and researchers who were not on the Panel, two sets of outreach meetings were held. The first set of meetings, in Washington, DC and Los Angeles in 2002, helped determine what the research, policy, and practitioner communities considered important research questions. The second set of meetings was held to obtain feedback on a draft of the final report. One meeting was held at the National Reading Conferenceās annual meeting in Scottsdale, Arizona in 2003 and the other at the International Reading Association meeting in Reno, Nevada in 2004.
NATURE OF THE REVIEW
Types of Research Evidence and Breadth of Research Methods
The Panel synthesized a complex body of empirical data drawn from many research methodologies. As has been indicated by the Committee on Scientific Principles for Education Research, āmultiple methods, applied over time and tied to evidentiary standards, are essential to establishing a base of scientific knowledgeā (Shavelson & Towne, 2002, p. 2). The committee goes on to indicate that one of the hallmarks of scientific inquiry is the application of āappropriateā methodology to the questions being asked.
Accordingly, the Panel relied on controlled experiments and quasi-experimental designs to evaluate the effectiveness of instructional approaches. However, the Panelās charge also required review of the research literature addressing questions about the nature of literacy development, relationships among various language-learning abilities, and the status of particular approaches in the education of language-minority students. These questions are more appropriately answered through descriptive, ethnographic, and correlational studies and for this reason studies that used these methods were included in the Panelās report.
Data Analysis
When at least five group comparison studies addressed the same hypothesis relevant to a research question, meta-analytic techniques were used. For questions for which quantitative techniques were not appropriate, a systematic interpretive procedure (Fitzgerald, 1995a, 1995b; Glaser, 1978) was used to summarize findings across studies. For each research question, studies were categorized by major themes. Studies in each group were reread and classified with regard to similarities, differences, and results to determine cross-cutting themes, as well as to methodological strengths and weaknesses.
SCOPE OF THE VOLUME
This volume is organized partly around the traditional distinction between basic and applied research, but is also structured to reflect specific areas of concern for educational policymakers. Chapters 3 and 4 address basic research questions about bilingualism, second-language acquisition, and relationships between first- and second-language oral proficiency and literacy. Chapter 5, on sociocultural context and literacy development, addresses both basic research about the relationship between sociocultural variables and student outcomes and more applied research related to the influence of sociocultural variables on the contexts in which students acquire second-language literacy. Chapters 6 and 7 are organized around more practical issues: program evaluations that explore the influence of native-language instruction on second-language literacy, effective instruction, schooling, professional developm...