1 | Methods for Studying Goals, Goal Structures, and Patterns of Adaptive Learning |
Eric M. Anderman
University of Kentucky
Carol Midgley
University of Michigan
The Patterns of Adaptive Learning Study is a large-scale, longitudinal study using goal orientation theory as the framework for examining the emotional, social, and academic well-being of young adolescents. This chapter provides information about the methods used in the study. Whereas additional detail about the methods used for specific studies is provided in subsequent chapters, this chapter is designed to give a comprehensive overview. In addition, we briefly compare our measurement of goals and goal structures on surveys to those used by others using survey methods.
The Patterns of Adaptive Learning Study included a large survey component, as well as classroom observations and interviews with students and parents. We begin with an overview of the research design; the characteristics of the sample; and the recruitment of the districts, schools, principals, teachers, and students. We then describe the survey component, including the collection of data from students, teachers, principals, and school records. We follow that with information about the development of measures; the coding, processing, and preliminary analysis of the data; and the validation of the scales assessing personal goals and perceptions of the goal structure in the classroom. Because chapter 9 (this volume) provides extensive information about the methods used in the observation component, we do not provide additional information here. Similarly, chapters 6 and chapter 7 (this volume) provide information about the methods used in two studies that included interviews with students, so only a brief overview is included in this chapter.
RESEARCH DESIGN
The research design reflects our special interest in examining the relation between studentsā personal goals and perceptions of the goal structure in the learning environment from the last year of elementary school through the first year of high school. We were particularly interested in changes associated with the transition to middle school and the transition to high school. Our original plan was to examine both within-year and across-year changes, collecting data from students in the late fall and early spring each year. We were able to do this in the fifth grade in elementary school (1994ā1995) and in the sixth grade in middle school (1995ā1996). However, we became aware during the second year in the middle schools (seventh grade) that both the students and teachers were finding that scheduling two waves of data collection each year was burdensome. Thus we modified the design and thereafter collected data from students only in the spring of the seventh-, eighth-, and ninth-grade years (1997, 1998, and 1999). In the eighth-grade year (1998), we collected data from a subsample of students in three of the middle schools in the fall (see Table 1.1 for a description of the waves and years that surveys were given to students, teachers, and principals).
RECRUITMENT OF DISTRICTS AND SCHOOLS
Because the nature of the study required many visits to schools to collect survey data, conduct observations, interview participants, and gather information from student records, we first identified districts that were within a 2-hour drive from the University of Michigan. From that pool of potential districts, we identified those that included some racial and economic diversity. This was important for several reasons. As noted recently, educational psychology as a field has been particularly negligent in terms of addressing socioeconomic issues and including socioeconomic variables (Murdock, 2000). In addition, studies using a goal theory framework have been criticized for not including African American participants (Graham, 1994). Finally, because we were interested in examining the effects of the transition to middle and high school, we selected districts in which students made those transitions in the sixth grade (middle school) and ninth grade (high school), because this is the most common pattern. We then contacted the district superintendents in the pool of potential districts, briefly describing the goals of the study, our commitment to providing them with helpful information, and our record regarding the conduct of high-quality research. In some of the districts the superintendent indicated that they either were heavily involved with other research projects, or were undergoing a particularly difficult situation, such as a potential teacher strike. Most of the superintendents agreed to look over a brief written description of the project that clearly stated what their commitment would be, and what they would gain from participation. Packets with this information were sent to the superintendents, and in follow-up phone calls, four districts were selected that met the criteria listed previously. In one district we were required to submit a proposal to a research review committee. This was a lengthy process, but the proposal was approved. After districts agreed to participate, a graduate student was appointed to serve as a liaison between the research team and the district (with a different graduate student assigned to each district).
TABLE 1.1
The Patterns of Adaptive Learning Study: Waves and Years
Fall 1994 | Student Survey, Wave 1 | Fifth Grade, Elementary |
Spring 1995 | Student Survey, Wave 2 | Fifth Grade, Elementary |
Fall 1994 | Teacher Survey Individual, Wave 1 | Fifth Grade, Elementary |
Spring 1995 | Teacher Survey Individual, Wave 2 | Fifth Grade, Elementary |
1994ā1995 | Teacher Survey General, Year 1 | All Elementary Teachers |
1994ā1995 | Principal Survey, Year 1 | All Elementary Principals |
Fall 1995 | Student Survey, Wave 3 | Sixth Grade, Middle School |
Spring 1996 | Student Survey, Wave 4 | Sixth Grade, Middle School |
Fall 1995 | Teacher Individual Survey, Wave 3 | Sixth Grade, Middle School Math Teachers |
Spring 1996 | Teacher Survey Individual, Wave 4 | Sixth Grade, Middle School Math Teachers |
1995ā1996 | Teacher Survey General, Year 2 | All Middle School Teachers |
1995ā1996 | Principal Survey, Year 2 | All Middle School Principals |
Spring 1997 | Student Survey, Wave 5 | Seventh Grade, Middle School |
1996ā1997 | Teacher Survey General Year 3 | Seventh Grade, Middle School Math Teachers |
Fall 1997 | Student Survey, Wave 6 | Eighth Grade, Middle School (subsample) |
Spring 1998 | Student Survey, Wave 7 | Eighth Grade, Middle School |
1997ā1998 | Teacher Survey General, Year 4 | Eighth Grade, Middle School Math Teachers |
Spring 1998 | Student Survey, Wave 8 | Ninth Grade, High School |
1998ā1999 | Teacher Survey General, Year 5 | All High School Teachers |
1998ā1999 | Principal Survey | All High School Principals |
All of the districts were located in southeastern Michigan. Some of the districts were quite close to the university, whereas others were located at greater distances. Three of the districts could best be described as working class, whereas the other one was predominantly middle class. In three of the four districts, over 50% of the students were African American. Those three districts had the largest numbers of classrooms in the study, as compared with the remaining district, which was over 90% Euro-American. The districts varied in size, from the smallest district, which contained 3 elementary schools, 1 middle school, and 1 high school, to the largest district, which contained 24 elementary schools, 4 middle schools, and 2 high schools. (See Table 1.2 for a description of the four participating districts.)
The superintendents helped us make arrangements to meet with the principals and fifth-grade teachers in the elementary schools. We described the study to them and answered questions. With only a few exceptions, all elementary schools and fifth-grade teachers agreed to participate. In the largest district, the Director of Evaluation, Testing, and Research selected a sample of schools that reflected diversity in student achievement and ethnicity, and school size. We also met with middle school administrators to discuss the study, because their support would be important during the second year. We did not meet with representatives from the high schools, because the original grant that funded the study covered only the elementary and middle school years. We later received funding to follow the sample into the high schools. In all, 21 elementary schools, 10 middle schools, and 5 high schools participated in the study.
RECRUITMENT OF THE STUDENT SAMPLE
Recruitment of a representative student sample was a high priority. In each district the graduate student research assistant (liaison) worked to develop a positive rapport with the participating fifth-grade teachers. The research assistant consulted with each teacher regarding his or her preferences for the distribution and collection of permission forms. The research assistants were present in the classrooms when permission forms were distributed in order to encourage students to return them to school. In some cases the assistants visited a classroom several times to help with recruitment. Pencils with a University of Michigan logo were given to students who returned their permission forms. Students received the pencils for returning the forms regardless of whether they had or had not received parental permission to participate.
TABLE 1.2
The Patterns of Adaptive Learning Study: Description of Districts
District | Number of Schools in the District | Number of Participating Schools | SES | Ethnicity |
District 1 | 3 elementary 1 middle 1 high school | 3 elementary 1 middle school | Working class | Over 90% Euro-American |
District 2 | 6 elementary 2 middle 1 high school | 5 elementary 2 middle 1 high school | Working class | About 60% Euro-American, 35% African American |
District 3 | 10 elementary 3 middle 2 high schools | 6 elementary 3 middle 2 high schools | Middle class | About 60% African American 35% Euro-American |
District 4 | 24 elementary 4 middle 2 high schools | 7 elementary 4 middle 2 high schools | Working class | About 55% African American, 30% Euro-American 15% Hispanic |
Note. One elementary school in District 2 declined to participate in the study; in District 3, we limited participation to six of the elementary schools; in District 4, a sample of seven elementary schools was selected for us.
Permission forms were given to 1,091 students; 899 were given permission by parents to participate, 83 were not granted permission, and 109 students failed to return their permission forms. The final participation rate was 82%. When the first wave of surveys was administered, 856 students completed the surveys, because some students had moved and others were absent over an extended period of time. Because there had been an economic downturn in some of the districts, we realized that families might be more mobile than would be the case in better economic times. Consequently, we recruited more students than we originally had proposed, anticipating greater than expected attrition in the sample.
Relocating the Participating Students in Middle School
Students moved from 21 elementary schools to 10 middle schools. The relocation of students after they had made the transition from elementary to middle school was a complex process because the students in some elementary schools were assigned to several different middle schools. Early in the fall of 1995, we contacted middle school teachers and principals. In collaboration with them, we were able to determine in which middle schools our participants were enrolled, and arranged for them to complete surveys. The third wave of surveys was administered during November and early December of 1995, and the fourth wave of surveys was administered during the spring of 1996. Attrition was as expected in three of the four districts, but was somewhat higher in one of the districts. Administrators in that district told us that they always experienced a significant loss of students after the fifth grade because some parents chose to send their children to private and parochial schools at that juncture.
As mentioned earlier, the original proposal for funding included examining the transition from elementary to middle school. When our proposal to obtain fu...