Measuring Noncognitive Variables

Improving Admissions, Success and Retention for Underrepresented Students

William Sedlacek

• Publisher: Stylus Publishing
• Year: 2017
• ISBN:
  9781620362587
• Topic: Education
• Subtopic: Research in Education

Co-published in association with Big Picture Learning. Measuring Noncognitive Variables: Improving Admissions, Success, and Retention for Underrepresented Students is written for admissions professionals, counselors, faculty and advisers who admit, teach, or work with students during the admissions process and post-enrollment period. It brings together theory, research and practice related to noncognitive variables in a practical way by using assessment methods provided at no cost. Noncognitive variables have been shown to correlate with the academic success of students of all races, cultures, and backgrounds. Noncognitive variables include personal and social dimensions, adjustment, motivation, and student perceptions, rather than the traditional verbal and quantitative areas (often called cognitive) typically measured by standardized tests.Key Features include:
* Models that raise concepts related to innovation, diversity and racism in proactive ways
* Examples of admission and post-enrollment applications that show how schools and programs can use noncognitive variables in a variety of ways
* Additional examples from foundations, professional associations, and K-12 programs
* An overview of the limitations of traditional assessment methods such as admission tests, grades, and courses takenEducation professionals involved in the admissions process will find this guide effectively informs their practice. This guide is also appropriate as a textbook in a range of courses offered in Higher Education and Student Affairs Masters and PhD programs.

1

THE INNOVATION PROCESS

Sextus, you ask how to fight an idea? Well I’ll tell you how . . . with another idea!

—Messala, Ben-Hur (Zimbalist & Wyler, 1959)

Five-Stage Proactive Innovation Model

There are many models to explain the process of an innovation having an impact in an area. The process of diffusion of an innovation has been studied by a number of researchers and theorists (Dooley, 1999; Rogers, 2003; Sahin, 2006; Sherry, 1997; Stuart, 2000). The Humanitarian Innovation Fund (www.elrha.org/hif/innovation-resource-hub/innovation-explained/humanitarian-innovation-process) developed a five-stage proactive innovation model that I will use to help explain the development and implementation of the noncognitive variable model that is the focus of this book.

Stage 1: Recognition of a Specific Problem

In 1970, I was a young assistant professor and director of research at the Counseling Center at the University of Maryland. Student unrest about many issues was rampant. In a study my students and I did, we found that fully half of the student body at the university had participated in a riot or demonstration during the 1971–1972 school year (Collins & Sedlacek, 1973; Kimball & Sedlacek, 1971; Schmidt & Sedlacek, 1971). Among the issues raised was the unfairness of admissions tests for Black students. The university used the ACT test and later switched to the SAT. I was the faculty adviser to a group called the Campus Coalition Against Racism, and on the basis of my training, I went to the literature to find alternatives to the prevailing tests that seemed more fair and that I could present to the campus administration. I was surprised to learn that there were no obvious methods that had been validated for university students (Sedlacek & Brooks, 1976).

We demonstrated the problem of low percentages of Black students and eventually other minority students enrolling in higher education in a series of national surveys initially sponsored by the American College Personnel Association (Brooks & Sedlacek, 1972; Sedlacek, 1987; Sedlacek & Brooks, 1970; Sedlacek, Brooks, & Horowitz, 1972; Sedlacek, Brooks, & Mindus, 1973a; Sedlacek, Lewis, & Brooks, 1974; Sedlacek, Merritt, & Brooks, 1975; Sedlacek & Pelham, 1976; Sedlacek & Webster, 1978). These studies showed that first-year enrollments of minorities in higher education were small and did not change a great deal during that period. Low minority enrollment (about 5% of the student population) was a particular issue at large universities, like the University of Maryland.

Stage 2: Invention of a Creative Solution or Novel Idea That Helps Address a Problem or Seize an Opportunity

I decided to develop my own measure based on the best untested ideas from the literature on human abilities, values, and performance. Because the psychology literature tended to label such attributes as cognitive, I settled on the term noncognitive to describe the measures I was developing. Glenwood Brooks, one of my best students, worked closely with me on the project.

I wanted to use an inductive method. I aimed to develop measures, predict student success using grades and retention, remeasure with changes, analyze results, measure again, and build constructs that would explain the results.

Stage 3: Development of an Innovation by Creating Practical, Actionable Plans and Guidelines

In a number of studies over many years, we were able to demonstrate the validity of a series of noncognitive variables useful in predicting the success of students of color; international students; LGBTQ students; and women in higher education at a variety of institutions and programs. We used the term nontraditional to describe this diverse group. Current measures did not predict their success in higher education as well as they did for White men (Ancis & Sedlacek, 1997; Bandalos & Sedlacek, 1989; Boyer & Sedlacek, 1988; DiCesare, Sedlacek, & Brooks, 1972; Farver, Sedlacek, & Brooks, 1975; Fuertes & Sedlacek, 1994, 1995; Fuertes, Sedlacek, & Liu, 1994; Pfeifer & Sedlacek, 1971, 1974; Sedlacek, 1972, 1974, 1977a, 1989, 1991, 1996, 1997, 1998a, 1998b, 2003a, 2003b, 2004b, 2004c, 2010, 2011; Sed-lacek & Adams-Gaston, 1992; Sedlacek & Prieto, 1990; Sedlacek & Sheu, 2004a, 2004b, 2005, 2008; Tracey & Sedlacek, 1981, 1984a, 1984b, 1985, 1987, 1988, 1989; Webb et al., 1997; T. J. White & Sedlacek, 1986). Much of this research was summarized and discussed in Sedlacek (2004b), and I present it in detail in later chapters of this book.

Edward St. John (2013) stressed the importance of using research and assessment in developing actionable plans to achieve social justice in higher education. He discussed four types of action: institutionalist action, when professionals are treated as a group; closed strategic action, when initiatives are not open to all groups; open strategic action, when actions can be tested openly; and communicative action, when partnerships are formed among diverse groups. Sedlacek (2007) discussed an approach to social change using a research base, defining and focusing on audiences that would take action on that research, and becoming a critical source of information for those audiences (also see Sedlacek & Brooks, 1972, 1973).

Stage 4: Implementation of an Innovation to Produce Real Examples of Changed Practice, Testing the Innovation to See How It Compares to Existing Solutions

Along with publishing research demonstrating the validity of noncognitive variables in predicting student success at different institutions, workshops on implementing the variables in educational settings were developed (D’Costa et al., 1974, 1975; Prieto et al., 1978; Prieto, Quinones, Elliott, Goldner, & Sedlacek, 1986; Westbrook & Sedlacek, 1988). These workshops focused on an eight-stage model of eliminating racism through the application of noncognitive variables in admissions and postenrollment programs and the development of a variety of measures to support the model (Garcia et al., 2001; McTighe Musil et al., 1999; Sedlacek, 1988, 1993, 1995b, 2003a, 2008, 2013, in press). The basic model is presented and discussed in Sedlacek and Brooks (1976), and I cover it in detail in chapter 3.

Stage 5: Diffusion of Successful Innovations: Taking Them to Scale and Leading to Wider Adoption Outside the Original Setting

A key concept in the diffusion of the noncognitive model was to make it available at no cost. Also, the model is not copyrighted, so it can be adjusted, revised, or partially employed. It is not one size fits all. The logic here was to reduce the reasons why people would not try it in a program or research study or for their own benefit in some way. It also allows individuals to take credit for the implementation themselves and receive the reinforcement they need in their system. This reasoning has been employed recently in the development of electric vehicle technology:

Yesterday, there was a wall of Tesla patents in the lobby of our Palo Alto headquarters. That is no longer the case. They have been removed, in the spirit of the open source movement, for the advancement of electric vehicle technology. We believe that Tesla, other companies making electric cars, and the world would all benefit from a common, rapidly evolving technology platform. Tesla Motors was created to accelerate the advent of sustainable transport. If we clear a path to the creation of compelling electric vehicles, but then lay intellectual property landmines behind us to inhibit others, we are acting in a manner contrary to that goal. Tesla will not initiate patent lawsuits against anyone who, in good faith, wants to use our technology. (Musk, 2014)

Here the concept of audience is critical (Sedlacek, 2007). What would motivate someone to use an innovation, in this case the noncognitive model? An admissions director’s perspective would likely differ from that of a parent of a student of color, a university teacher, a student affairs programmer, an official of a professional organization, a college president, an administrator at a university outside the United States, a multicultural office administrator, a high school student doing a term paper, a White student applying to college, a women’s studies department head, a foundation executive, a faculty researcher, a high school counselor, or a university student doing a thesis. Multiple reinforcements for multiple audiences was the principle employed in diffusing the innovation. I will provide examples for these and other audiences throughout this book.

Adopting the Innovation

Rogers (2003) developed a widely employed model in his book Diffusion of Innovations. He discussed five characteristics of an innovation that facilitate adoption of that innovation. The diffusion of the noncognitive variable model will be discussed in terms of Rogers’s characteristics.

Characteristic 1: Relative Advantage: How Is the Innovation Better Than That Currently Employed?

There were several advantages of the noncognitive variable model. Most important was that the model predicted the success of students of color and other nontraditional applicants better than the typical measures of grades and test scores (Sedlacek, 2004b, 2011, in press). Although the model was available at no cost, there are costs associated with adding a new system, and there are probable scoring costs. However, the non-cognitive model would likely cost less for applicants and institutions than traditional measures. Oregon State University (OSU) developed a system of using noncognitive variables in admissions without an increase in the budget (Sandlin, 2008). I will discuss the OSU program further in chapter 8.

Another advantage in employing noncognitive variables is their use in retention, teaching, advising, counseling, and student service programs (Helm, Sedlacek, & Prieto, 1998a; Liang & Sedlacek, 2003a, 2003b; Longerbeam, Sedlacek, & Alatorre, 2004; Noonan, Sedlacek, & Veerasamy, 2005; Roper & Sedlacek, 1988; Schlosser & Sedlacek, 2003; Sedlacek, 1994a; Sedlacek & Brooks, 1981; Sedlacek & Sheu, 2004a, 2004b, 2005; Sheu & Sedlacek, 2004; Warren & Hale, 2016). Grades and test scores are not designed to be helpful in this way. Noncognitive variables are intended to be useful in retaining students and assessing changes in their development and learning.

Characteristic 2: Compatibility: Does the Innovation Fit With the Procedures and Style Currently Used?

As noted previously, the noncognitive model can be used with any of the commonly employed methods in admissions or postenrollment programs such as teaching, advising, counseling, or other student services. Also, the noncognitive variable methodology can be modified to fit the situation. For example, some schools use a few of the variables and not others. Other programs combine or alter some of the variables to fit their needs. I will cover throughout the book examples of how the model has been employed in a variety of ways.

Characteristic 3: Simplicity or Complexity: Is the Innovation Easy or Difficult to Use?

It is relatively easy to get grades and test scores from applicants or testing organizations. However, the noncognitive model provides more information about students as they enter the institution or participate in a program. This makes it much easier to plan programs and work with students on their needs in a more direct way. A common dilemma at colleges and universities is that it is difficult to get student development information after students enroll. A student may be having difficulties before postenrollment data are collected. At most schools, data collection is decentralized, and one office or department may not coordinate with another. To avoid this problem, OSU provided a profile of student scores on noncognitive variables to all faculty and student service personnel (Sandlin, 2008).

Characteristic 4: Trialability: Can the Innovation Be Empirically Tested?

The noncognitive model has been empirically tested in many institutions and programs, using a variety of methodologies including multiple-choice items, short answer questions, essays, interviews, and portfolios (Sedlacek, 1997, 1998b, 2004b, 2004c). I will discuss methods employed by large universities, liberal arts colleges, community colleges, scholarship programs, professional schools, multicultural offices, and others throughout this book.

Characteristic 5: Observability: Are the Results of the Innovation Visible to All?

One of the characteristics of many admissions offices is the lack of candor in communicating requirements for admission. Often, admissions materials suggest seeking the well-rounded student, holistic assessment, students seeking a unique experience, and so on, with little detail on what is actually considered. As I watch a college football game on television, I am struck by the lack of differentiation among institutions in their halftime ads. They all have great students and great faculty and promise a fun time. I suggest promoting the institution by discussing how a student will develop on the noncognitive dimensions while enrolled. Throughout this book I will provide examples of schools and programs that have done this.

2

TRADITIONAL ADMISSIONS MEASURES

Tradition is the prison where change is detained.

—Israelmore Ayivor (n.d.)

Attributes That Determine Students’ Success in Higher Education

Should a student come to higher education fully developed, or do we wish to select based on dimensions on which a student will improve through experience at an institution? There has been a recent focus on “college readiness,” suggesting information separate from the wide range of attributes a student will need once enrolled (Conley, 2005). Although readiness for college includes taking the appropriate courses, getting good grades, and scoring well on admissions tests, there is evidence that many other attributes determine whether most students will succeed in higher education (Sedlacek, 2011).

Courses

Sedlacek (2011) suggested that although students continue to need courses in math, English, foreign languages, and so on, there has been a tendency among educators and college admissions staff to feel that more is better. The reasoning goes that if we would just require more courses in certain areas (e.g., math), students would be better prepared. However, the law of diminishing marginal utility from economics becomes relevant at some point (Diamond & Rothschild, 1989). The logic behind the law is that beyond a certain level, there is little or no increase in the value of more units in a given area. Thus, at some point, the number of courses in a subject or field may no longer be relevant as a predictor. We may have reached an asymptote, and the variable has become a constant.

For example, Sawyer (2008) studied 245,175 students from 9,507 high schools who took the EXPLORE (8th grade), PLAN (10th...

Table of contents

• Cover
• Title Page
• Copyright page
• Table of Contents
• Foreword
• Preface
• Acknowledgments
• Introduction
• 1. The Innovation Process
• 2. Traditional Admissions Measures
• 3. Noncognitive Measures
• 4. Self-Concept and Realistic Self-Appraisal
• 5. Understands and Knows how to Navigate The System and Racism
• 6. Long-Term Goals, Strong Support Person, Leadership, Community, and Nontraditional Learning
• 7. Additional Measures Of Diversity
• 8. The Waves Of Change Find Many Shores
• 9. The Future
• Exhibit 1. Description of Noncognitive Variables With Reliability Estimates of Scale Scores
• Appendix A1. Scoring System for Noncognitive Variables on Interviews and Essays
• Appendix A2. Example Cases for Training Raters in Evaluating Admissions or Financial Aid Applications
• Appendix B1. Noncognitive Items That Can Be Employed in Interview, Short-Answer, Essay, or Application Review Formats
• Appendix B2. Noncognitive Items in Likert (Agree–Disagree) Formats
• Appendix B3. Noncognitive Items in Multiple-Choice Formats
• Appendix C. Universal Diverse Orientation (UDO) Scale–Short Form
• Appendix D. Example Behaviors for Evaluating University Police Officers
• Appendix E. Principles of Interviewing for Noncognitive Variable Diagnosis
• References
• Index
• Also available from Stylus