Statistical Methods for Validation of Assessment Scale Data in Counseling and Related Fields

Dimiter M. Dimitrov

• Publisher: American Counseling Association
• Year: 2014
• ISBN:
  9781119019282
• Edition: 1
• Topic: Psychology
• Subtopic: Psychotherapy
• Index: Psychology

"Dr. Dimitrov has constructed a masterpiece—a classic resource that should adorn the shelf of every counseling researcher and graduate student serious about the construction and validation of high quality research instruments.

—Bradley T. Erford, PhD

Loyola University Maryland

Past President, American Counseling Association

"This book offers a comprehensive treatment of the statistical models and methods needed to properly examine the psychometric properties of assessment scale data. It is certain to become a definitive reference for both novice and experienced researchers alike."

—George A. Marcoulides, PhD

University of California, Riverside

This instructive book presents statistical methods and procedures for the validation of assessment scale data used in counseling, psychology, education, and related fields. In Part I, measurement scales, reliability, and the unified construct-based model of validity are discussed, along with key steps in instrument development. Part II describes factor analyses in construct validation, including exploratory factor analysis, confirmatory factor analysis, and models of multitrait-multimethod data analysis. Traditional and Rasch-based analyses of binary and rating scales are examined in Part III.

Dr. Dimitrov offers students, researchers, and clinicians step-by-step guidance on contemporary methodological principles, statistical methods, and psychometric procedures that are useful in the development or validation of assessment scale data. Numerous examples, tables, and figures provided throughout the text illustrate the underlying principles of measurement in a clear and concise manner for practical application.

*Requests for digital versions from ACA can be found on www.wiley.com.
*To purchase print copies, please visit the ACA website here.
*Reproduction requests for material from books published by ACA should be directed [email protected]

Part I
Scales, Reliability, and Validity

Chapter 1
Variables and Measurement Scales

The development of instruments for assessment in counseling, psychology, education, and other areas must be addressed within the framework of a more general goal of providing theoretical explanations of behaviors and phenomena in these areas. As Kerlinger (1986) noted, “a theory is a set of interrelated constructs (concepts), definitions, and propositions that present a systematic view of phenomena by specifying relations among variables, with the purpose of explaining and predicting the phenomena” (p. 9). To reach valid interpretations and conclusions through testing hypotheses, researchers must collect accurate measures of the variables involved in the hypothesized relations. Therefore, it is important that researchers understand well the nature of the study variables and the properties of their measurement scales.

In this chapter I describe the nature of variables in social and behavioral research, basic classifications of variables (observable vs. unobservable; discrete vs. continuous), levels of measurement (nominal, ordinal, interval, and ratio), binary scales, rating scales, and scaling. The focus is on binary scales and rating scales that are typically used for assessment in counseling and related fields (e.g., Likert scales, Likert-type scales, and frequency rating scales). Some basic transformation of scales is also discussed.

1.1 Variables in Social and Behavioral Research

In general, a variable is any characteristic of a person (or an object) that may vary across persons or across different time points. A person's weight, for example, is a variable with different values for different people, although some people may weigh the same. This variable can also take on different values at different points in time, such as when obtaining repeated measurements for one person (say, every month during a one-year period to monitor the effect of a weight-loss treatment). Most often, the capital letters X, Y, and Z (in italics) are used to denote variables. Alternately, if a study involves many variables, a capital letter with subscripts can be used to denote different variables (e.g., X₁, X₂, X₃). Variables can also be described as observable versus unobservable or continuous versus discrete. Constants (i.e., numbers that remain the same throughout an analysis) are represented by lowercase letters in italics (e.g., a, b, c, d).

1.1.1 Observable Versus Latent Variables

Variables that can be measured directly are referred to as observable variables. For example, gender, age, ethnicity, and socioeconomic status are observable variables. Variables such as intelligence, attitude, motivation, anxiety, self-esteem, and verbal ability, on the other hand, are not directly observable and are therefore referred to as latent (i.e., unobservable or hidden) variables or constructs. Typically, a construct is given an operational definition specifying which observed variables are considered to be measurable indicators of the construct. For instance, measurable indicators of anxiety can include the person's responses to items on an anxiety test, the person's heartbeat and skin responses, or his or her reactions to experimental manipulations.

It is important to note that the operational definition for a construct should be based on a specific theory; therefore, the validity of the measurable indicators of the construct will necessarily depend on the level of correctness of this theory. For example, if a theory of creativity assumes, among other things, that people who can provide different approaches to the solution of a given problem are more creative than those who provide fewer approaches, then the number of approaches to solving individual problems (or tasks) can be used as an indicator of creativity. If, however, this theory is proven wrong, then the person's score on this indicator cannot be used for valid assessment of creativity.

1.1.2 Continuous Versus Discrete Variables

It is also important to understand the differences between continuous and discrete variables. Continuous variables are those that can take any possible value within a specific numeric interval. For example, the height of the students in a middle school population is a continuous variable because it can take any value (usually rounded to the nearest inch or tenth of an inch) within a numeric interval on the height measuring scale. Other examples of continuous variables are the students' ages, time on task in a classroom observation, and abilities that underlie proficiency outcomes in subject areas such as math, science, or reading. Latent variables that are typically involved in counseling research are continuous in nature—for example, motivation, anxiety, self-efficacy, depression, social skills, multicultural competence, and attitude (e.g., toward school, religion, or minority groups).

Discrete variables, on the other hand, can take only separate values (say, integer numbers). The measurement of a discrete variable usually involves counting or enumeration of how many times something has occurred—for example, the number of spelling errors in a writing sample or the frequency with which a specific behavior (e.g., aggressiveness) has occurred during a period of time. Thus, while the measurement of a continuous variable relates to the question “How much?” the measurement of a discrete variable relates to the question “How many?”

---

Note 1.1

It may be confusing that values of continuous variables are reported as “discrete” values. This confusion arises because the values of a continuous variable are rounded. Take, for example, a weekly weather report on temperature (in Fahrenheit): 45°, 48°, 45°, 58°, 52°, 47°, 51°—values of the continuous variable temperature look discrete because they are rounded to the nearest integer. As another example, GPA scores rounded to the nearest hundredth (e.g., 3.52, 3.37, 4.00, and so forth) also look like discrete values, but they represent a continuous variable (academic achievement).

---

1.2 What Is Measurement?

Measurement can be thought of as a process that involves three components—an object of measurement, a set of numbers, and a system of rules—that serve to assign numbers to magnitudes of the variable being measured. The object of measurement can be an observable variable (e.g., weight or age) or a latent variable (e.g., self-efficacy, depression, or motivation). Any latent variable can be viewed as a hidden continuum with magnitudes increasing in a given direction, say, from left to right if the continuum is represented with a straight line. A latent variable is usually defined with observable indicators (e.g., test items). The person's total score on these indicators is the number assigned to the hidden magnitude for this person o...

Table of contents

• Cover
• Title Page
• Copyright
• Dedication
• Preface
• Acknowledgments
• About the Author
• Part I: Scales, Reliability, and Validity
• Part II: Factor Analysis in Construct Validation
• Part III: Psychometric Scale Analysis
• References
• Index
• Technical Support
• End User License Agreement