Applications of Item Response Theory To Practical Testing Problems

F. M. Lord

• Publisher: Routledge
• Year: 2012
• ISBN:
  9781136557316
• Edition: 1
• Topic: Education
• Subtopic: Education General

Published in 1980, Applications of Item Response Theory To Practical Testing Problems is a valuable contribution to the field of Education.

I

INTRODUCTION TO ITEM RESPONSE THEORY

1

Classical Test Theory—Summary and Perspective

1.1. INTRODUCTION

This chapter is not a substitute for a course in classical test theory. On the contrary, some knowledge of classical theory is presumed. The purpose of this chapter is to provide some perspective on basic ideas that are fundamental to all subsequent work.

A psychological or educational test is a device for obtaining a sample of behavior. Usually the behavior is quantified in some way to obtain a numerical score. Such scores are tabulated and counted. Their relations to other variables of interest are studied empirically.

If the necessary relationships can be established empirically, the scores may then be used to predict some future behavior of the individuals tested. This is actuarial science. It can all be done without any special theory. On this basis, it is sometimes asserted from an operationalist viewpoint that there is no need for any deeper theory of test scores.

Two or more “parallel” forms of a published test are commonly produced. We usually find that a person obtains different scores on different test forms. How shall these be viewed?

Differences between scores on parallel forms administered at about the same time are usually not of much use for describing the individual tested. If we want a single score to describe his test performance, it is natural to average his scores across the test forms taken. For usual scoring methods, the result is effectively the same as if all forms administered had been combined and treated as a single test.

The individual’s average score across test forms will usually be a bettermeasurement than his score on any single form, because the average score is based on a larger sample of behavior. Already we see that there is something of deeper significance than the individual’s score on a particular test form.

1.2. TRUE SCORE

In actual practice we cannot administer very many forms of a test to a single individual so as to obtain a better sample of his behavior. Conceptually, however, it is useful to think of doing just this, the individual remaining unchanged throughout the process.

The individual’s average score over a set of postulated test forms is a useful concept. This concept is formalized by a mathematical model. The individual’s score X on a particular test form is considered to be a chance variable with some, usually unknown, frequency distribution. The mean (expected value) of this distribution is called the individual’s true score T. Certain conclusions about true scores T and observed scores X follow automatically from this model and definition.

Denote the discrepancy between T and X by

E is called the error of measurement. Since by definition the expected value of X is T, the expectation of E is zero:

where μ denotes a mean and the subscripts indicate that T is fixed.

Equation (1-2) states that the errors of measurement are unbiased. This follows automatically from the definition of true score; it does not depend on any ad hoc assumption. By the same argument, in a group of people,

Equation (1-2) gives the regression of E on T. Since mean E is constant regardless of T, this regression has zero slope. It follows that true score and error are uncorrelated in any group:

Note, again, that this follows from the definition of true score, not from any special assumption.

From Eq. (1-1) and (1-3), since T and E are uncorrelated, the observed-score variance in any group is made up of two components:

The covariance of X and T is

An important quantity is the test reliability, the squared correlation between X and T, by (1-5),

If ρ_XT were nearly 1.00, we could safely substitute the available test score X for the unknown measurement of interest T.

Equations (1-2) through (1-6) are tautologies that follow automatically from the definition of T and E.

What has our deeper theory gained for us? The theory arises from the realizations that T, not X, is the quantity of real interest. When a job applicant leaves the room where he was tested, it is T, not X, that determines his capacity for future performance.

We cannot observe T, but we can make useful inferences about it. How this is done becomes apparent in subsequent sections (also, see Section 4.2).

An example will illustrate how true-score theory leads to different conclusions than would be reached by a simple consideration of observed scores. An achievement test is administered ...

Table of contents

• Front Cover
• Title Page
• Copyright
• Contents
• Preface
• PART I: INTRODUCTION TO ITEM RESPONSE THEORY
• PART II: APPLICATIONS OF ITEM RESPONSE THEORY
• PART III: PRACTICAL PROBLEMS AND FURTHER APPLICATIONS
• PART IV: ESTIMATING TRUE-SCORE DISTRIBUTIONS
• Answers to Exercises
• Author Index
• Subject Index