Handbook of Improving Performance in the Workplace, Measurement and Evaluation

James L. Moseley, Joan C. Dessinger, James L. Moseley, Joan C. Dessinger

• Publisher: Pfeiffer
• Year: 2009
• ISBN:
  9780470504000
• Edition: 1
• Topic: Business
• Subtopic: Human Resource Management
• Index: Business

HANDBOOK of IMPROVING PERFORMANCE IN THE WORKPLACE

Volume 3: Measurement and Evaluation

Volume Three of the Handbook of Improving Performance in the Workplace focuses on Measurement and Evaluation and represents an invaluable addition to the literature that supports the field and practice of Instructional Systems Design.

With contributions from leading national scholars and practitioners, this volume is filled with information on time-tested theories, leading-edge research, developments, and applications and provides a comprehensive review of the most pertinent information available on critical topics, including: Measuring and Evaluating Learning and Performance, Designing Evaluation, Qualitative and Quantitative Performance Measurements, Evidence-based Performance Measurements, Analyzing Data, Planning Performance Measurement and Evaluation, Strategies for Implementation, Business Evaluation Strategy, Measurement and Evaluation in Non-Profit Sectors, among many others. It also contains illustrative case studies and performance support tools.

Sponsored by International Society for Performance Improvement (ISPI), the Handbook of Improving Performance in the Workplace, three-volume reference, covers three core areas of interest including Instructional Design and Training Delivery, Selecting and Implementing Performance Interventions, and Measurement and Evaluation.

PART ONE
PERSPECTIVES IN MEASUREMENT AND EVALUATION

We live in a world of information. Whether reading a volume of selected texts penned with the authors’ personal flavors, hearing a movie critic’s reaction to the latest Hollywood blockbuster, writing a letter to friends about a recent memorable vacation, or engaging in conversations with colleagues on current affairs, we find ourselves immersed in the sensory sorting of information that ultimately leads to the formation of our perspectives. This first part of the book, “Perspectives in Measurement and Evaluation, ” is devoted to authors who wish to convey ideas, create impressions, and form mental images with the intent of leading the reader to develop new perspectives on each of the topics presented for consideration. Some perspectives will represent new infusions into traditional ways of considering measurement and evaluation. Whether the inclusions here are new or old, cathartic or traditional, you are invited to read, think, and react; you are invited to examine your own perspectives as well as the perspectives presented by the authors.

Chapter One: Measurement, Evaluation, and Research: Feedback for Decision Making. Binder’s chapter presents a different approach to the discussion of measurement and evaluation, overlapping at points with some of the more conventional discussions, but also stepping outside the mainstream measurement and evaluation thinking to highlight several key ideas.

Chapter Two: Measurement and Evaluation in the Workplace. Bosteder and Russ-Eft’s chapter is applications-based. It focuses on organizational level assessments, including financial and performance measures, audits, culture and climate surveys, and needs assessments. It considers team- or group-level assessments as well as assessments of conflict and communication style. It looks at individual assessments, including personality and skill assessments of leaders and supervisors, as well as employee satisfaction measures. It also examines customer loyalty measures.

Chapter Three: Unleashing the Positive Power of Measurement in the Workplace. Spitzer looks at measurement as the fundamental management system in any organization that triggers everything else that happens. The measurement system in most organizations is often left to chance, becomes dysfunctional, and causes counterproductive behaviors. Nothing works well when the measurement system is broken and, unfortunately, most organizational measurement systems are in various stages of brokenness.

Chapter Four: Relating Training to Business Performance: The Case for a Business Evaluation Strategy. The Tarnacki and Banchoff chapter is an adventurous case study about a young man’s progress in understanding how the essence of training and evaluation truly connect to business performance. A proactive alignment of the evaluation tactic, strategically linked to business results, whether quantitative or qualitative, is essential to avoid being a reactive servant to administrative waste.

Chapter Five: Success Case Methodology in Measurement and Evaluation. Apking and Mooney posit that our professions are top-heavy in formulas, equations, and techniques for evaluation. We do not need more magic formulas or more easy-to-use techniques. The key is to unlock the mystery and to develop a fresh perspective and a new strategy that looks at why we do evaluation and how we approach it.

CHAPTER ONE
Measurement, Evaluation, and Research: Feedback for Decision Making

Carl Binder

In his elegant little book devoted, not to measurement and evaluation, but to the essence of sustainable performance improvement, Esque (2001, p. 18] states that:

“In its simplest form, managing work consists of three components:

• Setting goals;
• Letting work happen and comparing work completed against goals; and
• Deciding whether to change how the goals are being pursued.”

In other words, there are three conditions that must be in place to say that performance is being managed: (1) clear, measurable goals; (2) measurement feedback provided to the performers in order to make decisions; and (3) the ability to control resources and conditions if the measurement feedback indicates need for a change.

In the field of human performance technology (HPT), this understanding of performance management provides a rationale for measurement and evaluation. We clearly identify the changes in performance we seek to produce. We measure and monitor the performance over time to determine whether our goals are being achieved, and at what rate. And we decide, based on the feedback provided by measurement, whether (and sometimes how) to change conditions when our goals are nor being met.

This logic applies at two levels in our field. First, when we as performance improvement professionals are called in to help address a performance challenge, we must ensure that the three conditions described by Esque (2001) are in place. In fact, unlike many of our colleagues in the performance improvement field who conduct cause analyses at the front end of projects to determine what interventions to propose, Esque follows a simpler path: He asks whether the three conditions are in place. Then, because they usually are not, he helps clients to establish clear, measurable goals; continuous data-based feedback loops to the performers; and processes for making decisions to change when goals are not being achieved. Once these conditions are in place, he coaches performers and their management through a continuous, data-based performance improvement process.

At a second level, whether we choose to take such a “lean” approach to human performance improvement or follow a more traditional sequence starting with front-end analysis, the three conditions that Esque describes should apply to our own performance as change agents, as well as to the performance that we seek to improve. For us to be effective as performance improvement professionals, we need the feedback provided by measurement to deter- mine whether to continue an intervention as planned—or to change. This is a simple cybernetic model of self-correction, inherent in both data-based performance improvement and in the fields of natural science and engineering upon which it has, at least in the past, been modeled. In the same way, this self-correcting approach is the raison d’etre for evaluation, the reason for its very existence, as described in this chapter.

A DIVERGENT PERSPECTIVE

This chapter takes a somewhat different approach to the discussion of performance measurement and evaluation, overlapping at points with some of the more conventional discussions provided in this volume, but also stepping outside of mainstream measurement and evaluation to highlight several key ideas. It covers much of the same ground as Binder’s (2001) article on “a few important ideas,” as well as elements of Binder’s (2002–2004) online column on measurement and evaluation entitled: “Measurement Counts!”

While the field of HPT originally emerged from the natural science of behavior (Binder, 1995), with its focus on standard units of measurement and replicable descriptions of procedures similar to accepted practice in the physical sciences, HPT has come to encompass a wider array of technical and conceptual inputs, many from the so-called “softer” fields of education and the social sciences. These other fields have introduced methods and approaches to measurement and evaluation that do not always align with generally accepted criteria in the natural sciences or engineering (Johnston & Pennypacker, 1993), especially with regard to the selection of measurement units and procedures. The principles and concepts presented in the current chapter reflect the author’s background and perspective and are as much as possible grounded in the philosophy and practice of natural science. One of the advantages of sticking closely to principles of natural science is that, in many respects, we can demystify measurement and evaluation and make it more accessible to front-line performance improvement practitioners. While this might seem, at the outset, counter-intuitive, continue reading to discover whether or not you think it is a fair statement. In many respects, natural science approaches to measurement and evaluation are simpler in concept, and less encumbered by statistical models and theoretical baggage, than are many approaches derived from the social sciences.

TERMINOLOGY

We use many terms in the field of performance measurement and evaluation, some of which have been defined in detail and with great technical sophistication elsewhere in this volume. For purposes of this chapter, here is a short list of concepts, defined with the intention of eliminating confusion, and appealing as much as possible to plain English explanations.

Measurement

Measurement is the process by which we identify the dimension, quantity, or capacity [of a thing] (American Heritage Dictionary, 2006). In the field of performance improvement, this term refers to the identification of what to count (business results, work output, and/or behavior); selection of relevant quantitative units of measurement (such as simple counts, kilograms, meters, liters, or other measures); and collection of data expressed in those units. For example, we might identify a successful business proposal as a countable work output and include criteria that define a “successful” proposal. We can then count successful proposals over successive time intervals prior to intervention to determine “baseline” levels of productivity. We might additionally count unsuccessful proposals and use the “success ratio” of successful to unsuccessful ones as a secondary measure. Once we have decided on an i...

Table of contents

• Cover
• Table of Contents
• Title
• Copyright
• LIST OF CASE STUDIES, EXHIBITS, FIGURES, PERFORMANCE SUPPORT TOOLS, AND TABLES
• INTRODUCTION TO VOLUME THREE
• ACKNOWLEDGEMENTS
• PART ONE: PERSPECTIVES IN MEASUREMENT AND EVALUATION
• PART TWO: PILLARS IN MEASUREMENT AND EVALUATION
• PART THREE: MOSAICS IN MEASUREMENT AND EVALUATION
• PART FOUR: FRONTIERS IN MEASUREMENT AND EVALUATION
• ABOUT THE EDITORS
• ABOUT THE CONTRIBUTORS
• NAME INDEX
• SUBJECT INDEX
• END USER LICENSE AGREEMENT