The Social Validity Manual

A Guide to Subjective Evaluation of Behavior Interventions

Stacy L. Carter

• Publisher: Academic Press
• Year: 2009
• ISBN:
  9780080954042
• Topic: Psychology
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

Applied Behavior Analysis is the science of applying experimentally derived principles of behavior to improve socially significant behavior. The research and application of ABA contributes to a wide range of practical areas, including AIDS prevention, education, gerontology, language acquisition and parenting, and ABA-based interventions have gained particular popularity in the last 20 years related to teaching students with autism spectrum disorders. Social Validity, a concept used in such behavioral intervention research, focuses on whether the goals of treatment, the intervention techniques used and the outcomes achieved are acceptable, relevant, and useful to the individual in treatment. Judgments are made (often via clinical trials) about the effects of the intervention based on statistical significance and magnitude of effect. Essentially, social validity alerts us as to whether or not the ABA-based intervention has had a palpable impact and actually helped people in ways that are evident in everyday life.

This clinical research volume offers a detailed evaluation of the extant findings on Social Validity, as well as discussion of newly emerging factors which reemphasize the need for well-developed methods of examining SV. Basic conceptualizations, measurement, research findings, applications, ethics, and future implications are discussed in full, and novel recommendations relating back to clinical treatment are provided. The volume will give readers a firm understanding of the general concept of SV, help them become familiar with the research methods and findings, and teach them how to establish and evaluate the Social Validity of individual interventions and treatment programs.

• Consolidates literature broadly distributed across journals and book chapters into single source
• Provides discussion of SV in greater depth and breadth than is found in other sources, which generally just focus on general conceptualization and broad research findings
• Describes how concept of SV can be influential in numerous areas of clinical practice

Chapter 1. Background of Social Validity

Defining Social Validity

What is Social Validity?

The construct of social validity has been conceptualized in many different ways. Schwartz and Baer (1991) point out that numerous meanings have been ascribed to the construct of social validity, which have thus catapulted the construct into a mass of confusion. Even the terminology associated with social validity has been used in various ways that make determining exactly what social validity consists of extremely confusing at times. While the literature within the field of behavior analysis coined the term “social validity,” the term has not been used consistently even within the field where it was first used. Other fields of social science, as well as business and marketing, have used various terminology to describe similar constructs, such as social importance, social relevance, social significance, consumer satisfaction, clinical importance, clinical significance, educational relevancy, applied relevance, applied importance, ecological validity, cultural validity, and cultural significance. The sheer number of terms, which have in some instances been used interchangeably with social validity, may have resulted in some confusion about exactly what is involved in defining, measuring, and evaluating social validity. While social validity can be associated with several different disciplines and defined in many different ways, the type of social validity referred to within this book will be the social validity that was developed from the field of applied behavior analysis. There have been several proposed variations on how social validity should be defined, how it should be assessed, and how it should be used within the field of applied behavior analysis. Several of these variations will be described within this book, and each will be discussed in relationship to the original definition of social validity proposed by Wolf (1978).

For the purposes of this book, the term social validity will refer to the evaluation of the degree of acceptance for the immediate variables associated with a procedure or program designed to change behavior. This definition is derived from the definition provided by Wolf (1978). He defines social validity as (1) the social significance of the goals of treatment; (2) the social appropriateness of the treatment procedures; and (3) the social importance of the effects of treatments (Table 1.1). The most frequent method for determining the degree of acceptance for a procedure or program has been to ask those receiving, implementing, or consenting to a treatment about their opinions of the treatment. These opinions are then used to make decisions about current or future uses of the treatment.

Table 1.1 Wolf’s Three-Component Definition of Social Validity

1. Social significance of the goals of treatment 2. Social appropriateness of the treatment procedures 3. Social importance of the effects of treatment

Development of a Concept

Wolf (1978) describes, with an apology, his personal experiences with social validity that were associated with the development of a definition for the term. He describes how a last-minute deadline resulted in the following quote from Don Baer being included in the introductory issue of the Journal of Applied Behavior Analysis (JABA) to describe the purpose of the journal: “It is for the publication of applications of the analysis of behavior to problems of social importance.” Wolf states that he continued to think about the meaning of the term social importance and how it might be viewed by others in the field. He explains that the term social importance relied on subjective information such as opinions or judgments made by other people. Wolf feels that including this type of subjective component in the seminal issue of JABA conflicted with the objective nature of behavior analysis, which was considered to be in line with other natural sciences that avoided subjective evaluations of variables. Wolf goes on to note that the field of behavior analysis had separated itself from other social sciences (such as sociology, anthropology, psychiatry, etc.) that included subjective observations, and the board of editors for JABA wanted the journal to exemplify only strict objective measurements. The tradition of direct observation of behavior was considered necessary by Skinner (1969), as he states:

Much can be done in the study of behavior with methods of observation no more sophisticated than those available, say, to Faraday, with his magnets, wires, and cells. Eventually the investigator may move to peripheral areas where indirect methods become necessary, but until then he must forego the prestige which attaches to traditional statistical methods (p. 111).

While Wolf continued to struggle with a definition of social importance that would fit the objective nature of the journal, he found that the journal readers, article authors, and reviewers of manuscripts did not complain about the lack of a definition of social importance and they appeared to be able to readily recognize its presence or absence. This appears to be similar to the statement made by Supreme Court Justice Potter Stewart, who avoided providing a succinct definition for pornography but stated “I know it when I see it….” The point is that Wolf finds that social importance is recognized, accepted, and valued even though it may not lend itself to an objective method of measurement.

Dimensions of Applied Behavior Analysis

In developing an understanding of social validity, it is important to review the development of the field of applied behavior analysis. The JABA has become the primary outlet for disseminating research in the area of applied behavior analysis. In the introductory issue of this journal, Baer, Wolf, and Risley (1968) outline the dimensions of applied behavior analysis, and these dimensions are still frequently referred to in the literature. They describe seven criteria needed for a study to be considered an example of applied behavior analysis (Table 1.2). These criteria are essential for adhering to the scientific principles established within the field of applied behavior analysis.

Table 1.2 Dimensions of ABA

Baer, Wolf, and Risley’s (1968) Seven Dimensions of Applied Behavior Analysis
1. Applied 2. Behavioral 3. Analytic 4. Technological 5. Conceptually systematic 6. Effective 7. Generality

Criteria I: Applied

The applied criterion refers to how target behaviors are selected. With applied research, target behaviors must be selected because of their importance to society. With nonapplied research, target behaviors may be selected for reasons that provide convenience to the researcher. Applied research focuses on target behaviors that are highly relevant to society and that typically reveal immediately important relationships between the behavior and certain stimuli.

Criteria II: Behavioral

The behavioral criterion focuses on the pragmatic aspect of research. Applied research should focus on the physical events performed by individuals rather than on other factors only related to physical performance of these events, such as verbal descriptions of performance, unless independently supported by other measures.

Criteria III: Analytic

The analytic criterion refers to the reliability of the research. In order for a study to meet the criterion of being analytic, it must reasonably demonstrate some level of control over a behavior. Nonapplied research may require extensive repetition of control over behavior, but applied research has to rely on achieving as much control over behavior as can be demonstrated to make the control believable to a select audience. In other words, applied research cannot always achieve the level of demonstrated control over behavior that can be achieved in nonapplied research, due to allowances provided within social settings.

Criteria IV: Technological

The technological criterion concerns the description of the techniques used to promote behavior change. Applied research requires that all components of a treatment be completely and accurately identified and reported. This criterion does not distinguish nonapplied research, but requires that enough information be provided on the techniques used in an experiment so that others might replicate these same techniques in exactly the same manner.

Criteria V: Conceptually Systematic

The conceptually systematic criterion relates to how techniques relate to basic principles that are considered part of the field of behavior analysis. Applied research must convey how specific procedures have been derived from basic principles so that similar procedures might be derived from those same basic principles. This criterion was deemed necessary to promote the field of applied behavior analysis beyond what might be considered a series of unrelated techniques or procedures.

Criteria VI: Effective

The effective criterion relates to the relevance of outcomes to practical situations. Nonapplied research may seek to produce outcomes that have theoretical importance, while applied research must produce results that have some practical value. Applied research requires outcomes that are large enough to be considered important to those who deal closely with a behavior.

Criteria VII: Generality

The generality criterion involves the enduring qualities of a treatment over time, over settings, or across related behaviors. Since applied research must deal with socially important behaviors, the changes that take place in these behaviors may be important in ways different from the way in which the changes were initially introduced. Demonstrating that treatments are effective in more than one way increases the value of the procedure and the importance to society.

Examining these criteria in light of social validity, it seems that the applied criteria and the effective criteria play the most important roles in understanding social validity, because both of these criteria focus on aspects of social determination or judgment. In addition, the generality criterion may be highly important toward gaining a more comprehensive understanding of the influences on social validity. Although some of the dimensions of applied behavior analysis may play more important roles in the assessment of social validity, as stated by Schwartz and Baer (1991), state-of-the-art social validity assessment should address all the dimensions of applied behavior analysis.

Importance of Social Validity

Is Social Validity Important?

Social validity has been viewed as essential to the survival of the field of applied behavior analysis and as a diversionary trap that would lead to the demise of the field (Baer & Schwartz, 1991). They state, “We know little about the accurate and valid assessment of what gets called social validity—and we know a great deal less about the survival of fields” (p. 231). While either of these views may potentially be correct, what appears most important is to gain an understanding of the construct and what role it might play in benefiting society. If social validity has some value to society, it should be embraced and further utilized in the field of behavior analysis rather than avoided and misunderstood. Baer and Schwartz state that the primary problems facing the field of applied behavior analysis with regard to social validity were how to assess it and how to apply it to improve behavior interventions.

In some cases it may be argued that social validity is not important. An expert may develop a program that is considered highly effective at achieving a desired outcome, which may on the surface appear to be a very useful program regardless of whether nonexperts agree. An extreme hypothetical example of this might be as follows:

An expert is assigned a case in which a client is displaying aggressive acts toward others within a residential facility. The client frequently strikes, pushes, and bites others throughout the day during typical daily activities. The expert designs a program intended to reduce the frequency of aggressive acts toward others. The expert trains the staff at the facility to place the client in a time-out room following each aggressive act. The program is determined to show an immediate reduction in the frequency of aggressive acts when compared to data obtained prior to the treatment.

On the surface, this treatment may appear to have produced the desired outcome (reduction of aggression), but it actually neglects to mention a number of critical elements that would need to be considered before the treatment could truly be considered a success. First, the treatment does not clarify the amount of time the client would be required to remain in the time-out room. This problem has been highly relevant in intermediate care facilities, due to the potential for abusing this type of procedure. The staff at the facility may choose to leave the client in the time-ou...

Table of contents

• Cover image
• Table of Contents
• Copyright
• Dedication
• List of Figures
• List of Tables
• Preface
• Chapter 1. Background of Social Validity
• Chapter 2. Conceptualizations of Social Validity
• Chapter 3. Instruments for Evaluating Social Validity
• Chapter 4. Research on Social Validity
• Chapter 5. Increasing the Social Significance of Treatment Goals
• Chapter 6. Enhancing the Appropriateness of Treatment Procedures
• Chapter 7. Improving the Importance of Treatment Effects
• Chapter 8. Social Validity and Ethics
• Chapter 9. Organization of Social Validity Data
• Chapter 10. The Future of Social Validity
• References
• Index