Designing and Conducting Health Surveys

A Comprehensive Guide

Lu Ann Aday, Llewellyn J. Cornelius

• Publisher: Jossey-Bass
• Year: 2011
• ISBN:
  9781118046678
• Edition: 3
• Topic: Medicine
• Subtopic: Public Health, Administration & Care

Designing and Conducting Health Surveys is written for students, teachers, researchers, and anyone who conducts health surveys. This third edition of the standard reference in the field draws heavily on the most recent methodological research on survey design and the rich storehouse of insights and implications provided by cognitive research on question and questionnaire design in particular. This important resource presents a total survey error framework that is a useful compass for charting the dangerous waters between systematic and random errors that inevitably accompany the survey design enterprise. In addition, three new studies based on national, international, and state and local surveys—the UNICEF Multiple Indicator Cluster Surveys, California Health Interview Survey, and National Dental Malpractice Survey—are detailed that illustrate the range of design alternatives available at each stage of developing a survey and provide a sound basis for choosing among them.

CHAPTER ONE

THINKING ABOUT TOPICS FOR HEALTH SURVEYS

Chapter Highlights

1. Surveys systematically collect information on a topic by asking individuals questions to generate statistics on the group or groups that those individuals represent.

2. Health surveys ask questions about a variety of factors that influence, measure, or are affected by people’s health.

3. Health survey researchers should review the major international, national, state, and local health surveys relevant to their interests before undertaking their own study.

4. Good survey design is basically a matter of good planning.

5. The total survey error and survey quality framework alerts researchers to ways to identify and mitigate both bias and variable errors in surveys.

This book provides guidance for designing and conducting health surveys. These surveys systematically collect information on a topic of interest (such as state health care reform legislation) by asking individuals questions (about whether they and their family members have insurance coverage) to generate statistics (percentage who are uninsured) for the group or groups those individuals represent (noninstitutionalized residents under sixty-five years of age).

This chapter addresses (1) the topics, techniques, and ethical issues that will characterize the design and conduct of health surveys in the future; (2) the defining features of surveys compared with other data collection methods; and (3) the reasons for studying health surveys. It also provides (4) a framework for classifying the topics addressed in health-related surveys, (5) illustrative examples of health surveys used in this book, and (6) an overview of the total survey design and survey quality approach to designing and conducting health surveys.

Future Health Surveys

Health surveys have been and will continue to be important sources of information for health care policymakers, public health professionals, private providers, insurers, and health care consumers concerned with the planning, implementation, and evaluation of health-related programs and policies. The design and conduct of health surveys in the future will be shaped by changes in the diversity, complexity, and sensitivity of the topics addressed in these studies; the innovative techniques and technologies that are being developed for carrying them out; and the new or intensified ethical dilemmas that are a result of these changes.

Topics

The topics addressed in health surveys have been and will continue to be sensitive and complex. Such sociomedical morbidities as HIV/AIDS, child abuse, sexual dysfunction, drug and alcohol addiction, and family violence, among others, are now encompassed in definitions of public health and medical problems. The issue of access to medical care focuses on vulnerable and hard-to-locate populations differentially experiencing these sociomedical morbidities: gay/lesbian/bisexual /transgendered (GLBT) persons, drug abusers, the homeless, medically fragile children and the elderly, and undocumented migrant and refugee populations. Health care program designers are concerned with the number of people in these vulnerable groups; the particular health problems they experience; the barriers to care they confront; the ways in which their knowledge, attitudes, and behaviors exacerbate the risk of their contracting serious illnesses; and the resources they have to deal with these problems.

These trends in asking tough questions of hard-to-locate respondents in order to gain information for the design of cost-effective public and private health programs to address the needs of these respondents will continue.

Techniques and Technologies

The topics to be addressed in health surveys present new and intensified challenges at each stage of the design and conduct of a study. Corresponding to these developments is the emergence of new technologies for assisting with these tasks.

Rapid growth in the number and diversity of journals and specialized publications dealing with health topics has made the job of identifying and evaluating the major research in any given area more challenging. Computerized text search programs have greatly facilitated access to published research, but knowledge of effective search techniques is required to carry out these searches efficiently. These databases encompass professional journals and related periodical literature, as well as increasingly expanded online access to books, government publications, and unpublished research in progress relevant to the topic of interest. For those health topics for which little information is available because of the newness of the topic or the corollary lag in the dissemination of research results, survey designers need to contact relevant public or private funding agencies and colleagues in the field who are known to have research in progress on the issue.

Training programs are needed to prepare both students and professionals for carrying out these searches and evaluating the credibility of sources that are identified. The credibility of Internet and unpublished research could be evaluated based on the authoritativeness and previous track record of the organization or individual to whom the work is credited (for example, National Center for Health Statistics reports are likely to be a more credible source than an unpublished manuscript posted by a university faculty member with little record of peer-reviewed publications or funding); the other research sources that the research draws on or references; as well as the standards for evaluating the possible sources of errors in research in general and surveys in particular discussed further later in this chapter and highlighted in each of the chapters that follow. (Also, see White, 1994, and Wortman, 1994, for a discussion of procedures and criteria for retrieving and evaluating scientific literature.)

In the interest of learning about health and health-related attitudes, knowledge, and behaviors, survey researchers are attempting to penetrate more deeply into the traditionally best-kept family and personal secrets. The application of principles of cognitive psychology to the design and evaluation of such questions has challenged many of the standardized approaches to asking questions. At a minimum, in the early stages of questionnaire development, survey designers should ask respondents what went through their minds when they were asked sensitive questions about themselves or other members of their family. Moreover, prominent survey methodologists have called for the development of theories of surveys. These theories would focus on the decisions that could be made at each stage of designing and carrying out a survey to maximize quality and minimize costs (Biemer & Lyberg, 2003; Dillman, 2000, 2002; Groves, 1987, 1989; Groves, Dillman, Eltinge, & Little, 2002; Groves, Singer, & Corning, 2000; Sudman, Bradburn, & Schwarz, 1996).

The technology that has had the largest influence on the techniques used in the design and conduct of health surveys is computerized information processing. These methods can be used to facilitate research on different survey techniques or methodologies (such as using different approaches to sampling respondents, phrasing questions, and training interviewers). The rapid turnaround of information made possible by computerized methods should expedite choices among design alternatives of this kind. More attention needs to be given to evaluating the overall quality of the information obtained using emerging computerized approaches, the impact on the interviewers and respondents of using computers to display the questions and enter respondents’ answers, and the costs at each stage of the study. Computerized survey technologies are wonderful innovations. As with any other new invention, however, the most effective and efficient means of producing and using it needs to be explored and tested rather than simply assumed (Jones, 1999).

The topics and technologies evolving for health surveys present both challenges and opportunities in designing the samples for these studies. Health surveys have increasingly focused on rare or hard-to-locate populations. Innovative approaches are required to identify the universe or target population of interest, develop cost-effective methods for drawing the sample, and then find individuals to whom the questionnaire or interview should be administered. Survey designers must be aware of the methods that have been developed to identify and oversample rare populations and be prepared to invest time and resources to come up with the best sample design for their study.

Ethics

Asking people questions in surveys about aspects of their personal or professional lives always involves a consideration of the ethical issues posed by this process. Are the participants fully informed about the study, and do they voluntarily agree to participate? What benefits or harm may they experience if they participate? Will their right to remain anonymous and the confidentiality of the information they provide be maintained when the findings are reported? The evolution of the topics, techniques, and technologies just reviewed promises to heighten, rather than diminish, the importance of these ethical questions in the design and conduct of health surveys (Sudman, 1998).

The Privacy Rule enacted under the Health Insurance Portability and Accountability Act (HIPAA) of 1996 established minimum federal standards for protecting the privacy of individually identifiable health information. The Privacy Rule confers certain rights on individuals, including rights to access and amend their personal health information and to obtain a record of when and why this information has been shared with others for certain purposes. The Privacy Rule establishes conditions under which covered entities can provide researchers access to and use of personal health information when necessary to conduct research (National Institutes of Health, 2004; U.S. Department of Health and Human Services, Office for Civil Rights, 2005). The HIPAA legislation and related Privacy Rule requirements have imposed significant constraints on researchers in addressing issues of informed consent, the benefit versus harm to study participants, and rights of anonymity and confidentiality, as well as in obtaining Institutional Review Board (IRB) approval for the conduct of survey research. (An informative overview of the implications of these and related security and privacy issues for health surveys may be found in the series of papers presented in “Session 5: Security and Privacy,” National Center for Health Statistics, 2004b.)

Informed Consent. The use of cold contact (unannounced) calls in random digit dialing telephone surveys permits very little advance information to be provided to the respondent about the nature of the study and the ways in which the information will be used. Survey designers are reluctant to spend much time giving respondents details on the survey for fear they will hang up. There is also little opportunity to elicit the formal written consent of respondents for what might be particularly sensitive topics. Respondents with what are perceived to be socially undesirable diseases and little means to pay for health care also may feel obligated to participate in a study if their providers ask them to do so, fearing that they will subsequently be refused treatment if they do not. From the providers’ or researchers’ perspectives, however, the increasingly complex and restrictive informed consent procedures may be viewed as seriously jeopardizing the ability to carry out reasonable scientific research protocols.

Benefit Versus Harm. Rational and ethical survey design attempts to ensure that the benefits outweigh the costs of participating. Asking people sensitive questions about threatening or difficult topics may call forth memories or emotions that are hard for them to handle. Most survey designers do not explicitly consider such costs to the respondents.

Providing monetary incentives does increase people’s willingness to participate in surveys. However, more research is needed to examine the effect of such incentives on the quality of the information provided. Do respondents feel more obligated, for example, to give answers they think the interviewer wants to hear? Offering large incentives may also be viewed as raising questions about whether this is unduly coercive for certain respondents, such as poor or uninsured patients.

Rights of Anonymity and Confidentiality. Finally, an important issue in the design and conduct of surveys is guaranteeing the anonymity of survey respondents and the confidentiality of the information they provide. This issue is made more salient with the possibility of computerized linkages between sources, such as databases that link telephone numbers to household addresses or between survey data and medical records or billing information from providers. These issues have taken on even greater salience in the context of the HIPAA legislation, which sharply restricted the use of patient identifiers and introduced more stringent informed clearance for permitting access to medical record and related health plan data (National Institutes of Health, 2004; U.S. Department of Health and Human Services, Office for Civil Rights, 2005).

The United States has become an increasingly litigious society, as evidenced by the growing number of malpractice suits brought against health care providers. Survey designers can thus expect to confront more detailed and cumbersome review procedures for evaluating how the rights of study participants will be protected in carrying out the survey.

This environment compels that researchers, IRBs, and participating providers and funders devise means for constructive problem solving to protect the rights of study participants and ensure the feasibility of conducting sound research. Reactive methodology attempts to understand the dynamics that come into play and how best to resolve potential conflicts between those involved in the conduct and approval of the research (Sieber, 2004). “Reactive” refers to reactions to the proposed research by the potential study subjects, their community, the researchers, and other external forces such as HIPAA and related IRB procedures. It most essentially acknowledges that although regulations may dictate specific constraints on those who seek to conduct research, it is the relationships between and among the players that ultimately dictate whether the research can be conducted both responsibly and feasibly.

Defining Features of Surveys

Several key dimensions define the survey approach: (1) a research topic or problem of interest has been clearly delineated, (2) information on the issue is gathered by asking individuals questions, (3) the data collection process itself is systematic and well defined, (4) the purpose of the study is to generate group-level summary statistics, and (5) the results are generalizable to the groups represented by the individuals included in the study (American Statistical Association, Section on Survey Research Methods, 1998a, 1998b).

A number of these features are not unique to surveys, but taken together they tend to distinguish this data-gathering method from such other approaches as using existing record data, conducting participant or nonparticipant observational studies, or carrying out case studies of one or a limited number of programs, institutions, or related units. Researchers should not necessarily assume that surveys are always the best approach to use in gathering data. That decision depends on what best enables investigators to address the research questions of interest to them. Furthermore, survey developers are increasingly making use of qualitative research methods such as focus groups, in-depth unstructured interviews, or ethnographies to guide the development and interpretation of more structured surveys (Sale, Lohfeld, & Brazil, 2002; Sudman et al., 1996). The similarities and differences among these methods and surveys are reviewed in the following discussion.

Existing Record Sources

Health care investigators might decide that existing record data, such as the medical records available in hospitals or physicians’ offices, claims data from private or public third-party insurers, or vital statistics records on births and causes of deaths, are the most useful and relevant sources for the types of information they need to gather for their study. Some of these sources may contain thousands of records that could be easily manipulated with high-powered personal computers. With these record sources, it will not be necessary to ask people questions directly to get the information. This is particularly true for factual data on...

Table of contents

• Title Page
• Copyright Page
• Table of Figures
• List of Tables
• Table of Exhibits
• Foreword
• PREFACE
• THE AUTHORS
• CHAPTER ONE - THINKING ABOUT TOPICS FOR HEALTH SURVEYS
• CHAPTER TWO - MATCHING THE SURVEY DESIGN TO SURVEY OBJECTIVES
• CHAPTER THREE - DEFINING AND CLARIFYING THE SURVEY VARIABLES
• CHAPTER FOUR - THINKING THROUGH THE RELATIONSHIPS BETWEEN VARIABLES
• CHAPTER FIVE - CHOOSING THE METHODS OF DATA COLLECTION
• CHAPTER SIX - DECIDING WHO WILL BE IN THE SAMPLE
• CHAPTER SEVEN - DECIDING HOW MANY WILL BE IN THE SAMPLE
• CHAPTER EIGHT - GENERAL PRINCIPLES FOR FORMULATING QUESTIONS
• CHAPTER NINE - FORMULATING QUESTIONS ABOUT HEALTH
• CHAPTER TEN - FORMULATING QUESTIONS ABOUT DEMOGRAPHICS AND BEHAVIOR
• CHAPTER ELEVEN - FORMULATING QUESTIONS ABOUT KNOWLEDGE AND ATTITUDES
• CHAPTER TWELVE - GUIDELINES FOR FORMATTING THE QUESTIONNAIRE
• CHAPTER THIRTEEN - MONITORING AND CARRYING OUT THE SURVEY
• CHAPTER FOURTEEN - PREPARING THE DATA FOR ANALYSIS
• CHAPTER FIFTEEN - PLANNING AND IMPLEMENTING THE ANALYSIS OF THE DATA
• CHAPTER SIXTEEN - WRITING THE RESEARCH REPORT
• RESOURCE A
• RESOURCE B
• RESOURCE C
• RESOURCE D
• RESOURCE E
• REFERENCES
• NAME INDEX
• SUBJECT INDEX