Mixed-Mode Official Surveys

Design and Analysis

Barry Schouten, Jan van den Brakel, Bart Buelens, Deirdre Giesen, Annemieke Luiten, Vivian Meertens

• Publisher: Chapman and Hall/CRC
• Year: 2021
• ISBN:
  9780429865855
• Edition: 1
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

Mixed-mode surveys have become a standard at many statistical institutes. However, the introduction of multiple modes in one design goes with challenges to both methodology and logistics. Mode-specific representation and measurement differences become explicit and demand for solutions in data collection design, questionnaire design, and estimation. This is especially true when surveys are repeated and are input to long time series of official statistics. So how can statistical institutes deal with such changes? What are the origins of mode-specific error? And how can they be dealt with? In this book, the authors provide answers to these questions, and much more.

Features

• 
  
  

• Concise introduction to all the key elements of mixed-mode survey design and analysis

• 
  
  

• Realistic official statistics examples from three general population surveys

• 
  
  

• Suitable for survey managers and survey statisticians alike

• 
  
  

• An overview of mode-specific representation and measurement errors and how to avoid, reduce and adjust them.

Part I

Introduction

1
Foreword to Mixed-Mode Official Surveys: Design and Analysis

DOI: 10.1201/9780429461156-1

1.1 Why a Book on Mixed-Mode Survey Design and Analysis?

The past two decades have seen the rise of online surveys and along with it a strong and renewed interest in mixed-mode surveys. The main reason for this renewed interest is that the web is a cheap and fast survey mode but has lower response rates than the traditional interviewer-assisted and paper survey modes. As a consequence, mixing of modes has been, and still is, a natural approach to elevate the lower response rates, especially in the larger official surveys. In a recent survey among European national statistical institutes (Murgia, Lo Conte, and Gravem, 2018), all but one country indicated to use mixed-mode designs and about half of the countries replied that web is one of the modes in these designs.

The use of mixed-mode designs is not at all new, but what is new is the much stronger focus on the implications of mixing modes for analysis and estimation. Mixed-mode designs have been around since the early days of surveys and have become a prominent option with the advent of telephone surveys. The impact of modes on the various survey errors, such as nonresponse and measurement, is known for a long time and has been the subject of research for several decades. Accounts of these can be found in Groves (1989), De Leeuw (1992), Groves et al. (2002), Biemer and Lyberg (2003), the four editions of Dillman, Smythe, and Christian (2014), Cernat and Sakshaug (2020), Olson et al. (2020), and Toepoel, De Leeuw, and Hox (2020). With the web as the fourth-largest survey mode, next to mailed questionnaires, telephone, and face-to-face interviews, the number of design options increased. However, another mode was introduced with quite different features from the two interviewer-assisted modes. These different features have not gone unnoticed and, since the web is a relatively cheap mode, have led to much more explicit cost–benefit analyses in the survey design. For several years, survey methodology conferences, such as AAPOR and ESRA, had many sessions just on mixed-mode survey design and their impact on survey errors, survey costs, and survey estimates. This interest was unprecedented up to the introduction of the web.

The motivation for this book came from the desire to provide a comprehensive overview of the role of mixed-mode surveys in all survey methodology stages, from the early design stage to the analysis and estimation stages. Such a complete account was not available at the time of writing this book, although strong and informative books have been written about design, such as Dillman, Smyth, and Christian (2014). The authors of this book, motivated by their experiences in the migration of many surveys to mixed-mode designs, strongly believe that it is important to consider all stages of survey research. The options to reduce mode impact range from prevention of effects in questionnaire design to avoidance of effects in mode choice and mode allocation to adjustment of effects in the estimation. However, the options must and cannot be seen as complementary or separate; they are intertwined and must be considered in conjunction with each other. In this book, all stages and all options are discussed, as well as strategies to report mode effects in publications.

A large wave of transitions of surveys to mixed-mode designs has passed, but this book will still be of use to survey institutes considering migration to surveys employing multiple modes and to scholars investigating design and analysis of the impact of modes (Signore, 2019; Olson et al., 2020). Institutes may consider a redesign of their surveys, may be in the middle of such redesigns, or may face questions about the handling of mode impact after redesigns. Furthermore, the rise of all kinds of online devices implies that mixed-mode surveys are not fixed and stable and will have to be reconsidered continuously for the near and longer-term future. It must also be remarked, however, that mode effects do not occur in mixed-mode designs only. It is in those designs that mode impact is most visible, but the impact existed all along with the introduction of a mode. Hence, even for single-mode surveys, it is important to consider the role of a survey mode and this book may provide useful background.

This book is intended for both survey designers and users of surveys. The authors of the book have very diverse backgrounds, from questionnaire development up to sampling design and nonresponse adjustment. Readers of the book may have equally different backgrounds and may profit from the different chapters. Some chapters are more psychological, some are more methodological, and some are more statistical. Some chapters deal with multiple disciplines and may be of interest to all kinds of readers. The authors have been active in large-scale redesign project teams at Statistics Netherlands and often were responsible for different stages of the redesigns. The book may add value to readers who participate in such multidisciplinary teams as well. It may help them understand the terminology and methodological options.

All the authors of the book are methodologists or developers at a northern-European national statistical institute. Surveys conducted at an NSI typically are larger, often repeated, and concern the general population or large subpopulations. This implies a relatively high budget per sample unit, a focus on a full coverage of the population, and increased attention for comparability of survey statistics across time and between relevant subpopulations. In this type of surveys, mode impact can strongly affect time series. Because comparability in time is deemed a key feature, there is a lot of interest to detect and reduce mode impact. General references to literature are included, but the book does tend toward mixed-mode designs for the general population, high-budget surveys. The northern-European context means a relatively rich administrative data context and a high population coverage of online devices. Both impact the way surveys can and are designed and analyzed. Although theory and takeaway messages translate to all surveys, this must be kept in mind. Readers are referred to best practice documents as prepared by AAPOR, see www.aapor.org, and ESOMAR, see www.esomar.org.

In the next two sections, the origin and type of modes and devices are introduced and an outline is provided of the book.

1.2 Modes and Devices

Response of a sampled population unit requires at least four steps: making contact, assessing eligibility, obtaining participation, and completing the questionnaire. This book deals with designs in which multiple modes may be used in parallel to complete the questionnaire. So, when multiple modes are used to make contact, to assess eligibility, or to obtain cooperation, but only one mode is used to complete the survey, then these are not viewed as mixed-mode surveys here. Nonetheless, contact modes will naturally come up as part of mixed-mode designs in various chapters.

In this context, a survey mode is a communication channel t...

Table of contents

• Cover
• Half-Title
• Series
• Title
• Copyright
• Contents
• Preface
• Acknowledgements
• Authors
• Part I Introduction
• Part II Mode Effects
• Part III Design
• Part IV Analysis
• Part V The Future of Mixed-Mode Surveys
• References
• Index