Bayesian Modeling Using WinBUGS

Ioannis Ntzoufras

• Publisher: Wiley
• Year: 2011
• ISBN:
  9781118210352
• Edition: 1
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

A hands-on introduction to the principles of Bayesian modeling using WinBUGS

Bayesian Modeling Using WinBUGS provides an easily accessible introduction to the use of WinBUGS programming techniques in a variety of Bayesian modeling settings. The author provides an accessible treatment of the topic, offering readers a smooth introduction to the principles of Bayesian modeling with detailed guidance on the practical implementation of key principles.

The book begins with a basic introduction to Bayesian inference and the WinBUGS software and goes on to cover key topics, including:

• Markov Chain Monte Carlo algorithms in Bayesian inference
  
  

• Generalized linear models
  
  

• Bayesian hierarchical models
  
  

• Predictive distribution and model checking
  
  

• Bayesian model and variable evaluation
  
  

Computational notes and screen captures illustrate the use of both WinBUGS as well as R software to apply the discussed techniques. Exercises at the end of each chapter allow readers to test their understanding of the presented concepts and all data sets and code are available on the book's related Web site.

Requiring only a working knowledge of probability theory and statistics, Bayesian Modeling Using WinBUGS serves as an excellent book for courses on Bayesian statistics at the upper-undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners in the fields of statistics, actuarial science, medicine, and the social sciences who use WinBUGS in their everyday work.

CHAPTER 1

INTRODUCTION TO BAYESIAN INFERENCE

1.1 INTRODUCTION: BAYESIAN MODELING IN THE 21 ST CENTURY

The beginning of the 21 st century found Bayesian statistics to be fashionable in science. But until the late 1980s, Bayesian statistics were considered only as an interesting alternative to the “classical” theory. The main difference between the classical statistical theory and the Bayesian approach is that the latter considers parameters as random variables that are characterized by a prior distribution. This prior distribution is combined with the traditional likelihood to obtain the posterior distribution of the parameter of interest on which the statistical inference is based. Although the main tool of Bayesian theory is probability theory, for many years Bayesians were considered as a heretic minority for several reasons. The main objection of “classical” statisticians was the subjective view point of the Bayesian approach introduced in the analysis via the prior distribution. However, as history had proved, the main reason why Bayesian theory was unable to establish a foothold as a well accepted quantitative approach for data analysis was the intractabilities involved in the calculation of the posterior distribution. Asymptotic methods had provided solutions to specific problems, but no generalization was possible. Until the early 1990s two groups of statisticians had (re)discovered Markov chain Monte Carlo (MCMC) methods (Gelfand and Smith, 1990; Gelfand et al., 1990). Physicists were familiar with MCMC methodology from the 1950s. Nick Metropolis and his associates had developed one of the first electronic supercomputers (for those days) and had been testing their theories in physics using Monte Carlo techniques. Implementation of the MCMC methods in combination with the rapid evolution of personal computers made the new computational tool popular within a few years. Bayesian statistics suddenly became fashionable, opening new highways for statistical research. Using MCMC, we can now set up and estimate complicated models that describe and solve problems that could not be solved with traditional methods.

Since 1990, when MCMC first appeared in statistical science, many important related papers have appeared in the literature. During 1990–1995, MCMC-related research focused on the implementation of new methods in various popular models [see, e.g., Gelman and Rubin (1992), Gelfand, Smith and Lee (1992), Gilks and Wild (1992), Dellaportas and Smith (1993)]. The development of MCMC methodology had also promoted the implementation of random effects and hierarchical models.

Green’s (1995) publication on reversible jump Markov chain Monte Carlo (RJMCMC) algorithm boosted research on model averaging, selection and model exploration algorithms [see, e.g., Dellaportas and Forster (1999), Dellaportas et al. (2002), Sisson (2005), Hans et al. (2007)]. During the same period, the early versions of BUGS software appeared. BUGS was computing-language-oriented software in which the user only needed to specify the structure of the model. Then, BUGS was using MCMC methods to generate samples from the posterior distribution of the specified model. The most popular version of BUGS (v.05) was available via the Internet in 1996 [manual date August 14, 1996; see, Spiegelhalter et al. (1996a)]. Currently WinBUGS version 1.4.3¹ is available via the WinBUGS project Webpage (Spiegelhalter et al., 2003d). Many add-ons, utilities, and variations of the package are also available. The development of WinBUGS had proved valuable for the implementation of Bayesian models in a wide variety of scientific disciplines. In parallel, many workshops and courses have been organized on Bayesian inference, data analysis, and modeling using WinBUGS software. WinBUGS is a key factor in the growing popularity of Bayesian methods in science.

Development, extensions, and improvement of MCMC methods have also been considered in statistical research since the mid-1990s. Automatic samplers, which will be directly applicable in any set of data, are within this frame of research and have led to the slice sampler (Higdon, 1998; Damien et al., 1999). Various samplers designed for mod...

Table of contents

• COVER
• TITLE
• COPYRIGHT
• PREFACE
• ACKNOWLEDGMENTS
• ACRONYMS
• CHAPTER 1: INTRODUCTION TO BAYESIAN INFERENCE
• CHAPTER 2: MARKOV CHAIN MONTE CARLO ALGORITHMS IN BAYESIAN INFERENCE
• CHAPTER 3: WinBUGS SOFTWARE: INTRODUCTION, SETUP, AND BASIC ANALYSIS
• CHAPTER 4: WinBUGS SOFTWARE: ILLUSTRATION, RESULTS, AND FURTHER ANALYSIS
• CHAPTER 5: INTRODUCTION TO BAYESIAN MODELS: NORMAL MODELS
• CHAPTER 6: INCORPORATING CATEGORICAL VARIABLES IN NORMAL MODELS AND FURTHER MODELING ISSUES
• CHAPTER 7: INTRODUCTION TO GENERALIZED LINEAR MODELS: BINOMIAL AND POISSON DATA
• CHAPTER 8: MODELS FOR POSITIVE CONTINUOUS DATA, COUNT DATA, AND OTHER GLM-BASED EXTENSIONS
• CHAPTER 9: BAYESIAN HIERARCHICAL MODELS
• CHAPTER 10: THE PREDICTIVE DISTRIBUTION AND MODEL CHECKING
• CHAPTER 11: BAYESIAN MODEL AND VARIABLE EVALUATION
• APPENDIX A: MODEL SPECIFICATION VIA DIRECTED ACYCLIC GRAPHS: THE DOODLE MENU
• APPENDIX B: THE BATCH MODE: RUNNING A MODEL IN THE BACKGROUND USING SCRIPTS
• APPENDIX C: CHECKING CONVERGENCE USING CODA/BOA
• APPENDIX D: NOTATION SUMMARY
• REFERENCES
• INDEX