At the heart of the Bayesian method is the reliance of probability to model uncertainty and the use of Bayes rule to naturally join past (prior) information and data into posterior predictions. Driven as much by the availability of new computational techniques as by any inherent philosophical advantages, the popularity of the Bayesian paradigm has increased dramatically in the last two decades. The course will begin by introducing the Bayesian approach and inference from first principals. We will then consider sophisticated concepts in Bayesian modeling and model checking, followed by an introduction to practical Bayesian computation. In the last part of the course we will consider more closely specific Bayesian models and applicationS. Throughout the course we will illustrate via real example the three steps of Bayesian statistics: (1) setting up a probability model; (2) updating the fit of the model; (3) evaluating the fit of the posterior. The focus of the course will be an applicative one so that emphasis will be put on techniques rather than the underlying theory. 6-8 problem sets will include theoretical as well as computational exercises. Submission of all problem sets is mandatory and will comprise 20% of the grade just for the (real) effort.
We will be using Bayesian Data Analysis (second edition) by Gelman, Meng, Stern, and Rubin as the main course text. Additional material will be used from the following
Part I: Bayesian Inference
Week 1: Introduction via Bayes and Laplace historical examples, Bayes rule, posterior means and variances, binomial model, proportion of female births
Week 2: Standard univariate models including the normal and Poisson models, cancer rates, non-informative prior distributions
Week 3: Multi-parameter models, normal with unknown mean and variance, the multivariate normal distribution, multinomial models, election polling, bioassay. Computation and simulation from arbitrary posterior distributions in two parameters.
Week 4: Inference from large samples and comparison to standard non-Bayesian methods
Part II: Bayesian data analysis
Week 5: Hierarchical models, estimation of population parameters from data, rat tumor rates, SAT coaching experiments, meta analysis
Week 6: Model checking, posterior predictive checking, sensitivity analysis, model comparison and expansion, checking the analysis of the SAT coaching experiments
Week 7: (may be omitted altogether) Data collection – ignorability, surveys, experiments, observational studies, unintentional missing data
Week 8: general rule-of-thumb advice, connections to other statistical methods, examples of potential pitfalls of Bayesian inference
Part III: Computations
Week 9: overview, use of simulations, Gibbs sampling
Week 10: Markov chain simulation
Part IV: Specific Models (selection and order of topics to be determined)
- Normal linear regression from a Bayesian perspective, Hierarchical linear models, forecasting Presidential elections, Generalized linear models
- Models for robust inference and sensitivity analysis, Student-t model, SAT coaching
- Models for missing data: multivariate normal and t models, multiple imputation, opinion poll
- Mixture models, multivariate models, priors for covariance matrices, hierarchical multivariate models, non-normal data, time series and spartial models
- Non-linear models