At the heart of the Bayesian method is the reliance of probability to model uncertainty and the use of Bayes rule to naturally join past (prior) information and data into posterior predictions. Driven as much by the availability of new computational techniques as by any inherent philosophical advantages, the popularity of the Bayesian paradigm has increased dramatically in the last two decades. The course will begin by introducing the Bayesian approach and inference from first principals. We will then consider sophisticated concepts in Bayesian modeling and model checking, followed by an introduction to practical Bayesian computation. In the last part of the course we will consider more closely specific Bayesian models and applicationS. Throughout the course we will illustrate via real example the three steps of Bayesian statistics: (1) setting up a probability model; (2) updating the fit of the model; (3) evaluating the fit of the posterior. The focus of the course will be an applicative one so that emphasis will be put on techniques rather than the underlying theory. 6-8 problem sets will include theoretical as well as computational exercises. Submission of all problem sets is mandatory and will comprise 20% of the grade just for the (real) effort.

We will be using Bayesian Data Analysis (second edition) by Gelman, Meng, Stern, and Rubin as the main course text. Additional material will be used from the following

- Carlin and Louis, Bayesian Methods for Data Analysis, Third Edition, 2008.
- Bernardo and Smith, Bayesian Theory, 2000.
- Robert, The Bayesian Choice, Second Edition, 2001.
- Berger, Statistical Decision Theory and Bayesian Analysis, 1993.

Part I: Bayesian Inference

Week 1: Introduction via Bayes and Laplace historical examples, Bayes rule, posterior means and variances, binomial model, proportion of female births

Week 2: Standard univariate models including the normal and Poisson models, cancer rates, non-informative prior distributions

Week 3: Multi-parameter models, normal with unknown mean and variance, the multivariate normal distribution, multinomial models, election polling, bioassay. Computation and simulation from arbitrary posterior distributions in two parameters.

Week 4: Inference from large samples and comparison to standard non-Bayesian methods

Part II: Bayesian data analysis

Week 5: Hierarchical models, estimation of population parameters from data, rat tumor rates, SAT coaching experiments, meta analysis

Week 6: Model checking, posterior predictive checking, sensitivity analysis, model comparison and expansion, checking the analysis of the SAT coaching experiments

Week 7: (may be omitted altogether) Data collection – ignorability, surveys, experiments, observational studies, unintentional missing data

Week 8: general rule-of-thumb advice, connections to other statistical methods, examples of potential pitfalls of Bayesian inference

Part III: Computations

Week 9: overview, use of simulations, Gibbs sampling

Week 10: Markov chain simulation

Part IV: Specific Models (selection and order of topics to be determined)

- Normal linear regression from a Bayesian perspective, Hierarchical linear models, forecasting Presidential elections, Generalized linear models

- Models for robust inference and sensitivity analysis, Student-t model, SAT coaching

- Models for missing data: multivariate normal and t models, multiple imputation, opinion poll

- Mixture models, multivariate models, priors for covariance matrices, hierarchical multivariate models, non-normal data, time series and spartial models

- Non-linear models