Bayesian econometrics


Bayes factor, computing marginal likelihoods, Savage-Dickey ratio, reversible jump MCMC, Bayesian model averaging and Deviance information criterion

Bayesian Model criticism

1. Prior and posterior model probabilities, marginal likelihood and Bayes factor

2. Computing the normalizing constant p(y)

  • · Laplace-Metropolis estimator
  • · Simple Monte Carlo estimator
  • · Monte Carlo estimator via importance function
  • · Annealed importance sampling estimator
  • · Brigde sampling estimator
  • · Path sampling estimator
  • · Chib’s estimator
  • · Chib and Jeliazkov’s estimator

3. Savage-Dickey density ratio

4. Trans-dimensional MCMC algorithms

  • · Green’s (1995) RJMCMC
  • · Carlin and Chib’s (1995) pseudo-priors
  • · Godsill’s (2001) partial analytic RJMCMC
  • · Delllaportas et al’s (2002) Metropolized Carlin-Chib

5.   Bayesian model averaging (BMA)

6.   Deviance information criterion (DIC)


The binomial regression revisited
(R code)

A probit regression (R code)

Bayesian multiple linear regression (R code) + (college.txt)


Additional links


A few additional references

  • · Newton, M. A. and Raftery, A. E. (1994) Approximate Bayesian inference by the weighted likelihood bootstrap (with discussion). Journal of the Royal Statistical Society, Series B, 56, 3-48.
  • · Chib, S. (1995) Marginal likelihood from the Gibbs output. Journal of the American Statistical Association}, 90, 773-95.
  • · Kass, R. E. and Raftery, A. E.(1995) Bayes factors. Journal of the American Statistical Association, 90, 773-95.
  • · Meng, X. L. and Wong, W. H.(1996) Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Statistica Sinica, 6, 831-60.
  • · DiCiccio, T. J., Kass, R. E., Raftery, A. E. and Wasserman, L. (1997) Computing Bayes factors by combining simulation and asymptotic approximations. Journal of the American Statistical Association, 92, 903-15.
  • · Gelman, A. and Meng, X. L.(1998) Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statistical Science, 13, 163-85.
  • · Gelfand, A. E. and Dey, D. K. (1994) Bayesian model choice: asymptotics and exact calculations. Journal of the Royal Statistical Society, Series B, 56, 501-14.
  • · Chib, S. and Jeliazkov, I. (2001) Marginal likelihood from the Metropolis-Hastings output. Journal of the American Statistical Association, 96, 270-81.
  • · Neal, R. M. (2001) Annealed importance sampling. Statistics and Computing, 11, 125-39.
  • · Carlin, B. P. and Chib, S. (1995) Bayesian model choice via Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 57, 473-84.
  • · Green, P. J. (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82, 711-32.
  • · Verdinelli, I. and Wasserman, L. (1995) Computing Bayes factor using a generalization of the Savage-Dickey density ratio, JASA, 90, 614-8.
  • · Raftery, Madigan and Hoeting (1997) Bayesian Model Averaging for Linear Regression Models, JASA, 92, 179-191.
  • · Hoeting, Madigan, Raftery and Volinsky (1999) Bayesian Model Averaging, Statistical Science, 14, 382-401.
  • · Godsill, S. J. (2001) On the relationship between Markov chain Monte Carlo methods for model uncertainty. Journal of Computational and Graphical Statistics, 10, 1-19.
  • · Dellaportas, P., Forster, J. and Ntzoufras, I. (2002) On Bayesian model and variable selection using MCMC. Statistics and Computing, 12, 27-36.
  • · Spiegelhalter, D. J., Best, N.G., Carlin, B.P., and van der Linde, A. (2002) Bayesian measures of model complexity and fit (with discussion and rejoinder), Journal of
    the Royal Statistical Society, Series B
    , 64, 583-639.
  • · Chapter 11 of Gary Koop’s book (Bayesian Econometrics, Wiley,2003)on “Bayesian Model Averaging” gives a detailed introduction of BMA in
    linear regression models.
  • · Lopes, H. F. and West, M. (2004) Bayesian Model Assessment in Factor Analysis. Statistica Sinica, 14, 41-67.
  • · van der Linde, A. (2005) DIC in variable selection. Statistica Neerlandica, 59, 45-56.