Course: Advanced Econometrics MPE-2024
Professor: Hedibert Freitas Lopes - www.hedibert.org

Objective: The main objective of this course is to introduce basic aspects i) Statistical Learning and ii) Bayesian Learning, as well as iii) Micro-econometrics, and iv) Macro-econometrics,  that are necessary for the master's degree program.

Brief course description:Regression with endogeneity, regression with measurement error, instrumental variables, potential outcomes, Neyman-Rubin model, selection bias, reverse causality, and omitted variables, panel data, hierarchical models, fixed effect, and random effect, difference-in-differences methods, ARIMA models; long memory; unit root, GARCH models, and stochastic volatility. Vector autoregressive models, factorial models with stochastic volatility, and multivariate models with time-varying parameters, logistic regression; performance metrics for classification, training and testing data; cross-validation, bias-variance trade-off, prior, posterior, and predictive distributions; sequential Bayes and conjugate analysis, Monte Carlo methods.

Teaching assistant: Guilherme Piantino (Doutorando no PhD em Economia dos Negócios)

Final exam:  You will not be asked to write R script or similar during the final exam

Evaluation:  30% final exam, 60% homework assignments (15% cada um), 10% participation

Homework assignments:  Homework can be done by groups of no more than 4 students.

1. HW1  - Due date: October 30th 2024, no later than 7:30pm (Our TA will set it up via blackboard) 
2. HW2 - Due date: November 6th 2024, no later than 7:30pm (Our TA will set it up via blackboard) 
3. HW3- Due date: November 13th 2024, no later than 7:30pm (Our TA will set it up via blackboard)
4. HW4 - Due data: November 22nd 2024, no later than 7:30pm (Our TA will set it up via blackboard) - (data question 1) + (data question 2)

Teaching material

1. Class 1 (09/10): Brief review of basic ingredients: i) Parametric models, likelihoods, estimators and their sampling distributions; ii) Gaussian linear regression: estimation and variable selection; iii) AR(1) model: estimation, unit root, equilibrium distribution; iv) AR(p) model: connection to Gaussian linear regression; v) VAR(1) model: multivariate estimation, matrix notation.
   • Poisson model: maximum likelihood estimation, sufficiency, unbiasedness, consistency, efficiency
   • Comparing estimators: Mean Square Error (MSE)
   • Poisson model: R code for the coal mining disaster data
   • Poisson model: Our first Bayesian experience
   • Gaussian linear regression (pages 1-22) - A few examples
   • Autoregressive model of order one (pages 1-17) - A few examples
   • Complementary bibliography:  Introdução à Inferência Estatística (2010, Heleno Bolfarine & Monica Sandoval) - Sections 1.1.4, 1.3, 2.1, 2.2, 3.1, 3.2.
2. Class 2 (16/10): Introduction to statistical learning: i) Linear and log-linear regression modeling; ii) Training and testing samples; iii) Validation. Bias-variance trade-off.
   • Multiple linear regression
     • CAPM model
     • Return on education
     • Professors salaries
     • Motorcycle example 
   • Logistic regression
     • Graduate school admission (Figures)
   • Gaussian vs Poisson regression: Counts of visitors to a website
   • Fundamentos de Machine Learning (Notas do Prof. Paulo C. Marques F.) - Código R
3. Classes 3+4 (23+30/10): Introduction to Bayesian learning: Prior, posterior, and predictive distributions; Sequential Bayesian updating and conjugate analysis, Bayes factor, posterior model probability, model selection; Monte Carlo & Markov Chain Monte Carlo methods; Sparsity in linear and log-linear models.
   • Bayesian ingredients 
   • Sequential Bayes:  i) Bernoulli trials with Beta prior and ii) Uniform trials with Pareto prior
   • Sequential Bayes:  Bernoulli trials with discrete sucess probability
   • Tiago Mendonca's shiny for the physicists example
   • Phisycists A, B, C and D: Normal model and 4 priors
   • Flipping one of three coins three times and observing three heads (R code)
   • Chapter 1 of "Bayes' Rule: A tutorial Introduction to Bayesian Analysis (Slide)
   •  
   • Bayesian inference for the correlation coefficient of bivariate Gaussian data
   • Bayesian stochastic volatility (SV) model with various error distributions (SV in R: Hosszejni & Kastner, 2021)
   • Bayesian Logistic Regression
   • Bayesian Beta Regression
   • Three examples of HMC/RSTAN in action: iid Gaussian, Gaussian linear regression, and stochastic volatility AR(1) model.
   • Sparse priors:  Brief summaries of Carvalho, Polson and Scott (2010) and Griffin and Brown (2010)
   • Sparse priors:  simulation exercise - graphs
4. Classes 5+6  (06+08/11): Introduction to causal inference: Simpson’s Paradox, Directed Acyclic Graphs (Paths, Junctions, Chains, Forks, Colliders and d-separation), Potential outcome, average treatment effect (ATE), quantile treatment effect (QTE), conditional average treatment effect (CATE), Stable Unit Treatment Value Assumption (SUTVA), Instrumental Variables (IV), Difference in Difference (DiD) and Regression Discontinuity Design (RDD).
   • Notes on causality
   • Example of IV - Card (1995) + dataset
   • Example of DiD - Card and Krueger + Card and Krueger's dataset
   • Example of RDD - Mastering 'Metrics + dataset
   • CausalML Book - Applied Causal Inference: An introduction to the emerging fusion of machine learning and causal inference.
5. Class 7 (13/11): More econometric models: i) General linear models: heteroskedasticity, Student's t errors and autoregressive errors); ii) Hierarchical models; iii) Limited dependent variable models  (Tobit, probit,  ordered probit,  multinomial  probit).
   • General linear models
   • Hierarchical linear models
     • Example from Gamerman and Lopes (2006) - Modeling the nutritional patterns of pregnant women
   • Limited dependent variable models
6. Class 8 (22/11): Introduction to univariate time series econometrics: i) Autoregressive moving average models; ii) Unit root econometrics; iii) Seasonal models; iv) ARCH/GARCH and related models; v) Stochastic volatility models.
   • Autoregressive (AR) models and moving average (MA) models
     • IBM returns, US real GNP business cycles
     • AR(2): h-step-ahead forecasting
   • Unit-root nonstationarity and long-memory processes
     • Random walk, RW with drift, US quarterly GDP, S&P500 returns
   • Seasonal models
     • Modeling monthly totals of international airline passengers from 1949-1960
   • Glossary of ARCH models
   • Stochastic volatility (SV) models
     • Modeling Petrobras returns via SV, SVt, SVl & SVtl
7. Class 9 (27/11): Introduction to multivariate time series econometrics: i) Factor SV models; ii) Dynamic Conditional Correlation (DCC) models; iii) Vector autoregressive (VAR) models, impulses-response funtion, structural VAR (SVAR); iv) Large BVAR, Factor-augmented VAR (FAVAR); v) Time-varying parameter (TVP)-VAR; vi) Bayesian VAR (BVAR) and Bayesian FAVAR (BFAVAR).
   • Factor Stochastic Volatility(FSV)
   • FSV x Dynamic Conditional Correlation (DCC)
   • Vector autoregressive models (VAR)
   • Large BVAR, FAVAR, TVP-BVAR & BFAVAR
8. Class 10 (11/12): Final exam

Basic bibliography

• Mostly Harmless Econometrics: An Empiricist's Companion (Angrist and Pischke, 2009)
• Analysis of Financial Time Series, 3rd Edition (Tsay, 2010)
• An Introduction to Statistical Learning (James, Witten, Hastie and Tibshirani, 2023) – https://www.statlearning.com
• Introduction to Bayesian Econometrics (Greenberg, 2013)

Additional bibliography

• Introduction to Econometrics, 3^rd edition (Stock and Watson, 2010)
• Introductory Econometrics: A Modern Approach (Wooldridge, 2012)
• Time Series Analysis (Hamilton, 1994)
• Aprendizado de Máquina: Uma Abordagem Estatística (Izbicki and Mendonça, 2020) - https://tiagoms.com/publications/ame
• Estatística e Ciência de Dados (Morettin and Singer, 2021) -https://www.ime.usp.br/~pam/cdadosf3.pdf
• Introduction to Modern Bayesian Econometrics (Lancaster, 2004)
• Bayesian Econometric Methods, 2a edição (Chan, Koop, Poirier and Tobias)
• Bayesian Statistics and Marketing (Rossi, Allenby and McCulloch, 2005)
• Time Series: Modeling, Computation, and Inference (Prado and West, 2010)

Course: BAYESIAN LEARNING  - Professional Master in Economics - 2024

Professor: Hedibert Freitas Lopes - www.hedibert.org (hedibertfl@insper.edu.br)

Teaching assistant: Luiza Tuler Veloso (luizatv@insper.edu.br)

Syllabus: The ultimate goal of this course is to enable graduates to critically decide between the classical or Bayesian approach, or a combination of both, when faced with real-world decision-making problems under uncertainty. Areas where these real-world problems arise, as examples discussed throughout the course, include microeconomics, macroeconomics, finance, quantitative marketing, among many others.  With this objective in mind, we will study the basic ingredients of the Bayesian paradigm: formulation of the binomial model-prior, model comparison and combination, computational aspects, and Bayesian decision-making. In the second part of the course, the Bayesian approach to traditional linear regression and logistic regression models will be introduced, as well as their modern versions where priors are treated as regularization mechanisms and sparsity inducers. Sparsity will be present throughout the 2nd and 3rd parts of the course when dealing with highly dimensional and/or highly complex models.  In the third and final part of the course, we will present several statistical models currently used for this purpose, such as mixture models, hierarchical models, factor models, and regression tree models, as well as models based on neural networks and models that use texts and documents as data (text modeling). All, it is worth mentioning, under the unified and coherent Bayesian approach. All calculations during the course will be performed using the R statistical package.

Homework assignments: HW1 + HW2 + HW3 + HW4

Final project - paper presentation

Additional examples that were developed/discussed in class

• Class of April 23rd 2024:
  • Markowitz's minimum variance portfolio: Static vs Time-varying covariance estimation
  • Salaries for Professors: Bayesian linear regression
  • Graduate school admission: Bayesian logistic regression (Figures)
• Class of April 30ht 2024:
  • Counts of visitors to a website: Gaussian regression vs Poisson regression
• Class of May 7th 2024:
  • Banana-shaped posterior:  an introduction to MC via SIR
  • Banana-shaped posterior:  Implementing a MCMC algorithm
• Class of May 14th 2024:
  • Mais uma explicação do algoritmo SIR para auxiliar a inferência Bayesiana
  • Gaussian linear regression & Beta regression (dataset + graphical output)
  • Frequentist and Bayesian GLM with logit link (dataset + graphical output)
• Class of May 21th 2024:
  • Student's t model:  SIR - RWM - GIBBS (graphical output)
  • Markov chain & Gaussian linear regression (Gibbs sampler)
• Class of May 28th 2024:
  • Bayesian Gaussian linear regression with shrinkage/sparsity-inducing priors
• Class of June 7th 2024:
  • Inducing shrinkage and selection via Laplace, horseshoe & normal-gamma densities
• Class of June 11th 2024:
  • Modeling manganese prices/returns: Stochastiv volatility modeling (weekly prices) (Hosszejni & Kastner, 2021)
  • Modeling Petrobras returns: Stochastiv volatility with t-errors and leverage effect (daily prices)

Course notes (+ R code & references)

• Bayesian ingredients  
  • Bayesian update: a toy example
  • Flipping one of three coins three times and observing three heads (R code)
  • Chapter 1 of "Bayes' Rule: A tutorial Introduction to Bayesian Analysis (Slide)
  • Tiago Mendonca's shiny for the physicists example
  • Phisycists A, B, C and D: Normal model and 4 priors

• Bayesian computation
  • Banana-shape posterior: Posterior inference via SIR 
  • Learning the number of degree of freedom of a Student's t  (Rmarkdown)
  • Bayesian regression with the normal-gamma (NG) prior
  • A bit of Monte Carlo simulation and integration
  • Back to the physicists example
  • Normal vs skew-normal distributions: an exercise in Bayesian learning
  • The potential fallacy of using the prior as proposal in SIR algorithms
  • Bayesian learning of correlation in bivariate normal data via SIR algorithm
  • Smith and Gelfand (1992) Bayesian Statistics without Tears: A Sampling-Resampling Perspective. The American Statistician, 46(2), 84-88.

• Bayesian linear regression
  • CAPM regression
  • Return on education:  Gaussian linear model with conjugate prior
  • Boston housing data: ML and Bayesian inference
  • Motorcycle example
  • Regressing weight on height: SIR for Gaussian and Student's t models
  • Gibbs sampler in action: Bayesian linear regression
  • Bayesian regularization
  • iid Bernoulli or logit regression or probit regression? (Rmarkdown)
  • Bayesian Poisson regression model versus i.i.d. Poisson model
  • Another example of Poisson regression
  • A few R packages for Bayesian inference in linear models
  • Stan/rstan: Bayes Sparse Regression

• Bayesian classification via logistic regression
  • Sparse logistic regression for the spam/ham dataset (data)
  • Marketing campaigns of a Portuguese banking institution (data)
  • Sparse logistic regression: comparison of regularization and Bayesian implementations
  • Gelman, Jakulin, Pittau and Zu (2008) A weakly informative default prior distribution for logistic and other regression models, AOAS, 2(4), 1360-1383.
  • Polson, Scott and Windle (2013) Bayesian Inference for logistic models using Pólya-Gamma latent variables, JASA, 108, 1339-1349.

• Other important modeling structures
  • Factor models (Additional material)
    • Indice de Desenvolvimento Humano
    • US market (data)
    • Bayesian factor models
    • Reducao de ruido com PCA e FA
  • Time-varying variance/covariance
    • Stochastic volatility
    • Factor Stochastic Volatility(FSV)
    • FSV x Dynamic Conditional Correlation (DCC)
  • Finite mixture of distributions

• Machine Learning 1: Tree models
  • Classification and regression trees (CART)
    • CART step-by-step for the Boston housing data
    • CART for the motorcycle data
    • ROC and AUC
  • Bayesian CART
  • Bootstrap aggregating  (bagging)
    • Ilustracao do bootstrap
  • Boosting (weak/stronger learners)
  • Random forests
    • Predicao da covid-19 com dados publicos do Hospital Israelita Albert Einstein/SP
  • Bayesian additive regression trees (BART) + (Rob's notes) + (a list of slides/examples/people/papers)

• Machine Learning 2: Modeling text
  • Latent Dirichlet Allocation (LDA)
  • Short list of slides and papers
  • Example 1 & 2: Twitter & BBC
  • Example 3: Dickens, Wells, Verne or Austen?

• Machine Learning 3:  Neural nets
  • Neural Networks
  • short list of slides and papers

Additional supporting material

• Stan/rstan for posterior inference: Hamiltonian MC (HMC) methods - by Hedibert Lopes (February 2021)
• MC and MCMC: Key References - by Hedibert Lopes (February 2021)
• R packages for Bayesian linear regression - by Hedibert Lopes (February 2020)
• R packages for Bayesian Econometrics - by Hedibert Lopes (March 2014)
• CRAN Task View on Bayesian Inference (July 2023)
• Mathematics for Machine Learning - by Deisenroth, Faisal and Ong (2020)
• Conceitos e analises estatisticas com R e JASP - by  Luis Anunciação (September 2021)
• Data Science, Marketing and Business by Pedro Fernandes & Paulo Marques (October 2019)
• Aprendizado de Máquina: Uma Abordagem Estatística (by Rafael Izbicki & Tiago Mendonça)
• Estatística e Ciência de Dados (by Pedro Morettin & Julio Singer)

Course: ADVANCED BAYESIAN ECONOMETRICS PhD-2024
Professor: Hedibert Freitas Lopes - www.hedibert.org

Objective: The end of the course goal is to allow the student to critically decide between a Bayesian, a frequentist or Bayesian-frequentist compromise when facing real world problems in the fields of micro- and macro-econometrics and finance, as well as in quantitative marketing, strategy and business administration.  With this end in mind, we will visit well known Bayesian issues, such as prior specification and model comparison and model averaging, but also study regularization via Bayesian LASSO, Spike-and-Slab and related schemes, “small n, large p” issues, Bayesian statistical learning via additive regression trees, random forests, large-scale VAR and (dynamic) factor models.

Course description: Basic ingredients: prior, posterior, and predictive distributions, sequential Bayes, conjugate analysis, exchangeability, principles of data reduction and decision theory.  Model criticism: Bayes factor, computing marginal likelihoods, Savage-Dickey ratio, reversible jump MCMC, Bayesian model averaging and deviance information criterion.  Modern computation via (Markov chain) Monte Carlo methods: Monte Carlo integration, sampling-importance resampling, Gibbs sampler, Metropolis-Hastings algorithms.  Mixture models, Hierarchical models, Bayesian regularization, Instrumental variables modeling, Large-scale (sparse) factor modeling, Bayesian additive regression trees (BART) and related topics, Dynamic models, Sequential Monte Carlo algorithms, Bayesian methods in microeconometrics, macroeconometrics, marketing and finance.

• Part I Bayesian ingredients: i) Inference: likelihood, prior, predictive and posterior distributions; ii) Model criticism: Marginal likelihoods, Bayes factor, model averaging and decision theory; and iii) Computation: An introduction (Markov chain and sequencial) Monte Carlo methods.
• Part II Multivariate models: i) Large-scale vector autoregressive models; ii) Factor models and other dimension reduction models; and iii) Time-varying high-dimensional covariance models.
• Part III Modern Bayesian statistical learning: i) Mixture models and the Dirichlet process: handling non-Gaussian models; ii) Regularization: sparsity via shrinkage and variable selection; iii) Large vector-autoregressive and factor models: combining sparsity and parsimony; iv) Classification and support vector machines; v) Regression trees and random forests; and vi) Latent Dirichlet allocation: Text as data, text mining.

Take-home midterm exam:  Start: 10am, October 5th, 2024 - End: 10pm, October 7th, 2024 (60 hours later!) - Derivations + R code

Paper presentations:  List of papers - On November 12, 2024, between 9 AM and 12 PM, 10 presentations will be given, each lasting no less than 10 minutes and no more than 15 minutes. On the same day, and no later than 9 AM, a PDF summary of 5 to 7 pages must be submitted directly to me via my institutional email hedibertfl@insper.edu.br.

Homework assignments

• First homework assignment: iid Bernoulli-mixture of Beta prior (Due date: September 3rd, 2024, 9am) - Posterior derivations + R code
• Second homework assignment: MC integration and sampling (Due date: September 10th, 2024, 9am) - Solution in R
• Third homework assignment: Nonlinear Gaussian regression (Due date: September 17th, 2024, 9am)
• Quiz: Poisson-Gamma model (Solution) - September 24th 2024, 10am-11am
• Fourth homework assignment: AR(1) plus noise NDLM (Due date: Monday, September 30th, 2024, 9am) - Full conditional distributions - R code

Examples developed in class

• Class 1 - August 27th, 2024: Basic Bayes via iid Bernoulli vs logit Bernoulli models
• Class 2 - September 3rd, 2024: Generalized linear model (logit, probit, cloglog) - dose-response trials
• Class 3 - September 10th, 2024:
  • Gaussian linear regression model: Conjugate analysis (Example + R code)
  • Sampling importance resampling (SIR)
  • Banana-shaped posterior inference via SIR 
  • Example from Smith and Gelfand (1992)
  • Examples 3.6 and 3.7 from my book MCMC: Stochastic Simulation for Bayesian Inference (Gamerman and Lopes, 2006)
  • Smith and Gelfand (1992) Bayesian Statistics without Tears: A Sampling-Resampling Perspective. The American Statistician, 46(2), 84-88
  • Newton and Rafter (1994) Approximate Bayesian inference with weighted likelihood bootstrap.  JRSS-B, 56(1), 3-48
• Class 4 - September 17th, 2024:
  • Normal vs skew-normal distributions
  • Bayesian inference for the correlation coefficient of bivariate Gaussian data
  • Simple linear regression with Gaussian, Student's t or Laplace errors
  • Markov chain & MCMC-based posterior inference for the Gaussian linear regression via Gibbs sampler
  • Student's t model:  SIR, random walk Metropolis & Gibbs sampler (graphical output)
  • Bayesian linear regression: our first implementation of an MCMC algorithm (wnba/nba data)
  • Bayesian GARCH
  • Bayesian stochastic volatility (SV) model with various error distributions (SV in R: Hosszejni & Kastner, 2021)
• Class 6 - October 1st, 2024:
  • Part I.5 - Statistical Decision Theory (Travel insurance example) - see lecture notes below
  • Dynamic linear regression
  • Steve Scott’s BSTS tutorial (my own shorter tutorial)
  • BSTS package for state-space modeling of NO3 (Data)
  • Normal dynamic linear model as a multivariate normal (derivations)
    • Chan and Jeliazkov (2009) Efficient simulation and integrated likelihood estimation in state space models, Int. J. Mathematical Modelling and Numerical Optimisation, 1(1/2), 101-120.
    • McCausland, Miller and Pelletier (2011) Simulation smoothing for state space models: A computational efficiency analysis, Computational Statistics and Data Analysis, 55, 199-212.

•   Class 7 - October 8th, 2024:
  • Three examples of HMC/RSTAN in action: iid Gaussian, Gaussian linear regression, and stochastic volatility AR(1) model.
•   Class 8 - October 15th, 2024:
  • Sparse priors:  Brief summaries of Carvalho, Polson and Scott (2010) and Griffin and Brown (2010)
  • Sparse priors:  simulation exercise - graphs
  • Stock and Watson's (2002) US Industrial Production: ridge, lasso and horseshoe priors
  • Predicting wages: In-sample vs out-of-sample MSE/MAE - R code - wage data
  • Bhadra, Datta, Polson and Willard (2019) Lasso Meets Horseshoe: A Survey, Statistical Science, 34(3), 405-427.
•   Class 9 - October 22nd, 2024:
•   Class 10 - October 29th, 2024:
•   Class 11 - November 5th, 2024:
•   Class 12 - November 12th, 2024:  Final presentations

LECTURE NOTES

PART I: Bayesian ingredients

1. Basic Bayes
2. Exchangeability
3. Principles of data reduction
4. More on estimators
5. Decision theory (Nuisance parameters + travel insurance example)
   • Decision Theory: Principles and Approaches, by Parmigiani and Inoue (with contributions by Lopes), 2009, Wiley. (TOC)
6. Bayesian model criticism (pages 1-6 & 32-34)
7. Additional reading material:
   • Chapter 2 of Gamerman and Lopes (2006) - Compact, but easy to read.
   • Chapters 2-4 of Migon, Gamerman and Louzada (2014) - Integrates classical and Bayesian inference.
   • Chapter 1 and 2 of Gelman et al. (2013) - Application-oriented.
   • Chapter 4 (Sections 4.1-4.4) of Berger (1985) – More technical.
   • van de Schoot, R., Depaoli, S., King, R. et al. Bayesian statistics and modelling. Nat Rev Methods Primers 1, 1 (2021). 
8. Discussion about p-values

PART II: Bayesian Computation

1. Monte Carlo (MC) methods
2. Markov chain: a brief review
3. Markov chain Monte Carlo (MCMC) algorithms
4. Hamiltonian Monte Carlo: A toy example
5. Stan/rstan for posterior inference: Hamiltonian MC (HMC) methods
6. MC and MCMC: Key References
7. More on Bayesian model criticism

PART III: Bayesian Learning

1. Fundamentos de Aprendizagem Estatística + R code + MC exercise
2. Multiple linear regression: selection, shrinkage, sparsity
   • Motorcycle example
   • Slides from the 2015 School of Time Series and Econometrics tutorial
   • Hahn, He and Lopes (2018) Gaussian linear regression with arbitrary sparsity + slides of a talk + R package bayeslm
   • Fava and Lopes (2021) The illusion of the illusion of sparsity + slides of a talk + UFPE webinar (62' and forward)
   • van Erp, Oberski and Mulder (2019) Shrinkage priors for Bayesian penalized regression, Journal of Mathematical Psychology, 89, 31-50.
   • Michael Betancourt's Bayes Sparse Regression (stan/rstan example)
3. Classification: logistic regression and discriminant analysis
   • Sparse logistic regression for the spam/ham dataset (data)
   • Marketing campaigns of a Portuguese banking institution (data)
   • Sparse logistic regression: comparison of regularization and Bayesian implementations
   • Gelman, Jakulin, Pittau and Zu (2008) A weakly informative default prior distribution for logistic and other regression models, AOAS, 2(4), 1360-1383.
   • Polson, Scott and Windle (2013) Bayesian Inference for logistic models using Pólya-Gamma latent variables, JASA, 108, 1339-1349.
4. Multivariate models and dimension reduction
   • Bayesian factor analysis (BFA)
   • BFA in R: a simple illustration (additional routines)
   • Factor models: an annotated bibliography (Lopes, 2003) + review chapter (Lopes, 2014)
   • Principal components analysis (PCA), PCA-based and FA-based regressions
   • Probabilistic PCA + Rcode + data
5. Classification and regression trees (CART)
   • I highly recommend checking out (studying!) the slides of the Machine Learning course by Paulo Orenstein, Assistant Professor at IMPA. 
   • Chapter 12 and Chapter 13 are about "Tree-based methods". 
   • Also, Chapter 15 and Chapter 16 are about "Deep learning".
6. Bayesian CART
7. Bootstrap aggregating (bagging)
   • Bootstrap illustration
   • CART step-by-step for the Boston housing data
   • CART for the motorcycle data
8. Bayesian additive regression trees (BART)
   • Example 1: ICU data: CART, BART and random forest (R code)
   • Example 2: Stock and Watson’s (2002) macro data (data)
   • More examples: Four BART applications & 2 reviews + cute CART trees and 3D plots
   • More recent references
9. Latent Dirichlet Allocation (LDA)
   • Twitter + BBC + Dickens, Wells, Verne or Austen? + The cat in the hat by Dr. Seuss
10. Neural Networks

Complementary material to PART III

1. Boosting (weak/stronger learners)
2. Random forests
3. Bayesian instrumental variables
4. General linear and hierarchical models
5. Limited dependent variable models
6. Finite mixture of distributions
7. Spatial models
8. P.Richard Hahn's top 25 books on Statistics, Causal Inference, Statistical Computing, Machine Learning and Data Science

Bibliography: Bayesian econometrics

1. Zellner (1971) An Introduction to Bayesian Inference in Econometrics
2. Goel and Iyngar (1992) Bayesian Analysis in Statistics and Econometrics
3. West and Harrison (1997) Bayesian Forecasting and Dynamic Models (2nd edition)
4. Dorfman (1997) Bayesian Economics Through Numerical Methods
5. Bauwens, Lubrano and Richard (2000) Bayesian Inference in Dynamic Econometric Models
6. Koop (2003) Bayesian Econometrics
7. Geweke (2005) Contemporary Bayesian Econometrics and Statistics
8. Lancaster (2004) Introduction to Modern Bayesian Econometrics
9. Rossi, Allenby and McCulloch (2005) Bayesian Statistics and Marketing
10. Prado and West (2010) Time Series: Modeling, Computation and Inference
11. Geweke, Koop and Van Dijk (2011) The Oxford Handbook of Bayesian Econometrics
12. Greenberg (2013) Introduction to Bayesian Econometrics
13. Herbst and Schorfheide (2015) Bayesian Estimation of DSGE Models
14. Chan, Koop, Poirier and Tobias (2019) Bayesian Econometric Methods (2nd edition)
15. Broemeling (2019) Bayesian Analysis of Time Series
16. Bernardi, Grassi and Ravazzolo (2020) Bayesian Econometrics

Bibliography: Bayesian statistics

1. Berger (1985) Statistical Decision Theory and Bayesian Analysis
2. Bernardo and Smith (2000) Bayesian Theory
3. Gelman and Hill (2006) Data Analysis Using Regression and Multilevel/Hierarchical Models
4. Robert (2007) The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation
5. Hoff (2009) A First Course in Bayesian Statistical Methods
6. Carlin and Louis (2009)  Bayesian Methods for Data Analysis (3rd edition)
7. Gelman, Carlin, Stern, Dunson, Vehtari and Rubin (2016) Bayesian Data Analysis
8. Migon, Gamerman and Louzada (2015) Statistical Inference: An Integrated Approach (2nd edition)
9. Reich and Ghosh (2019) Bayesian Statistical Methods
10. Held and Sabanes-Bove (2020) Likelihood and Bayesian Inference: With Applications in Biology and Medicine

Bibliography: Bayesian computation

1. Gilks, Richardson and Spiegelhalter (1995) Markov Chain Monte Carlo in Practice
2. Doucet, de Freitas and Gordon (2001) Sequential Monte Carlo Methods in Practice
3. Robert and Casella (2004) Monte Carlo Statistical Methods (2nd edition)
4. Gamerman and Lopes (2006) MCMC: Stochastic Simulation for Bayesian Inference, Second Edition
5. Marin and Robert (2007) Bayesian Core: A Practical Approach to Computational Bayesian Statistics
6. Albert (2009) Bayesian Computation with R
7. Brooks, Gelman, Jones and Meng (2011) Handbook of Markov Chain Monte Carlo
8. Givens and Hoeting (2012) Computational Statistics (2nd edition)
9. Marin and Robert (2014) Bayesian Essentials with R (complete solution manual)
10. Turkman, Paulino and Mueller (2019) Computational Bayesian Statistics: An Introduction
11. McElreath (2020) Statistical Rethinking: A Bayesian course with Examples in R and STAN
12. Chopin and Papaspiliopoulos (2020) An Introduction to Sequential Monte Carlo

Bibliography: (Bayesian) statistical learning

1. Bishop (2006) Pattern Recognition and Machine Learning
2. Hastie, Tibshirani and Friedman (2008) The Elements of Statistical Learning, 2nd edition
3. Murphy (2012) Machine Learning: A Probabilistic Perspective
4. Barber (2012) Bayesian Reasoning and Machine Learning
5. James, Witten, Hastie and Tibshirani (2013) An Introduction to Statistical Learning
6. Hastie, Tibshirani and Wainwright (2015) Statistical Learning with Sparsity
7. Efron and Hastie (2016) Computer Age Statistical Inference: Algorithms, Evidence and Data Science
8. Fernandez and Marques (2018) Data Science, Marketing and Business
9. Izbicki & Santos (2020) Aprendizado de máquina: uma abordagem estatística

Bibliography: Classical Monte Carlo papers

• Metropolis and Ulam (1949) The Monte Carlo method. JASA, 44, 335-341.
• Metropolis, Rosenbluth, Rosenbluth, Teller and Teller (1953) Equation of state calculations by fast computing machines. Journal of Chemical Physics, 21, 2087-1092.
• Hastings (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97-109.
• Peskun (1973) Optimum Monte Carlo sampling using Markov chains. Biometrika, 60, 607-612.
• Besag (1974) Spatial Interaction and the Statistical Analysis of Lattice Systems. JRSS-B, 36, 192-236.
• Kirkpatrick, Gelatt and Vecchi (1983) Optimization by Simulated Annealing. Science, 220 (4598), 671-680.
• Geman and Geman (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Analysis and Machine Intelligence, 6, 721-741.
• Pearl (1987) Evidential reasoning using stochastic simulation of causal models. Artificial intelligence, 32, 245-257.
• Tanner and Wong (1987) The Calculation of Posterior Distributions by Data Augmentation. JASA, 82, 528-540.
• Geweke (1989) Bayesian Inference in Econometric Models Using Monte Carlo Integration. Econometrica, 57, 1317-1339.
• Gelfand and Smith (1990) Sampling-Based Approaches to Calculating Marginal Densities. JASA, 85, 398-409.
• Casella and George (1992) Explaining the Gibbs Sampler. The American Statistician, 46, 167-174.
• Gilks and Wild (1992) Adaptive Rejection Sampling for Gibbs Sampling. Applied Statistics, 41, 337-348.
• Smith and Gelfand (1992) Bayesian Statistics without Tears: A Sampling-Resampling Perspective. The American Statistician, 46, 84-88.
• Chib and Greenberg (1995) Understanding the Metropolis-Hastings algorithm. The American Statistician, 49, 327-335.