Time Series - HW2

Regression Model With Autocorrelated Errors

1 The data

Consider the following Gaussian and linear regression model: \[ y_t = \alpha+\beta x_t + u_t \] where \(u_t\) follows a Gaussian white noise process with variance one, i.e. \(u_t \sim N(0,\sigma^2)\).

x=c(3.0,3.9,6.0,4.2,5.2,4.7,5.1,4.5,6.0,3.9,4.1,2.2,1.7,2.7,3.3,4.8)
y=c(9.500,12.649,18.794,12.198,14.372,13.909,14.556,14.700,18.281,13.890,10.318,5.473,4.044,6.361,7.036,13.368)
n=length(x)

1. Fit the above regression via ordinary least squares (OLS) and discuss your findings
2. Plot the fitted residuals \({\hat u}_t=y_t-{\hat \alpha}+{\hat \beta} x_t\) and discuss your findings. For instance, do the residuals look iid?

2 Likelihood and prior

Let us assume we would like to entertain a more elaborated model. More precisely, let us consider a simple and Gaussian linear regression model where the errors follow a first order autogressive structure, where, for \(t=1,\ldots,n\), \[\begin{eqnarray*} y_t &=& \beta x_t + u_t\\ u_t &=& \rho u_{t-1} + \epsilon_t \end{eqnarray*}\] where \(\epsilon_t\)s follows a Gaussian white noise process with variance one.

3. Show that the above model can be written as \[ y_t = \rho y_{t-1} + \beta (x_t-\rho x_{t-1}) + \epsilon_t. \] So, for the sake of the exercise, assume that the likelihood is conditional on \(y_1\) and \(x_1\). Therefore \[ L(\beta,\rho|\mbox{data}) \equiv p(y_{2:n}|x_{2:n},\theta) \propto \exp\left\{-\frac{1}{2} \sum_{t=2}^n (y_t - \rho y_{t-1} - \beta (x_t-\rho x_{t-1}))^2 \right\}, \] and assume that the prior for \((\beta,\rho)\) is noninformative \[ p(\beta,\rho) \propto 1. \] This is an improper prior, but the implied posterior is proper as long as \(n \geq 3\).

4. Plot \(L(\beta,\rho|\mbox{data})\) for \((\beta,\rho) \in (-5,5) \times (-5,5)\). Comment the behavior of the likelihood for \(\beta\) for values of \(\rho=-1,0.5,1\).

3 Posterior

5. Use the sampling importance resampling algorithm to obtain draws \[ \left\{(\beta^{(i)},\rho^{(i)})\right\}_{i=1}^N \sim p(\beta,\rho|\mbox{data}) \propto L(\beta,\rho|\mbox{data}). \] Suggestion: Use the uniform in the square \((-5,5) \times (-5,5)\) as candidate distribution. Try \(M=1,000\), \(M=10,000\) and \(M=100,000\) and discuss your findings

6. The scatterplot of the posterior draws of \(\beta\)s vs \(\rho\)s resembles the contours of the likelihood \(L(\beta,\rho|\mbox{data})\)? Comment your findings.