PS1 Resolution

Exercise 1

library("astsa")

## Warning: package 'astsa' was built under R version 4.0.3

plot.ts(EQ5, main="Ex 1",ylab='units')+
lines(EXP6, col=4)

## integer(0)

In the graph above, the blue line denotes Explosion, while the black line is Earthquake As we can see, in some moments of time, the variance in blue line is lesser than in the black line. But when these variances increases, the blue line tends to increse a bit more.

Exercise 2

a)

w = rnorm(200)
s = c(rep(0,100), 10*exp(-(1:100)/20)*cos(2*pi*101:200/4))
x= s+w
plot.ts(x)

b)

w = rnorm(200)
s = c(rep(0,100), 10*exp(-(1:100)/200)*cos(2*pi*101:200/4))
x= s+w
plot.ts(x)

c)

x1 = exp(-(1:100)/20)
x2 = exp(-(1:100)/200)
plot.ts(x1)+
lines(x2, col=4)

## integer(0)

I the graph above the blue line shows \(\exp-t/200\) while the black line represents \(\exp-t/20\)

In both graphs, we have a decreasing term multiplying a cyclical one. The cyclical component is the same in both graphs, but the first one (letter a), has a decreasing component that decreases much faster than the second; after a while, the shock to the standard normal deviation is almost imperceptible, and thus the graph comes back to be almost a standard normal. The second graph has a decreasing component that decreases much slower, and thus, we just observe a decrease in the amplitude of the cyclical component, plus a standard normal.

Exercise 3

a)

w = rnorm(150,0,1) # 50 extra to avoid startup problems
x = filter(w, filter = c(0,-.9), method="recursive")[-(1:50)] # remove first 50
v = filter(x, rep(1/4, 4), sides = 1)
plot.ts(x, main="autoregression")+
lines(v,lty=2, col = 4, lwd=3)

## integer(0)

Xt is a AR(2) process in such way that the value of X in moment t is much similar to its value in t-2, but with the opposite signal. Hence, we have a seasonal pattern of 2 positive values followed by 2 negative ones When we input the moving average filter, we have that the variance of the series decreases, since we are almost always summing two positive and two negative values

b)

x = cos(pi*1:100*2/4)
v = filter(x, rep(1/4, 4), sides = 1)
plot.ts(x, main="Cos (2*pi*t/4)")+
lines(v,lty=2, col = 4, lwd=3)

## integer(0)

In this case, xt is just a cyclal, that repeats after 4 observations, in such way that, if if take the average of 4 consecutive observations, this average is 0.

c)

set.seed(124)
w = rnorm(100,0,1) 
x = cos(pi*1:100*2/4) + w
v = filter(x, rep(1/4, 4), sides = 1)
plot.ts(x, main="Cos (2*pi*t/4) + wt")+
lines(v,lty=2, col = 4, lwd=3)

## integer(0)

Since the cosin function cycles around 0, x is almost an average between an cosin function and a white noise. Thus, the moving average, in this case, is just the average of the white noise, since the average of the cosin is 0.

d)

From b, the moving average is just 0, since xt is cyclical around 0 with a periodicity of 4. Thus, in c, the moving average is just the average of a white noise. In a, e have a similar result to c, since the moving average is almost the average of a white noise.

Exercise 8

\[ x_{t} = \delta + x_{t-1} + w_{t} \]

a)

We must prove the following \[ x_{t} = \delta t+\sum_{k=1}^{t}w_{k} \] Let’s prove by induction. First, for t=1: \[ \begin{align} x_1 &= \delta + x_0 + w_1 \\ & = \delta1 + \sum_{k=1}^{1}w_k \end{align} \]

Now, suppose \[ x_{t} = \delta t+\sum_{k=1}^{t}w_{k} \] and let’s prove for t+1

\[ \begin{align} x_{t+1} &= \delta + x_{t} + w_{t+1}\\ & = \delta +(\delta t+\sum_{k=1}^{t}w_{k}) + w_{t+1} & \text{(induction hypothesis)} \\ & = \delta (t+1) + \sum_{k=1}^{t+1}w_k \end{align} \]

b)

\[ x_{t} = \delta t+\sum_{k=1}^{t}w_{k}\\ \] The mean function is given:

\[ \begin{align} E[x_t] &= E[\delta t+\sum_{k=1}^{t}w_{k}] \\ &= \delta t+\sum_{k=1}^{t}E[w_{k}] \\ &= \delta t \end{align} \]

For the autocovariance function, lets first define the variance of \(x_t\):

\[ \begin{align} V(x_t) &= V(\sum_{k=1}^{t}w_{k}) \\ &= \sum_{k=1}^{t}V(w_{k}) & \text{since wt are i.i.d}\\ &=t\sigma_w^2\end{align}\]

Now, for the autocovariance, suppose, that s<t. We have:

\[ \begin{align} \gamma _x(s,t) = cov(x_s,x_t) &= cov(x_s, x_s+ \delta + w_{s+1} + \delta + w_{s+2} + ... + \delta + w_t ) \\ &= cov(x_s, x_s) + cov(x_s, w_{s+1}) + cov(x_s, w_{s+2}) + ... + cov(x_s, w_t) \\ &= cov(x_s,x_s) \\ &= V(x_s) \\ &= s\sigma_w^2 \end{align} \]

In general terms, we have: \[ \gamma _x(s,t) = cov(x_s,x_t) = min\{s,t\}\sigma_w^2 \]

c)

\(x_t\) is not stationary because both its mean and variance depend on time.

d)

Autocorrelation function is given by: \[ \rho_x(s,t) = \frac{\gamma_x(s,t)}{\sqrt{\gamma(s,s)\gamma(t,t)}} \] In particular, for \(\rho(t,t-1)\), we have:

\[ \begin{align} \rho_x(t-1,t) &= \frac{\gamma_x(t-1,t)}{\sqrt{\gamma(t-1,t-1)\gamma(t,t)}}\\ &=\frac{(t-1)\sigma_w^2}{\sqrt{(t-1)\sigma_w^2(t)\sigma_w^2}} \\ &=\frac{(t-1)\sigma_w^2}{\sigma_w^2\sqrt{(t-1)t}} \\ &=\frac{(t-1)}{\sqrt{(t-1)t}} \\ &= \sqrt{\frac{(t-1)(t-1)}{(t-1)t}} \\ &= \sqrt{\frac{t-1}{t}} \end{align}\]

Now, it’s easy to see that \[\lim_{t\to{\infty}} \sqrt{\frac{t-1}{t}} = 1 \]

This result (obviously) makes sense. Since a random walk is a ‘accumulation of noises’, it’s straightfoward to see that higher \(t\) means \(x_t\) will be a higher accumulation of noises, in such a way that a marginal increment of \(w_t\) will make little difference in the value of \(x_t\), because it will depend mostly on the value of \(x_{t-1}\).

e)

We could take the first differece. Define \(y_t = \Delta x_t = x_t - x_{t-1}\). We have:

\[ \begin{align} y_t &= \delta t+\sum_{k=1}^{t}w_{k} - (\delta (t-1)+\sum_{l=1}^{t-1}w_{l}) \\ &= \delta + w_t \end{align} \]

Now, note that \(E[y_t]\) is constant:

\[ \begin{align} E[y_t] &= E[\delta + w_t]\\ &= \delta + E[w_t]\\ &= \delta \end{align} \]

Also, the \(cov(y_t,y_{t-1})\) is given by:

\[ \begin{align} cov(y_t,y_{t-l}) &= cov(\delta + w_t, \delta + w_{t-l})\\ &= cov(w_t,w_{t-l}) \\ &= 0 \end{align} \]

In the case above, we supposed \(l \neq 0\) .If \(l=0\), we have \(cov(y_t,y_t) = V(y_t) = V(\delta + w_t) = V(w_t) = \sigma_w^2\) Hence, the autocovariance depends only on \(l\). Thus, we have a stationary process.

Exercise 19

\[ x_t = \mu + w_t + \theta w_{t-1} \]

a)

\[ \begin{align} E[x_t] &= E[\mu + w_t + \theta w_{t-1}] \\ &= \mu + E[w_t] + \theta E[w_{t-1}] \\ &= \mu + 0 + \theta0 \\ &= \mu \end{align} \]

b)

First, let’s compute \(\gamma(0)\)

\[ \begin{align} \gamma(0) = cov(x_t,x_t) = V(x_t) &= E[x_t^2] - E[x_t]^2 \\ &= E[(\mu + w_t + \theta w_{t-1})(\mu + w_t + \theta w_{t-1})] - \mu^2 \\ &= E[\mu^2 + 2\mu w_t + 2\mu\theta w_{t-1} + w_t^2 + 2w_t\theta w_{t-1} + \theta^2 w_{t-1}^2] - \mu^2 \\ &= E[w_t^2] + E[\theta^2 w_{t-1}] \\ &= V(w_t) + \theta^2V( w_{t-1}) \\ &= \sigma_w^2 + \theta^2\sigma_w^2 \\ &= \sigma_w^2(1+\theta^2) \end{align}\]

Now, \(\gamma(1)\) is given by:

\[ \begin{align} \gamma(1) = cov(x_t, x_{t-1}) &= cov(\mu + w_t + \theta w_{t-1},\mu + w_{t-1} + \theta w_{t-2}) \\ &= cov(w_t + \theta w_{t-1}, w_{t-1} + \theta w_{t-2}) \\ &= cov(w_t, w_{t-1}) + cov(w_t, \theta w_{t-2}) + cov(\theta w_{t-1}, w_{t-1}) + cov(\theta w_{t-1},w_{t-2}) \\ &= \theta cov(w_{t-1},w_{t-1}) \\ &= \theta\sigma_w^2 \end{align}\]

Finally, \(\gamma(l)\), such that \(|l| >1\) ,is given by: \[ \begin{align} \gamma(l) = cov(x_t, x_{t-l}) &= cov(\mu + w_t + \theta w_{t-1},\mu + w_{t-l} + \theta w_{t-l-1}) \\ &=cov(w_t + \theta w_{t-1}, w_{t-l} + \theta w_{t-l-1}) \\ &= cov(w_t,w_{t-l}) + \theta cov(w_{t},w_{t-l-1}) + \theta cov(w_{t-1},w_{t-l}) + \theta^2cov(w_{t-1},w_{t-l-1}))\\ &= 0 \end{align} \]

For \(\gamma(1)\) and \(\gamma(l)\) above, I used the fact that \(w_t\) is i.i.d., and, thus, any $cov(w_t,w_l)=0 $ \(\forall t\neq l\)

c)

Note first that \(E[x_t]\) is constant for all values of \(\theta\), thus, we need to analyse just the values of the autocovariances. \(\gamma(l)\) for \(|l|> 1\), is equal to 0, hence, it’s also constant and does not depend on \(\theta\) nor on \(t\). For \(\gamma(0)\), note that, for any value of \(\theta\), we have that \(\sigma_w^2(1+\theta^2)\) is also constant and does not depend on \(t\). For \(\gamma(1)\), we have an analogous situation. Thus, since the mean of \(x_t\) is constant, and its covariances depends only on the lag, but not on \(t\), we must have that \(x_t\) is stationary.

d)

First, note that: \[\begin{align} V(\bar{x}) &= \frac{1}{n}\sum_{h=-n}^n(1-\frac{|h|}{n})\gamma_x(h)\\ &=\frac{1}{n}[(1-\frac{|-n|}{n})0+(1-\frac{|-n+1|}{n})0+...+(1-\frac{|-1|}{n})\theta\sigma_w^2+(1-\frac{|0|}{n})\sigma_w^2(1+\theta^2)+(1-\frac{|n|}{n})\theta\sigma_w^2+(1-\frac{|2|}{n})0+(1-\frac{|3|}{n})0+...+(1-\frac{|n|}{n})0]\\ &=\frac{1}{n}[\frac{n-1}{n}\theta\sigma_w^2+\sigma_w^2(1+\theta^2)+\frac{n-1}{n}\theta\sigma_w^2]\\ &=\frac{2\theta\sigma_w^2(n-1)+\sigma_w^2n(1+\theta^2)}{n^2} \end{align}\]

For \(\theta=0\), it’s easy to see that \(V(\bar{x})=\frac{\sigma_w^2}{n}\).

For \(\theta=-1\), we have:

\[ \begin{align} V(\bar{x}) &= \frac{-2\sigma_w^2(n-1)+2\sigma_w^2n}{n^2}\\ &= \frac{2\sigma_w^2(n-n+1)}{n^2} \\ &= \frac{2\sigma_w^2}{n} \end{align} \]

Finally, For \(\theta=1\), we have:

\[ \begin{align} V(\bar{x}) &= \frac{2\sigma_w^2(n-1)+2\sigma_w^2n}{n^2}\\ &= \frac{2\sigma_w^2(2n-1)}{n^2} \end{align} \]

e)

First, for \(\theta=0\), it does not change nothing, since we do not have \(n-1\) in the formula. However, it does change for \(\theta=-1\) and \(\theta=1\). For \(\theta=-1\), we have that the closes \(n-1\) becomes to \(n\), the closer the numerator of \(V\bar{x}\) comes to 0. Hence, a higher sample makes the accuracy almost perfect. Simillarly, for \(\theta=1\) we have that \(V(\bar{x}) \approx \frac{4\sigma_w^2}{n}\) and, thus, a high sample means a lesser variance and a better estimative.

Exercise 20

a)

In this case (figures below), we have that the empirical data is in line with the theoretical result. We have that the ACF for a lag of 0 is the variance of \(\sigma_w^2\), while for other lags is equal to 0.

par(mfrow=c(1,2))
set.seed(123)
w = rnorm(500,0,1)
plot.ts(w, main='Gaussian White Noise')
acf(w,lag.max=20,main=" ACF: Gaussian White Noise")

b)

Here, the results are similar. But, since we do not have a large sample, the confidence interval is wider than in the previous case. IN addiation, it’s harder to identify visually that the graph of the series is actually a white noise.

par(mfrow=c(1,2))
set.seed(123)
w = rnorm(50,0,1)
plot.ts(w, main='Gaussian White Noise')
acf(w,lag.max=20,main=" ACF: Gaussian White Noise")

Exercise 21

a)

In this case, again, we have a result in line with the theoretica model. for a lag of zero, we have a correlation of 1, for a lag of 1, the acf is near 2/3, and for a lag of 2, the acf is approximately 1/3. FOr the other lags, the acf is zero.

par(mfrow=c(1,2))
set.seed(123)
v = rep(NA,1000)
w = rnorm(1001,0,1)
for (i in 2:1000){
  v[i]= (w[i-1]+w[i]+w[i+1])/3
}
v = v[501:1000]

plot.ts(v, main='Moving Average')
acf(v,lag.max=20,main=" ACF: Moving Average")

b)

The result now is just similar to the theoretical case. For a lag of zero, we still have a acf of 1, but this is always the case. For a lag of 2, the acf is not so close anymore of 2/3, and for a lag of 1/3, you cannot reject the hyphothesis of the autocorrelation being 0. IN addition, we have that the confidence interval to not reject this hypothesis is wider now.

par(mfrow=c(1,2))
set.seed(123)
v = rep(NA,100)
w = rnorm(101,0,1)
for (i in 2:1000){
  v[i]= (w[i-1]+w[i]+w[i+1])/3
}
v = v[51:100]

plot.ts(v, main='Moving Average')
acf(v,lag.max=20,main=" ACF: Moving Average")