# Wooldridge Source: F. Vella and M. Verbeek (1998), “Whose Wages Do Unions Raise? 
# A Dynamic Model of Unionism and Wage Rate Determination for Young Men,” 
# Journal of Applied Econometrics 13, 163-183. I obtained the data from the Journal 
# of Applied Econometrics data archive at http://qed.econ.queensu.ca/jae/. 
# This is generally a nice resource for undergraduates looking to replicate or extend a 
# published study. Data loads lazily.
# Reference: http://www.jstor.com/stable/223257

install.packages("wooldridge")
install.packages("bayesreg")
library("wooldridge")
library("bayesreg")

data('wagepan')

names(wagepan)

dim(wagepan)

?wagepan
attach(wagepan)

n = nrow(wagepan)

y  = scale(lwage)

X1 = cbind(educ,exper,hours,year-1979)
X2 = cbind(black,hisp,married,union,nrthcen,nrtheast,south)
X3 = cbind(occ1,occ2,occ3,occ4,occ5,occ6,occ7,occ8)
X4 = cbind(agric,min,construc,trad,tra,fin,per,ent,manuf,pro,pub)
X5 = matrix(rnorm(n*100),n,100)
X   = cbind(scale(X1),X2,X3,X4,X5)
X   = scale(X)

# Obs:  Notice that, I have added 100 irrelevant covariates.  There are enough observations for 
# the analysis, i.e. n=4360
# The idea is to see what happens with those extra 100 useless covariates when you run
# i) ordinary least squares or sparse regressions with ii) ridge penalty, iii) lasso penalty, 
# or iv) horseshoe penalty. You can look at my R script for inspiration (don't use without understanding!)
# https://hedibert.org/wp-content/uploads/2022/03/more-on-sparse-regressions-R.txt

#Your task:

# a) Fit ols regression 
# b) Fit ridge, lasso and horseshoe regressions
# c) Which fit has the within-sample smallest root mean square error (rmse)?
# d) Randomly split the data in two equal parts (2180 observations each), use the first half for 
# "training" (fit) and the second half for computing room mean square errors.  Which out-of-sample 
# rmse is the smallest?
# e) Repeat d) 100 times (with the halves randomly picked each time) and compare the 100 out-of-sample 
# rmse for the four fits: ols, ridge, lasso and hourseshoe.   Comment your findings.