Comparing estimators

Comparing sample mean, median and trimmed mean

1 Estimation problem

Suppose we are interested in estimating the population mean of each one of the following three distributions, based on a random sample \(x_1,\ldots,x_n\) from

1. \(N(0,1)\): Standard normal distribution, population mean is zero;

2. \(t_4(0,1)\): Standard Student’s \(t\) distribution with 4 degrees of freedom, population mean is zero;

3. \(Beta(3,9)\): Beta distribution, population mean is \(3/(3+9)=0.25\).

x  = seq(-6,6,length=1000)
x1 = seq(0,1,length=1000)
par(mfrow=c(1,2))
plot(x,dnorm(x),xlab="x",ylab="Density",type="l",lwd=2)
lines(x,dt(x,df=4),col=2,lwd=2)
legend("topright",legend=c("Normal","Student's t"),col=1:2,lwd=2,lty=1,bty="n")
plot(x1,dbeta(x1,3,9),xlab="x",ylab="Density",type="l",lwd=2)

2 Comparing 4 estimators

Let \(x_{(1)},\ldots,x_{(n)}\) be the random sample ordered from the smallest to the largest values. We will compare 4 estimators: the sample mean, a weighted sample mean, a trimmed mean and the sample median. More precisely, \[\begin{eqnarray*} {\widehat \theta}_1 &=& \frac{x_1+\cdots+x_n}{n} \\ {\widehat \theta}_2 &=& a_1 x_1 + \cdots + a_n x_n, \qquad \sum_{i=1}^n a_i=1 \ \ \mbox{and} \ \ a_i \geq 0 \ \ \forall i\\ {\widehat \theta}_3 &=& \frac{x_{(k+1)}+\cdots+x_{(n-k)}}{n-2k} \ \ \mbox{for} \ \ k << n \\ {\widehat \theta}_4 &=& x_{((n+1)/2)}, \ \ \mbox{for odd $n$} \end{eqnarray*}\]

For illustration, we will consider \(n=21\), such that the median is \(x_{(11)}\). Also, \(w_i=i\), for \(i=1,\ldots,11\) and \(w_i=22-i\), for \(i=12,\ldots,21\). The weights are then \(a_i=w_i/\sum_{j=1}^{21} w_j\). Finally, \(k=2\).

estimators = function(x){
  n = length(x)
  return(c(mean(x),sum(a*x),median(x),mean(sort(x)[(k+1):(n-k)])))
}
# Monte Carlo exercise
n    = 21
k    = 2
mid  = (n+1)/2
w    = c(1:mid,(mid-1):1)
a    = w/sum(w)
Rep  = 10000
est1 = matrix(0,Rep,4)
est2 = matrix(0,Rep,4)
est3 = matrix(0,Rep,4)
for (r in 1:Rep){
  x1  = rnorm(n,0,1)
  x2  = rt(n,df=4)
  x3  = rbeta(n,3,9)
  est1[r,] = estimators(x1)
  est2[r,] = estimators(x2)
  est3[r,] = estimators(x3)
}

names = c("mean","Weighted mean","median","Trimmed")

par(mfrow=c(1,3))
boxplot(est1,outline=FALSE,names=names,main="Normal data\nSymmetric")
abline(h=0,lwd=2,col=2)
boxplot(est2,outline=FALSE,names=names,main="Student's t data\nSymmetric")
abline(h=0,lwd=2,col=2)
boxplot(est3,outline=FALSE,names=names,main="Beta data\nSkewed")
abline(h=0.25,lwd=2,col=2)