Sequential Bayesian Learning

Bernoulli trials with discrete $ heta$

1 Binomial model and discrete prior

The data is a sequence of Bernoulli trials, i.e. \(x_1,\ldots,x_n\) are iid \(Bernoulli(\theta)\) for \(\theta \in \{0.3,0.5,0.8\}\). The prior for \(\theta\) is given by \[\begin{eqnarray*} Pr(\theta=0.3) &=& 0.4\\ Pr(\theta=0.5) &=& 0.5\\ Pr(\theta=0.8) &=& 0.1 \end{eqnarray*}\]

1.1 Simulaing some data

set.seed(12345)
n      = 50
theta0 = 0.8
x      = rbinom(n,1,theta0)
s      = cumsum(x)

1.2 Bayesian inference

theta     = c(0.3,0.5,0.8)
ptheta    = c(0.4,0.5,0.1)
prob = matrix(0,n,3)
for (i in 1:n){
  prob[i,] = ptheta*theta^s[i]*(1-theta)^(i-s[i])
  prob[i,] = prob[i,]/sum(prob[i,]) 
}
par(mfrow=c(1,1))
plot(prob[,1],ylim=c(0,1),xlim=c(0,n),type="b",xlab="Sample size (n)",
     ylab="Posterior probabilities",lwd=2,pch=16)
lines(prob[,2],type="b",col=2,lwd=2,pch=16)
lines(prob[,3],type="b",col=3,lwd=2,pch=16)
points(x,col=6,pch=16)
title(paste("True theta=",theta0,sep=""))
for (i in 1:3){
  points(0,ptheta[i],pch=16,col=i)
  segments(0,ptheta[i],1,prob[1,i],col=i,lwd=2)
}
legend("right",legend=c(
paste("Pr(theta1=",theta[1],"|x1,...xn)",sep=""),
paste("Pr(theta2=",theta[2],"|x1,...xn)",sep=""),
paste("Pr(theta3=",theta[3],"|x1,...xn)",sep="")),
col=1:3,bty="n",lty=1,lwd=2)