Question

Suppose you choose a coin at random from an urn with 3 coins, where coin i has P(H) = i/4. What is the pmf for your prior distribution of the probability of heads for the chosen coin? What is your posterior given 1 head in 1 flip? 2 heads in 2 flips? 10 heads in 10 flips?Hint: Compute the odds for each coin first.

Answer 1

For each of the three coins, we are given here that:
P(H | coin 1) = 1/4
P(H | coin 2) = 2/4
P(H | coin 3) = 3/4

As each of the coin is to be randomly selected, the probability for the three coins here is given as:
P(coin 1) = P(coin 2) = P(coin 3) = 1/3

Using law of total probability, we get here:
P(H) = P(H | coin 1)P(coin 1) + P(H | coin 2)P(coin 2) + P(H | coin 3)P(coin 3)
P(H) = (1/4)*(1/3) + (2/4)*(1/3) + (3/4)*(1/3) = 2/4 = 0.5

Therefore 0.5 is the required probability here. (This is the prior distribution of getting a head )

Now, we need to get posterior distribution for getting a head in 1 flip. The posterior distribution here is given as:
P( coin 1 | H) = P(H | coin 1)P(coin 1) /P(H) = (1/4)*(1/3) / (0.5) = 1/6
P( coin 2 | H) = P(H | coin 2)P(coin 2) /P(H) = (2/4)*(1/3) / (0.5) = 1/3
P( coin 3 | H) = P(H | coin 3)P(coin 3) /P(H) = (3/4)*(1/3) / (0.5) = 1/2

This is the posterior distribution for the coin selected given that there was 1 head in 1 flip of coin.

Now the prior distribution is computed for 2 heads in 2 flips as:

P(2 heads | coin 1) = (1/4)² = 1/16
P(2 heads | coin 2) = (2/4)² = 1/4
P(2 heads | coin 3) = (3/4)² = 9/16

Using law of total probability, we get here:
P(2 heads) = P(2 heads | coin 1)P(coin 1) + P(2 heads | coin 2)P(coin 2) + P(2 heads | coin 3)P(coin 3)
P(2 heads) = (1/16)*(1/3) + (1/4)*(1/3) + (9/16)*(1/3) = (14/16)*(1/3) = 7/24

Therefore, now the posterior probabilities for 2 heads in 2 tosses are computed as:  (using bayes theorem)
P(coin 1 | 2 heads) = P(2 heads | coin 1)P(coin 1) / P(2 heads)
P(coin 1 | 2 heads) = (1/16)*(1/3) / (7/24) = 1/14
P(coin 2 | 2 heads) = (1/4)*(1/3) / (7/24) = 2 / 7
P(coin 3 | 2 heads) = (9/16)*(1/3)/ (7/24) = 9/14

This is the required posterior distribution here.

For 10 heads in 10 flips, the prior distribution is computed as:
P(10 heads | coin 1) = (1/4)¹⁰ = 1/1048576
P(10 heads | coin 2) = (2/4)¹⁰ = 1/1024
P(10 heads | coin 3) = (3/4)¹⁰ = 59049 / 1048576

The posterior distribution is computed here as:
P(coin 1 | 10 heads ) = P(10 heads | coin 1) P(coin 1) / P(10 heads ) =  (1/3)*(1/4)¹⁰ / (1/3)((1/4)¹⁰ + (2/4)¹⁰ + (3/4)¹⁰) = 1 / (1 + 2^10 + 3^10 ) approx. 0
P(coin 2 | 10 heads ) = (1/3)* (2/4)¹⁰ / (1/3)((1/4)¹⁰ + (2/4)¹⁰ + (3/4)¹⁰) = 2^10 / (1 + 2^10 + 3^10 ) = 0.0170
P(coin 3 | 10 heads ) = (1/3)*(3/4)¹⁰ / (1/3)((1/4)¹⁰ + (2/4)¹⁰ + (3/4)¹⁰) = 3^10 / (1 + 2^10 + 3^10 ) = 0.9829