Question

A gambler tosses a coin and a tetrahedron at each stage. If "H", he receives the amount appearing at the face of the tetrahedron. If "T", he pays the amount. The tetrahedron is fair, but the probability of H is ⅔. The game ends if the gambler reaches 0 or less wealth, or accumulates 6 or more wealth.

Define X_n = 0 if wealth is less than or equal to zero.
Define X_n = i if wealth is i = 1,2,3,4,5.
Define X_n = 6 if wealth is greater than or equal to 6.

Determine the transition probability matrix

Answer 1

Tetrahedron has four faces, so the probability of getting 1, 2, 3, or 4 is 1/4

Let X be the amount of gain/loss at each stage.

P(X = 1) = P(X = 2) = P(X = 3) = P(X = 4) = (1/4) * (2/3) = 1/6

P(X = -1) = P(X = -2) = P(X = -3) = P(X = -4) = (1/4) * (1/3) = 1/12

Thus, the transition probabilities from state 0 are,

P(X_n+1 = 0 | X_n = 0) = 1 (State 0 is an absorbing state as the game ends here)

The transition probabilities from state 1 are,

P(X_n+1 = 2 | X_n = 1) = P(X_n+1 = 3 | X_n = 1) = P(X_n+1 = 4 | X_n = 1) = P(X_n+1 = 5 | X_n = 1) = 1/6

P(X_n+1 = 0 | X_n = 1) = P(X = -1) + P(X = -2) + P(X = -3) + P(X = -4) = 4 * 1/12 = 1/3

The transition probabilities from state 2 are,

P(X_n+1 = 3 | X_n = 2) = P(X_n+1 = 4 | X_n = 2) = P(X_n+1 = 5 | X_n = 0) = P(X_n+1 = 6 | X_n = 2) = 1/6

P(X_n+1 = 1 | X_n = 2) = P(X = -1) = 1/12

P(X_n+1 = 0 | X_n = 2) = P(X = -2) + P(X = -3) + P(X = -4) = 3 * 1/12 = 1/4

The transition probabilities from state 3 are,

P(X_n+1 = 4 | X_n = 3) = P(X_n+1 = 5 | X_n = 3) = 1/6

P(X_n+1 = 6 | X_n = 3) = P(X = 3) + P(X = 4) = 2 * 1/6 = 1/3

P(X_n+1 = 2 | X_n = 3) = P(X_n+1 = 1 | X_n = 3) = P(X = -1) = P(X = -2) = 1/12

P(X_n+1 = 0 | X_n = 2) = P(X = -3) + P(X = -4) = 2 * 1/12 = 1/6

The transition probabilities from state 4 are,

P(X_n+1 = 5 | X_n = 4) = 1/6

P(X_n+1 = 6 | X_n = 4) = P(X = 2) + P(X = 3) + P(X = 4) = 3 * 1/6 = 1/2

P(X_n+1 = 3 | X_n = 4) = P(X_n+1 = 2 | X_n = 4) = P(X_n+1 = 1 | X_n = 4) = P(X = -1) = P(X = -2) = P(X = -3) = 1/12

P(X_n+1 = 0 | X_n = 4) = P(X = -4) = 1/12

The transition probabilities from state 5 are,

P(X_n+1 = 6 | X_n = 5) = P(X = 1) + P(X = 2) + P(X = 3) + P(X = 4) = 4 * 1/6 = 2/3

P(X_n+1 = 4 | X_n = 5) = P(X_n+1 = 3 | X_n = 5) = P(X_n+1 = 2 | X_n = 5) = P(X_n+1 = 1 | X_n = 5) = P(X = -1) = P(X = -2) = P(X = -3) = P(X = -4) = 1/12

The transition probabilities from state 6 are,

P(X_n+1 = 6 | X_n = 6) = 1 (State 6 is an absorbing state as the game ends here)

The transition probability matrix with states 0, 1, 2, 3, 4, 5, 6 are