In: Statistics and Probability
Let Xn be a Markov chain with states 0,1,...,9 and transition probabilities P0,0 = P0,1 = P9,8 = P9,9 = 1/2 an Pi,i = Pi,i+1 = Pi,i−1 = 1/3 for all 1 ≤ i ≤ 8.
(a) Draw the transition diagram.
(b) What is the probability that X1,X2,X3,X4 are all smaller than 3 given that X0 = 1?
Hint: Create a simpler Markov chain with 4 states.