In: Statistics and Probability
A die is thrown repeatedly until it lands ‘6’ three times in a row. Model this by a Markov chain
The problem can be modeled as Markov chain with 4 states - '0' , '6', '66' and '666' where
State '0' - State with no '6' on the dice
State '6' - State with a '6' on the dice
State '66' - State with a two '6' on the dice
State '666' - State with a three '6' on the dice. This is a an Absorbing state, which denotes the state when the dice 'lands '6' three times in a row.
The transition probability from state '0' to state '6'; state '6' to state '66'; and state '66' to state '666' is 1/6 (Rolling a 6 on the dice)
The transition probability from state '0' to state '0'; state '6' to state '0'; and state '66' to state '0' is 5/6 (Rolling a number other than 6 on the dice)
The transition probability from state '666' to state '666' is 1. The transition probability from state '666' to any other state is 0.
The transition probability matrix and diagram is,