Question

A company has three machines. On any day, each working machine breaks down with probability 0.4, independent of other machines. At the end of each day, the machines that have broken down are sent to a repairman who can work on only one machine at a time. When the repairman has one or more machines to repair at the beginning of a day, he repairs and returns exactly one at the end of that day. Let Xn be the number of working machines at the beginning of day n

1. Is {Xn : n ≥ 0} a Markov chain? Why?
2. Given that one machine is working today what is the probability that two machines will be working tomorrow?
3. Given that two machines are working today what is the probability that all machines will be working two days later?

Answer 1

Xn be the number of working machines at the beginning of day n. The possible values of Xn are 0, 1, 2, 3.

The transition probability from state 0 to state 1 is 1 (Since there are no machines to break down on that day and repairman will repair one machine on that day)

The transition probability from state 1 to state 1 is 0.4 (Since a machine can fail with probability 0.4 on that day and repairman will repair another machine on that day)

The transition probability from state 1 to state 2 is 0.6 (Since no machine can fail with probability 0.6 on that day and repairman will repair another machine on that day)

The transition probability from state 2 to state 1 is 0.16 (Since two machines can fail with probability 0.4² = 0.16 on that day and repairman will repair another machine on that day)

The transition probability from state 2 to state 2 is 0.48 (Since one machines can fail with probability 2 * 0.4 * (1 - 0.4)= 0.48 on that day and repairman will repair another machine on that day)

The transition probability from state 2 to state 3 is 0.36 (Since no machine can fail with probability 0.6² = 0.36 on that day and repairman will repair another machine on that day)

The transition probability from state 3 to state 0 is 0.064 (Since three machines can fail with probability 0.4³ = 0.064 on that day and there are no machines on that day to repair)

The transition probability from state 3 to state 1 is 0.48 (Since two machines can fail with probability 3 * 0.4² * (1 - 0.4)= 0.288 on that day and there are no machines on that day to repair)

The transition probability from state 3 to state 2 is 0.36 (Since one machines can fail with probability 3 * 0.4 * (1 - 0.4)²= 0.432 on that day and there are no machines on that day to repair)

The transition probability from state 3 to state 3 is 0.216 (Since no machines can fail with probability 0.6³ = 0.216 on that day and there are no machines on that day to repair)

Since the transition to any states depends only on the current state, {Xn : n ≥ 0} a Markov chain.

The transition probability matrix is,

Given that one machine is working today what is the probability that two machines will be working tomorrow

= P(Xn+1 = 2 | Xn = 1) = 0.6

Given that two machines are working today what is the probability that all machines will be working two days later

= P(Xn+2 = 3 | Xn = 2) =  P(Xn+2 = 3, Xn+1 = 2, Xn = 2) + P(Xn+2 = 3, Xn+1 = 3, Xn = 2)

= P(Xn+1 = 2, Xn = 2) P(Xn+2 = 3, Xn+1 = 2) + P(Xn+1 = 2, Xn = 3) P(Xn+2 = 3, Xn+1 = 3)

= 0.48 * 0.36 + 0.36 * 0.216

= 0.25056