Question

John finds a bill on his desk. He has three options: ignore it and leave it on his own desk, move the bill over to his wife Mary's desk, or pay the bill immediately. The probability that he leaves it on his own desk is 0.6. The probability that he moves it to Mary's desk is 0.3. The probability that he pays the bill immediately is 0.1.

Similarly, if Mary finds a bill on her desk she can choose to leave it on her own desk, put it on John's desk, or pay it immediately. The probability that it remains on her desk is 0.6. The probability she moves it to John's desk is 0.1. The probability she pays it immediately is 0.3.

Once a bill is paid it will not return to either of the desks. In other words, there is a 0% chance that a bill will return to John's desk or Mary's desk once it goes to the mailbox.

Assume this is a Markov Chain process. Set up the transition matrix and use it to answer the following questions. (Hint: When determining what your matrix labels should be, think of the location of the bill, not the action done to it. For example, the label "moves to the other desk" would not be a valid label.)

(a) What is the probability that a bill currently on John's desk will be paid within two days?


(b) What is the probability that a bill currently on John's desk will be on Mary's desk 3 days later?

Answer 1

Answer: The transition Matrix will be

J represents John desk in matrix.

M represent Mary desk in matrix.

I represent immediate in matrix.