Question

A local car dealer closes on Sundays at 6:00 pm. He counts the number

of cars available at that time. If there are two or less, order enough to

raise the level to six. Cars are delivered at night and are available when

the exhibition hall opens at 9:00 a.m. on Monday morning. Let Pd (x) be

the probability that the demand during the week is equal to x: Suppose that:

Pd (0) = 0.2 ,Pd (1) = 0.5 ,Pd (2) = 0.2 ,Pd (3) = 0.1

If there is more demand than cars during the week than those available at the

start of it, the excess demand is wasted and the distributor ends the week with

zero cars available. Define a markov chain where states are numbers of cars

available on Monday morning at 9:00 am. Find the transition matrix.

Answer 1

We start with making a empty matrix, where the rows are the states (or number of cars) on a Monday and the columns are the states on the following Monday.

Every cell contains the probability of transitioning from having cars on a Monday to having cars on the next Monday, henceforth noted as .

The only way to have the same number on next Monday is if there is no demand in that week. The probability of that happening is . Thus,

If the number of cars goes to , the number is reset to .

When only one car is sold and the number stays ,

Since there is no way the number can be higher than last week without resetting to ,

The remaining values can similarly be calculated as

The transition matrix is now complete with rows denoting the state on a Monday and the columns denoting the state on the next Monday. The final matrix is:

Notice that the sum of each rows is . It is so because starting from any state, the probabilities of transitioning to all states must sum to 1.