Question

to meet the demand for water in a region it is decicded that a dam should bu built in a river.the capacity of the dam is 3 units.the probability distribution of the number of units of water that flow into the dam during each week is given by p0=1/8, p1=1/4, p2=1/2 and p3=1/8(i.e. pi=probability that i units will flow into the dam during a week).If the supply of water by the river exceeds the remaining capacity of the dam,the excess that water is lost.the demand for water is 2 units in each week where it is assumed that the water has to be provided at the beginning of each week.ıf the dam does not contain enough water to satisfy the demand,the shortage will be fulfilled at a cost of 10 per unit shortage.the weekly depreciation cost of the dam is 2.5

a)find the transition matrix of the markov chain.

b)determine the long runaverage cost per week.

hint:let the state be the number of units of water in the dam at the end of each week.

Answer 1

a)

Let the state be the number of units of water in the dam at the end of each week. So, there would be 4 states - 0, 1, 2, 3 representing the number of units of water in the dam at the end of each week.

The probability distribution of the number of units of water that flow into the dam during each week is given by p0=1/8, p1=1/4, p2=1/2 and p3=1/8. The average demand for water is 2 units.

So, the number of units of water in the dam at the end of each week will changed by -2, -1, 0, 1 units with probabilities p0=1/8, p1=1/4, p2=1/2 and p3=1/8

For state 0, the number of units of water in the dam at the end of week is 0. So, the number of units of water in the dam at the end of next week is 0, 0, 0, 1 with probabilities p0=1/8, p1=1/4, p2=1/2 and p3=1/8. That is, the transition probabilities from state 0 to state 0 is (1/8 + 1/4 + 1/2) = 7/8 and from state 0 to state 1 = 1/8

For state 1, the number of units of water in the dam at the end of week is 1. So, the number of units of water in the dam at the end of next week is 0, 0, 1, 2 with probabilities p0=1/8, p1=1/4, p2=1/2 and p3=1/8. That is, the transition probabilities from state 1 to state 0 is (1/8 + 1/4 ) = 3/8 and from state 1 to state 1 = 1/2 and from state 1 to state 2 is 1/8.

For state 2, the number of units of water in the dam at the end of week is 2. So, the number of units of water in the dam at the end of next week is 0, 1, 2, 3 with probabilities p0=1/8, p1=1/4, p2=1/2 and p3=1/8. That is, the transition probabilities from state 2 to state 0 is 1/8 and from state 2 to state 1 = 1/4 and from state 2 to state 2 is 1/2 and from state 2 to state 3 is 1/8.

For state 3, the number of units of water in the dam at the end of week is 3. So, the number of units of water in the dam at the end of next week is 1, 2, 3, 3 with probabilities p0=1/8, p1=1/4, p2=1/2 and p3=1/8. That is, the transition probabilities from state 3 to state 1 is 1/8 and from state 3 to state 2 = 1/4 and from state 3 to state 3 is (1/2 + 1/8) = 5/8

So, the transition matrix of the markov chain is given as,

b)

There are 2 units short for state 0. So, average cost per week for state 0 = 10 * 2 + 2.5 = 22.5

There are 1 units short for state 1. So, average cost per week for state 1 = 10 * 1 + 2.5 = 12.5

There are units short for state 2 and 3.

So, average cost per week for state 2 or 3 = 10 * 0 + 2.5 = 2.5

Long run probabilities for each state is calculated as below.

Let = [a, b , c, d] be the long run probabilities for states 0, 1, 2, 3

Then P = P which gives the below equations,

7/8 a + 3/8 b + 1/8 c = a

1/8 a + 1/2 b + 1/4 c + 1/8 d = b

1/8 b + 1/2 c + 1/4 d = c

1/8 c + 5/8 d = d

a + b + c + d = 1

Solving, we get

a = 0.7021, b = 0.2128, c = 0.0638, d = 0.0213

The long runaverage cost per week = 0.7021 * 22.5 + 0.2128 * 12.5 + 0.0638 * 2.5 + 0.0213 * 2.5

= 18.67