In: Statistics and Probability
There are two urns that, between them, contain five balls. At each time step, one of the five balls is moved to the other urn. Let the state variable be the number of balls in Urn 1. Find the fixed vector.
a) Draw a state transition diagram and find the transition matrix.
b) Is this a regular chain? Is this an ergodic chain?
c) Find the fixed vector. What is the probability that in the long run Urn 1 has at most one ball?
ANSWER::
a) If Xn denotes the number of balls in urn 1. Then state space is {0,1,2,3,4,5}. If there are "i" balls in urn 1 then there will be "i+1" balls in urn 1 if a ball is moved from 2nd urn with probability 0.5. Similarly the number of balls is reduced to "i-1" with probability 0.5. Hence the transition matrix is given by:
b) Yes, it is a regular chain. Also, it is ergodic chain, because all states can be visited from any state.
c) Let the fixed vector be given by: . This satisfies the following:
, thus we have the following equations:
.
Now we know that:
Thus in long run probability that Urn 1 has at mot one ball is