Question

Consider a student who is rather irregular about class attendance. If she attends class one day, the probability is .8 that she will attend class the next day. And if she misses class, the probability is .4 that she will miss again the following day.

a. Set up the transition matrix for this stochastic process.

b. If the student attends the first day of class, what is the probability she will miss the third day of class?

c. In the long run what proportion of the time will the student attend class.

d, If student misses class one day, what is the average number of classes going by before she misses

      class again.

Answer 1

a.

Let the states of the Markov chain be A and M denoting the the student attend or misses the class on a day.

The transition probability from state A to state A is 0.8

The transition probability from state A to state M is 1 - 0.8 = 0.2

The transition probability from state M to state M is 0.4

The transition probability from state M to state A is 1 - 0.4 = 0.6

The transition matrix is,

b.

If the student attends the first day of class, the probability she will miss the third day of class

= P(X3 = M | X1 = A)

= P(X3 = M, X2 = A, X1 = A) + P(X3 = M, X2 = M, X1 = A)  

= 0.8 * 0.2 + 0.2 * 0.4

= 0.24

c.

Let = [a, b] be the long run proportion. Then P =

which gives,

0.8a + 0.6b = a => 0.2a = 0.6b => a = 3b

0.2a + 0.4b = b

and a + b = 1

=> 3b + b = 1

=> b = 1/4

a = 3b = 3/4

The long run proportion in state A and M is [3/4 , 1/4]

d.

If student misses class one day, let X be the number of classes going by before she misses class again.

Then X follows Geometric distribution with the probability of success (misses the class) is p = 0.4

Average number of classes = (1 - p)/p = (1 - 0.4) / 0.4 = 1.5