Question

1. Every day, Eric takes the same street from his home to the university. There are 4 street lights along his way, and Eric has noticed the following Markov dependence. If he sees a green light at an intersection, then 60% of time the next light is also green, and 40% of time the next light is red. However, if he sees a red light, then 75% of time the next light is also red, and 25% of time the next light is green. Let 1 = “green light” and 2 = “red light” with the state space {1, 2}.

(a) Construct the 1-step transition probability matrix for the street lights.

(b) If the first light is red, what is the probability that the third light is red?

(c) Eric’s classmate Jacob has many street lights between his home and the university. If the first street light is red, what is the probability that the last street light is red? (Use the steady-state distribution.)

Answer 1

Let state 1 = "green light" and state 2 = "red light"

If he sees a green light at an intersection, then 60% of time the next light is also green. p11 = 0.6

If he sees a green light at an intersection, then 40% of time the next light is red. p12 = 0.4

If he sees a red light at an intersection, then 75% of time the next light is also red. p22 = 0.75

If he sees a red light at an intersection, then 25% of time the next light is green. p21 = 0.25

If the first street light is red, then the probability that the last street light is red is 8/13 i.e 0.6154.