Question

Once every weekend, I go out for breakfast – usually at Sweet Paris, Mess, or First Watch. However, my decision usually depends on where I ate in the prior week. Basically, my breakfast eating habits are a Markov process and I’ve summarized the transition matrix below.

Sweet Paris Mess First Watch

Sweet Paris 0.3 0.3 0.4

Mess 0.6 0.4   0

First Watch 0.3 0.7 0

Sweet Paris has a policy where you earn a free crepe after 16 visits – thus I might be interested in the 16th passage time – how long it would take me to visit 16 times and earn a free crepe.

As it turns out, I already have 15 visits. I only need one more to earn my free crepe!!! I.e., I’m interested in my first passage to Sweet Paris.

1. (a) If I most recently ate at Sweet Paris, determine the probability I will earn a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks.

2. (b) If I most recently ate at Mess, determine the probability I will earn a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks.

3. (c) If I most recently ate at First Watch, determine the probability I will earn a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks.

Answer 1

Transition matrix is given below

                        Sweet Paris        Mess             First Watch

Sweet Paris            0.3                0.3                     0.4

Mess                      0.6    0.4                      0

First Watch           0.3                0.7                      0

Let our Transition Probability matrix be denoted by M

We will use R software for just calculation process

One step Transition Probability matrix is

M=matrix(c(0.3 ,0.3, 0.4,0.6,0.4,0,0.3 , 0.7, 0),3,3,byrow=T)

> M                                                                    # M
        [,1] [,2] [,3]
[1,] 0.3 0.3 0.4
[2,] 0.6 0.4 0.0
[3,] 0.3 0.7 0.0

We will calculate M², M³ , M⁴ which will represent probability of earning a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks

D2=M%*%M
D2                                                              # M²
     [,1] [,2] [,3]
[1,] 0.39 0.49 0.12
[2,] 0.42 0.34 0.24
[3,] 0.51 0.37 0.12

D3=D2%*%M                                                    # M³
D3                        
      [,1] [,2] [,3]
[1,] 0.447 0.397 0.156
[2,] 0.402 0.430 0.168
[3,] 0.411 0.385 0.204

D4=D3%*%M                     # M⁴                                  
D4
       [,1]   [,2]   [,3]
[1,] 0.4191 0.4021 0.1788
[2,] 0.4290 0.4102 0.1608
[3,] 0.4155 0.4201 0.1644

Thus we have transition probability of 1^st week( M) , 2^nd weeks( M²) , 3^rd weeks( M³) , and 4^th weeks ( M⁴)

(a) If I most recently ate at Sweet Paris ( can be denoted as state 1 ), determine the probability I will earn a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks

Now we need to calculate transition from state 1 to state 1 { P₁₁ } , in step 1,2,3,4 , probability given above as ( M , M² , M³ , M⁴ ) respectively

Probability I will earn a free crepe in 1 week here we need to calculate (i.e probability of returning to state 1 in 1 step) P(X2=1|X1=1) = P₁₁ = 0.3

Probability          =    0.3

M
        [,1] [,2] [,3]
[1,] 0.3 0.3 0.4
[2,] 0.6 0.4 0.0
[3,] 0.3 0.7 0.0

Probability I will earn a free crepe in 2 week (i.e probability of returning to state 1 in 2 step),we calculate it from M²

   Probability          =    0.39

D2                                                              # M²
     [,1] [,2] [,3]
[1,] 0.39 0.49 0.12
[2,] 0.42 0.34 0.24
[3,] 0.51 0.37 0.12

Probability I will earn a free crepe in 3 week ,we calculate it from M³ = 0.447

Probability          =    0.447

D3                                                                       # M³
      [,1] [,2] [,3]
[1,] 0.447 0.397 0.156
[2,] 0.402 0.430 0.168
[3,] 0.411 0.385 0.204

Probability I will earn a free crepe in 4 week ,we calculate it from M⁴ = 0.4191

Probability          =    0.4191

D4                                  #   M⁴
       [,1]   [,2]   [,3]
[1,] 0.4191 0.4021 0.1788
[2,] 0.4290 0.4102 0.1608
[3,] 0.4155 0.4201 0.1644

b) If I most recently ate at Mess, determine the probability I will earn a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks.

Now we need to calculate transition from state 2 to state 1 { P₂₁ } , in step 1,2,3,4 , probability given above as ( M , M² , M³ , M⁴ ) respectively

which represents week 1 , week,2 ,weeks 3 , weeks 4

Thus , I most recently ate at Mess

Probability I will earn a free crepe in 1 week ( from M ) = P⁽¹⁾₃₁ = 0.6

Probability          =    0.6

Probability I will earn a free crepe in 2 week ( from M² ) = P⁽²⁾₃₁ = 0.51

Probability          =    0.51

Probability I will earn a free crepe in 3 week ( from M³ ) = P⁽³⁾₃₁ = 0.411

Probability          =   0.411

Probability I will earn a free crepe in 4 week ( from M⁴ ) = P⁽⁴⁾₃₁ = 0.4290

Probability          =    0.4290

(c) If I most recently ate at First Watch, determine the probability I will earn a free crepe in 1 week, 2 weeks, 3 weeks, and 4 weeks

Now we need to calculate transition from state 3 to state 1 { P⁽ⁿ⁾₃₁ } , in step n=1,2,3,4 , probability given above in ( M , M² , M³ , M⁴ ) respectively

Thus , I most recently ate at First Watch

Probability I will earn a free crepe in 1 week ( from M ) = P⁽¹⁾₂₁ = 0.3

Probability          =    0.3

Probability I will earn a free crepe in 2 week ( from M² ) = P⁽²⁾₂₁ = 0.42

Probability          =    0.42

Probability I will earn a free crepe in 3 week ( from M³ ) = P⁽³⁾₂₁ = 0.402

Probability          =   0.402

Probability I will earn a free crepe in 4 week ( from M⁴ ) = P⁽⁴⁾₂₁ = 0.4155

Probability          =    0.4155