Question

Everyday before work Sally feeds her cat either tuna or chicken cat food. If she feeds her cat tuna today, then tomorrow she will roll a 6-sided die. If the roll is a 5 or a 6 then she will feet her cat tuna again tomorrow otherwise she will feed her cat chicken. If she feed her cat chicken cat food today, she will feed her cat tuna tomorrow and not chicken. On the first day, Sally flips a coin to decide what to feed her cat. If its heads - its tuna and if she flips tails its chicken.

Name the states for this Markov chain and then draw a transition diagram.

Provide a transition matrix P, and the initial state distribution So for this Markov chain.

Answer 1

States of Markov chain-

Suppose, 0 represents the state that tuna cat food is fed. Also, suppose, 1 represents the state that chicken cat food is fed.

Transition diagram-

If tuna cat food is fed in a certain day, it is again fed in the next day with probability 1/3 and chicken cat food is fed in the next day with probability 2/3.

If chicken cat food is fed in a certain day, tuna cat food is fed in the next day.

Corresponding transition diagram is as follows.

Transition matrix-

Transition matrix corresponding to the given Markov chain is as follows.

Initial state distribution-

On the first day, Sally flips a coin to decide whether to feed tuna or chicken cat food. She selected tuna cat food for heads and chicken cat food otherwise. Since (in an unbiased coin) occurrence of head and tail are equally likely with probability 0.5 for each we have initial state distribution as follows.