An individual has three umbrellas, some at his office, and some
at home. If he is leaving home in the morning (or leaving work at
night) and it is raining, he will take an umbrella, if there is one
there. Otherwise, he gets wet. Assume that, independent of the
past, it rains on each trip with probability 0.2. To formulate a
Markov chain, let Xn be the number of umbrellas at his current
location “before” he starts his n-th trip....