Question

Q1. Let {Xn : n ≥ 0} denote the random walk on 9-cycle. Express it as a random walk on a group (G, ·) with transition probabilities given by pxy = µ(y · x −1 ) for an appropriate distribution µ on G.

Q2. Consider the stochastic process {Xn|n ≥ 0}given by X0 = 1, Xn+1 = I{Xn = 1}Un+1 + I{Xn 6= 1}Vn+1, n ≥ 0, where {(Un, Vn)|n ≥ 1} is an i.i.d. sequence of random variables such that Un is independent of Vn for each n ≥ 1 and U1−1 is Bernoulli(p) and V1−1 is Bernoulli(q) random variables. Show that {Xn|n ≥ 1} is a Markov chain and find its transition matrix. Also find P{Xn = 2}

. Q3. Let {Xn|n ≥ 0} be the Markov chain given Q.2. Find a Markov chain {Yn|n ≥ 0} of the form Yn+1 = f(Yn, Zn), n ≥ 1, Y0 = 1 where {Zn|n ≥ 1} is an i.i.d. sequence of R-valued random variables such that finite dimensional distributions of {Xn|n ≥ 0} and {Yn|n ≥ 0} coincide.

Q4. (Unit demand inventory system) Consider an inventory system in discrete time with the following description. At the beginning of the period the inventory decreases by one unit if the inventory level at the beginning is positive other the level remains zero till the end of the period. At the end of the period nth period, the inventory is increased by an amount Vn, where {Vn|n ≥ 1} is i.i.d. with P{V1 = i} = pi , i ≥ 0. Let Xn denote the level of the inventory at the beginning(just before the probable inventory decrease) of the period [n, n + 1). Show that {Xn|n ≥ 0} is a Markov chain under the assumption that X0 is Z + valued random variable which is independent of {Vn|n ≥ 1}. Also find its transition matrix.

Answer 1