Let X = {Xn:n= 0,1,...,} be a DTMC on state space S. Define Yn =
(Xn,...
Let X = {Xn:n= 0,1,...,} be a DTMC on state space S. Define Yn =
(Xn, Xn+1). Prove that Y = {Yn: n = 0,1,2,...} is a DTMC. Specify
its state space and the transition matrix.
Q1. Let {Xn|n ≥ 0} is a Markov chain with state space S. For i ∈
S, define τi = min{n ≥ 0|Xn = i}. Show that τi is a stopping time
for each i. Q2. Let τi as in Q1. but for any discrete time
stochastic process. Is τi a stopping time? Q3. Let {Xn|n ≥ 0} be a
Markov chain and i is a state. Define the random time τ = min{n ≥
0|Xn+1 = i}. If τ...
Let (xn), (yn) be bounded sequences.
a) Prove that lim inf xn + lim inf yn ≤
lim inf(xn + yn) ≤ lim sup(xn +
yn) ≤ lim sup xn + lim sup yn.
Give example where all inequalities are strict.
b)Let (zn) be the sequence defined recursively by
z1 = z2 = 1, zn+2 = √
zn+1 + √ zn, n = 1, 2, . . . . Prove that
(zn) is convergent and find its limit. Hint; argue...
1. . Let X1, . . . , Xn, Y1, . . . , Yn be mutually independent
random variables, and Z = 1 n Pn i=1 XiYi . Suppose for each i ∈
{1, . . . , n}, Xi ∼ Bernoulli(p), Yi ∼ Binomial(n, p). What is
Var[Z]?
2. There is a fair coin and a biased coin that flips heads with
probability 1/4. You randomly pick one of the coins and flip it
until you get a...
Q5. Let {Xn|n ≥ 0} is a Markov chain with state space S = {0, 1,
2, 3}, and transition probability matrix (pij ) given by 2 3
1 3 0 0 1 3 2 3 0 0 0 1 4 1 4 1 2 0 0 1 2 1 2 Determine all
recurrent states. 1 2 Q6. Let {Xn|n ≥ 0} is a Markov chain with
state space S = {0, 1, 2} and transition...
Let {Xn|n ≥ 0} is a Markov chain with state space S = {0, 1, 2,
3}, and transition probability matrix (pij ) given by 2 3 1 3
0 0 1 3 2 3 0 0 0 1 4 1 4 1 2 0 0 1 2 1 2 Determine all
recurrent states. Q3. Let {Xn|n ≥ 0} is a Markov chain with state
space S = {0, 1, 2} and transition probability matrix (pij...
Q1. Let {Xn|n ≥ 0} is a Markov chain with state space S = {0, 1,
2, 3} and transition probability matrix (pij ). Let τi = min{n ≥ 1
: Xn = i}, i = 0, 1, 2, 3. Define Bij = {Xτj = i}. Is Bij ∈ σ(X0, ·
· · , Xτj ) ? Q2. Let {Xn|n ≥ 0} is a Markov chain with state space
S = {0, 1, 2, 3}, X0 = 0, and transition...
Consider a Markov chain {Xn|n ≥ 0} with state space S = {0, 1, ·
· · } and transition matrix (pij ) given by pij = 1 2 if j = i − 1
1 2 if j = i + 1, i ≥ 1, and p00 = p01 = 1 2 . Find P{X0 ≤ X1 ≤ · ·
· ≤ Xn|X0 = i}, i ≥ 0
. Q2. Consider the Markov chain given in Q1. Find P{X1,...
1.14 Let Xn be a Markov chain on state space {1,2,3,4,5} with
transition matrix
P=
0
1/2
1/2
0
0
0
0
0
1/5
4/5
0
0
0
2/5
3/5
1
0
0
0
0
1/2
0
0
0
1/2
(a) Is this chain irreducible? Is it
aperiodic?
(b) Find the stationary probability vector.
6.42 Let X1,..., Xn be an i.i.d. sequence of Uniform (0,1)
random variables. Let M = max(X1,...,Xn).
(a) Find the density function of M. (b) Find E[M] and V[M].