Question

In: Statistics and Probability

Model the following systems as discrete time Markov chains. Define the Markov chains by precisely stating...

Model the following systems as discrete time Markov chains. Define the Markov chains by precisely stating the quantity of interest, give the state space, draw a state diagram, and write down the one step transition probability matrix.

(a) Every evening I eat at one of my three favorite restaurants in Bruntsfield. Having eaten at a restaurant one night, the next evening I’ll go to one of the other two restaurants equally likely.

(b) Consider a special gambler’s ruin problem, where the total wealth is £5, and you play many rounds against an opponent. You win each round with probability 2/3 and then you get £1 from your opponent, otherwise you lose and you give £2 to your opponent (or £1 if you only have £1). If someone gets to own all wealth they keep that in all further rounds.

Solutions

Expert Solution

orchestra answered 2 years ago
Next > < Previous

Related Solutions

Define and discuss the following concepts in the theory of Markov chains: (A) Irreducibility and related...
Define and discuss the following concepts in the theory of Markov chains: (A) Irreducibility and related properties. (B) Periodicity and related properties. (C) Recurrent/transient states and related properties.
Xn is a discrete-time Markov chain with state-space {1,2,3}, transition matrix, P = .2 .1 .7...
Xn is a discrete-time Markov chain with state-space {1,2,3}, transition matrix, P = .2 .1 .7 .3 .3 .4 .6 .3 .1 a) find E[X1|X0=2] = b)  The P(X9=1|X7=3) = C) The P(X2=2) =
We can approximate the continuous-time tank model of the previous problem by a discrete model as...
We can approximate the continuous-time tank model of the previous problem by a discrete model as follows. Assume that we only observe the tank contents each minute (time is now discrete). During each minute, 20 liters (or 10% of each tank’s contents) are transferred to the other tank. Let x1(t) and x2(t) be the amounts of salt in each tank at time t. We then have: x1(t + 1) = 9 /10 x1(t) + 1 /10 x2(t) x2(t + 1)...
the following are impulse responses/outputs of discrete -time LTI systems. Determine whether each system is causal...
the following are impulse responses/outputs of discrete -time LTI systems. Determine whether each system is causal and/or stable. justify your answers 1. h [n] = 1/5^n u [n] 2. h [n] = 5^n u [3-n] 3. y [n] = 3x [n] - 0.15y [n-1] 4. y [n] = 2e^-x [n] 5. y [n] = n^2 4x [n] B. Show if the systems defined in 1 to 5 above have bounded input and output (BIBO) from the summation of their impulse...
Classify each of the following processes as discrete-time or continuous-time, and discrete-space or continuous-space. (a) The...
Classify each of the following processes as discrete-time or continuous-time, and discrete-space or continuous-space. (a) The baud rate of a modem, recorded every 60 s (b) The number of people logged into Facebook throughout the day (c) The operational state, denoted 1 or 0, of a certain machine recorded at the end of each hour (d) The noise (in dB) in an audio signal measured throughout transmission
2. Classify each of the following processes as discrete-time or continuous-time, and discrete-space or continuous-space. (a)...
2. Classify each of the following processes as discrete-time or continuous-time, and discrete-space or continuous-space. (a) The baud rate of a modem, recorded every 60 s (b) The number of people logged into Facebook throughout the day (c) The operational state, denoted 1 or 0, of a certain machine recorded at the end of each hour (d) The noise (in dB) in an audio signal measured throughout transmission
Derive the equivalent discrete-time model of a wireless communication system with quasi-static frequency-selective fading.
Derive the equivalent discrete-time model of a wireless communication system with quasi-static frequency-selective fading.
Classify each of the following random variables as discrete or continuous. a) The time left on...
Classify each of the following random variables as discrete or continuous. a) The time left on a parking meter So, we want to ask do we COUNT the time left on the parking meter or do we MEASURE it. We measure the time, so measuring is continuous. b) The number of bats broken by a major league baseball team in a season We can count the number of physical bats there broken during a season. Since we can COUNT it,...
Q1.Consider an inventory system in discrete time with the following description. At the beginning of the...
Q1.Consider an inventory system in discrete time with the following description. At the beginning of the period the inventory decreases by one unit if the inventory level at the beginning is positive other the level remains zero till the end of the period. At the end of the period nth period, the inventory is increased by an amount Vn, where {Vn|n ≥ 1} is i.i.d. with P{V1 = i} = pi , i ≥ 0. Let Xn denote the level...
Q1.Consider an inventory system in discrete time with the following description. At the beginning of the...
Q1.Consider an inventory system in discrete time with the following description. At the beginning of the period the inventory decreases by one unit if the inventory level at the beginning is positive other the level remains zero till the end of the period. At the end of the period nth period, the inventory is increased by an amount Vn, where {Vn|n ≥ 1} is i.i.d. with P{V1 = i} = pi , i ≥ 0. Let Xn denote the level...
ADVERTISEMENT
Subjects
ADVERTISEMENT
Latest Questions
ADVERTISEMENT