Question

A computer is shared by 2 users who send tasks to a computer remotely and work independently. At any minute, any connected user may disconnect with probability 0.5, and any disconnected user may connect with a new task with probability 0.2. Let X(t) be the number of concurrent users at time t (in minutes). This is a Markov chain with 3 states: 0, 1, and 2. The probability transition matrix can be calculated and it is Generate 10000 transitions of the Markov chain. What percent of times do you find your generated Markov chain in each of its 3 states? Write using R code.

Answer 1

The transition probability from state 0 to 0 is 0.8 * 0.8 = 0.64 ((both disconnected stays connected )

The transition probability from state 0 to 1 is 0.2 * 0.8 + 0.8 * 0.2 = 0.32 ((A disconnected user connect and another one stays disconnected and vice-versa)

The transition probability from state 0 to 2 is 0.2 * 0.2 = 0.04 ((both disconnected user now connected)

The transition probability from state 1 to 0 is 0.5 * 0.8 = 0.4 ((A connected user disconnect and disconnected user stay disconnect)

The transition probability from state 1 to 1 is 0.5 * 0.2 + 0.5 * 0.8 = 0.5 (A connected user disconnect and disconnected user connects or A connected user stay connected and disconnected user stay disconnected)

The transition probability from state 1 to 2 is 0.5 * 0.2 = 0.1 ((A connected user stay connected and disconnected user connects)

The transition probability from state 2 to 0 is 0.5 * 0.5 = 0.25 ((both connected user disconnects)

The transition probability from state 2 to 1 is 0.5 * 0.5 + 0.5 * 0.5 = 0.5 ((A connected user disconnect and another one connected and vice-versa)

The transition probability from state 2 to 2 is 0.5 * 0.5 = 0.25 ((both stay connected)

The transition probability matrix is,

R code :

p = matrix(data = c(0.64,0.4,0.25,0.32,0.5,0.5,0.04,0.1,0.25), nrow = 3)
all.states = c(0,1,2)
state = 0 #initial state
transitions = c()
for (i in 1:10000) {
if(state == 0) {
state = sample(all.states, 1, prob = c(0.64,0.32,0.04)) #Get next state transitioning from state 0
}
if(state == 1) {
state = sample(all.states, 1, prob = c(0.4,0.5,0.1)) #Get next state transitioning from state 0
}
if(state == 2) {
state = sample(all.states, 1, prob = c(0.25,0.5,0.25)) #Get next state transitioning from state 0
}
transitions = c(transitions, state) #Append states
}
sum(transitions == 0)/10000
sum(transitions == 1)/10000
sum(transitions == 2)/10000

After running the R code, we get the output as,

> sum(transitions == 0)/10000
[1] 0.6482

> sum(transitions == 1)/10000
[1] 0.3227

> sum(transitions == 2)/10000
[1] 0.0291

Thus, percent of times the generated Markov chain is in each of its 3 states are 64.82%, 32.27% and 2.91%