Question

In: Advanced Math

Information theory Consider a random variable representing coin throws (Bernoulli Variable with Σ = {0,1} )....

Information theory

Consider a random variable representing coin throws (Bernoulli Variable with Σ = {0,1} ). Let the true
probability distribution be p(0) = r, p(1) = 1-r.
Someone guesses a different distribution q(0) = s, q(1) = 1-s.
(a) Find expressions for the Kullback–Leibler distances D(p||q) and D(q||p) between the
two distributions in terms of r and s.
(b) Show that in general, D(p||q) ≠ D(q||p) and that equality occurs iff r = s.
(c) Compute D(p||q) and D(q||p) for the case r = 1/2 and s = 1/4.

Solutions

Expert Solution

All the logarithms used are base 2 logarithms

Colby Messinger answered 1 year ago
Next > < Previous

Related Solutions

Consider a r.v. representing coin throws (Bernoulli Variable with Σ = {0,1} ). Let the true...
Consider a r.v. representing coin throws (Bernoulli Variable with Σ = {0,1} ). Let the true probability distribution be p(0) = r, p(1) = 1-r. Someone guesses a different distribution q(0) = s, q(1) = 1-s. (a) Find expressions for the Kullback–Leibler distances D(p||q) and D(q||p) between the two distributions in terms of r and s. (b) Show that in general, D(p||q) ≠ D(q||p) and that equality occurs iff r = s. (c) Compute D(p||q) and D(q||p) for the case...
Problem(3) Consider a r.v. representing coin throws (Bernoulli Variable with Σ = {0,1} ). Let the...
Problem(3) Consider a r.v. representing coin throws (Bernoulli Variable with Σ = {0,1} ). Let the true probability distribution be p(0) = r, p(1) = 1-r. Someone guesses a different distribution q(0) = s, q(1) = 1-s. (a) Find expressions for the Kullback–Leibler distances D(p||q) and D(q||p) between the two distributions in terms of r and s. (b) Show that in general, D(p||q) ≠ D(q||p) and that equality occurs iff r = s. (c) Compute D(p||q) and D(q||p) for the...
1. A coin is tossed 3 times. Let x be the random discrete variable representing the...
1. A coin is tossed 3 times. Let x be the random discrete variable representing the number of times tails comes up. a) Create a sample space for the event;    b) Create a probability distribution table for the discrete variable x;                 c) Calculate the expected value for x. 2. For the data below, representing a sample of times (in minutes) students spend solving a certain Statistics problem, find P35, range, Q2 and IQR. 3.0, 3.2, 4.6, 5.2 3.2, 3.5...
Consider a Bernoulli random variable X such that P(X=1) = p. Calculate the following and show...
Consider a Bernoulli random variable X such that P(X=1) = p. Calculate the following and show steps of your work: a) E[X] b) E[X2] c) Var[X] d) E[(1 – X)10] e) E[(X – p)4] f) E[3x41-x] g) var[3x41-x]
Consider two ips of a coin. Let X1 be the random variable which is 1 if...
Consider two ips of a coin. Let X1 be the random variable which is 1 if the rst coin is heads and 0 otherwise; let X2 be the random variable which is 1 if the second coin is heads and 0 otherwise; and let X3 be the random variable which is 1 if the two coins are the same and 0 otherwise. Show that these three variables are pairwise independent but not independent.
Let X be a random variable representing a quantitative daily market dynamic (such as new information...
Let X be a random variable representing a quantitative daily market dynamic (such as new information about the economy). Suppose that today’s stock price S0 for a certain company is $150 and that tomorrow’s price S1 can be modeled by the equation S1 = S0 · eX. Assume that X is normally distributed with a mean of 0 and a variance of 0.5. (a) Find the probability that X is less than or equal to 0.1 (b) Suppose the daily...
Let X be a random variable representing a quantitative daily market dynamic (such as new information...
Let X be a random variable representing a quantitative daily market dynamic (such as new information about the economy). Suppose that today’s stock price S0 for a certain company is $120 and that tomorrow’s price S1 can be modeled by the equation S1 = S0 · e X. Assume that X is normally distributed with a mean of 0 and a variance of 0.25. (a) Find the probability that X is less than or equal to 0.1. (b) Suppose the...
1. Consider a standard normal random variable with μ = 0 and standard deviation σ =...
1. Consider a standard normal random variable with μ = 0 and standard deviation σ = 1. (Round your answers to four decimal places.) (a)    P(z < 2) = (b)    P(z > 1.16) = (c)    P(−2.31 < z < 2.31) = (d)    P(z < 1.82) = 2. Find the following probabilities for the standard normal random variable z. (Round your answers to four decimal places.) (a)    P(−1.49 < z < 0.65) = (b)    P(0.56 < z < 1.74) = (c)    P(−1.54 < z < −0.46) = (d)    P(z...
1. Consider a standard normal random variable with μ = 0 and standard deviation σ =...
1. Consider a standard normal random variable with μ = 0 and standard deviation σ = 1. (Round your answers to four decimal places.) (a)    P(z < 2) = (b)    P(z > 1.16) = (c)    P(−2.31 < z < 2.31) = (d)    P(z < 1.82) = 2. Find the following probabilities for the standard normal random variable z. (Round your answers to four decimal places.) (a)    P(−1.49 < z < 0.65) = (b)    P(0.56 < z < 1.74) = (c)    P(−1.54 < z < −0.46) = (d)    P(z...
Let X be a random variable with mean μ and standard deviation σ. Consider a new...
Let X be a random variable with mean μ and standard deviation σ. Consider a new random variable Z, obtained by subtracting the constant μ from X and dividing the result by the constant σ: Z = (X –μ)/σ. The variable Z is called a standardised random variable. Use the laws of expected value and variance to show the following: a E(Z) = 0 b V (Z) = 1
ADVERTISEMENT
Subjects
ADVERTISEMENT
Latest Questions
ADVERTISEMENT