Question

Show that the conclusion of the mean-square law of large numbers, remains
valid if the assumption that the Xi are independent is replaced by the weaker assumption that they are
uncorrelated.

Answer 1

Mean square law of large numbers :

​​​​I have numbers  the following line in my notes (which I believe is flawed):

E[(μ^(N)−μ)2]=E⎡⎣(1N∑i=1N(xi−μ))2⎤⎦=σ2NE[(μ^(N)−μ)2]=E[(1N∑i=1N(xi−μ))2]=σ2/N

I think the error is in brackets, 1N1N us not multiplied by ∑Ni=1(xi−μ)∑i=1N(xi−μ), but only: ∑Ni=1xi∑i=1Nxi, since μ^μ^ is defined as the sample mean: 1N∑Ni=1xi1N∑i=1Nxi.

But even given that (and supposing that I am correct), I cannot arrive at the required result of σ2/N

This is what I get:

E⎡⎣(1N∑i=1Nxi−μ)2⎤⎦=1N2E[∑i=1Nx2i]−2μNE[∑i=1Nxi]+μ2

According to the law, the average of the results obtained from a large number of trials should be close to the expected value and will tend to become closer to the expected value as more trials are performed.

Example ;

For i.i.d. random variables X1,X2,...,XnX1,X2,...,Xn, the sample mean, denoted by X¯¯¯¯X¯, is defined as

X¯¯¯¯=X1+X2+...+Xnn.X¯=X1+X2+...+Xnn.

Another common notation for the sample mean is MnMn. If the XiXi's have CDF FX(x)FX(x), we might show the sample mean by Mn(X)Mn(X) to indicate the distribution of the XiXi's.

Note that since the XiXi's are random variables, the sample mean, X¯¯¯¯=Mn(X)X¯=Mn(X), is also a random variable. In particular, we have

E[X¯¯¯¯]=EX1+EX2+...+EXnn=nEXn=EX.(by linearity of expectation)(since EXi=EX)E[X¯]=EX1+EX2+...+EXnn(by linearity of expectation)=nEXn(since EXi=EX)=EX.

Also, the variance of X¯¯¯¯X¯ is given by

Var(X¯¯¯¯)=Var(X1+X2+...+Xn)n2=Var(X1)+Var(X2)+...+Var(Xn)n2=nVar(X)n2=Var(X)n.(since Var(aX)=a2Var(X))(since the Xi's are independent)(since Var(Xi)=Var(X))

It is weak assumption :

Now let us state and prove the weak law of large numbers (WLLN).

The weak law of large numbers (WLLN)

Let X1X1, X2X2 , ... , XnXn be i.i.d. random variables with a finite expected value EXi=μ<∞EXi=μ<∞. Then, for any ϵ>0ϵ>0,

limn→∞P(|X¯¯¯¯−μ|≥ϵ)=0.

The proof of the weak law of large number is easier if we assume Var(X)=σ2Var(X)=σ2 is finite. In this case we can use Chebyshev's inequality to write

P(|X¯¯¯¯−μ|≥ϵ)=Var(X)nϵ2,≤Var(X¯¯¯¯)ϵ2P(|X¯−μ|≥ϵ)≤Var(X¯)ϵ2=Var(X)nϵ2,

which goes to zero as n→∞n→∞.

Please rate !