Question

home / study / math / statistics and probability / statistics and probability questions and answers / suppose that we fit model (1) to the n observations (y1, x11, x21), …, (yn, x1n, x2n). yi ... Your question has been answered Let us know if you got a helpful answer. Rate this answer Question: Suppose that we fit Model (1) to the n observations (y1, x11, x21), …, (yn, x1n, x2n). yi = β0 + ... Suppose that we fit Model (1) to the n observations (y1, x11, x21), …, (yn, x1n, x2n). yi = β0 + β1x1i + β2x2i + εi , i = 1, …., n, (1) where ε’s are identically and independently distributed as a normal random variable with mean zero and variance σ2, i = 1, …, n , and all the x’s are fixed. a) Suppose that Model (1) is the true model. Show that at any observation yi , the point estimator of the mean response and its residual are two statistically independent normal random variables. b) Suppose the true model is Model (1), but we fit the data to the following Model (2) (that is, ignore the variable x2). yi = β 0 + β 1x1i + εi , i = 1, …., n. Assume that average of x1 =0, average of x2=0. The sum of x1i and x2i equals 0. Derive the least-squares estimator of β1 obtained from fitting Model (2). Is this least-squares estimator biased for β1 under Model (1)?

Answer 1

OLS estimator. Let β₁-hat be the estimator; y-hat = β₁-hat * x be the predicted value, so u-hat = y - y-hat = y - β₁-hat*x is the residual.

To get the sum of squared residuals:

u-hat = y - β₁-hat*x

(u-hat)² = (y - β₁-hat*x)² = y² - 2*β₁-hat*x*y + β₁-hat²*x²

Σ (u-hat)² = Σ [y² - 2*β₁-hat*x*y + β₁-hat²*x²]  = Σ (y²) - Σ(2*β₁-hat*x*y) + Σ(β₁-hat²*x²)

Σ u-hat² = Σ y² - 2β₁-hat* Σ(x*y) + β₁-hat² * Σ x²

Take the derivative w.r.t. beta-hat, and set equal to zero, in order to minimize sum of squared residuals:

∂/∂β₁-hat [Σ y² - 2β₁-hat * Σ(x*y) + β₁-hat² * Σ x²] = 0 - 2Σ(x*y) + 2 β₁-hat* Σ x² = 0

2Σ(x*y) = 2 β₁-hat* Σ x²

β₁-hat = Σ(x*y) / Σ x²

Expected value: E[β₁-hat] = E[Σ(x*y) / Σ x² ] = E[Σ(x*(β₁x + u)) / Σ x² ] = E[Σ(β₁x² + xu)) / Σ x² ] =  E[β₁Σx² / Σ x² + xu / Σ x² ] = E[β₁Σx² / Σx² ] + E[ xu / Σ x² ] = E[β₁] + E[ xu / Σ x² ] = β₁ + E[ x/Σ x² * u]. If x and u are independent, then this last term is zero, so E[β₁-hat] = β₁. Yes, unbiased.

Var[β₁-hat] = E[(Σ(x*y) / Σ x² - β₁)²] = E[(Σ(x*(xβ₁+u)) / Σ x² - β₁)²] =  E[(Σ(x*(xβ₁+u)) / Σ x² - β₁)²] =  E[(Σ(x²β₁+xu)) / Σ x² - β₁)²] = E[(Σx²β₁+Σxu) / Σ x² - β₁)²] = E[(Σx²β₁/Σ x²+Σxu/Σ x²) - β₁)²] = E[β₁+Σxu/Σ x²) - β₁)²] = E[(Σxu/Σ x²)²] =E[(Σxu)²/(Σ x²)²] = E[(Σx²E[u]²/(Σ x²)²] = E[u]²/E[Σ x²].