In: Statistics and Probability
Collinearity
a. Discuss the problems that result when collinearity is present in a regression analysis.
b. How can you detect collinearity?
c. What remedial measures are available when collinearity is detected?
Statistical indicatiors of collinearity and its affect on regression models are :
1.) The Variance Inflation Factor (VIF) is 1/Tolerance, it is always greater than or equal to 1. There is no formal VIF value for determining presence of multicollinearity. Values of VIF that exceed 10 are often regarded as indicating multicollinearity, but in weaker models values above 2.5may be a cause for concernarge changes in the estimated regression coefficients when a predictor variable is added or deleted
2.)Insignificant regression coefficients for the affected variables in the multiple regression, but a rejection of the joint hypothesis that those coefficients are all zero (using anF-test)
3. If a multivariable regression finds an insignificant coefficient of a particular explanator, yet a simple linear regression of the explained variable on this explanatory variable shows its coefficient to be significantly different from zero, this situation indicates multicollinearity in the multivariable regression.
4.) Large changes in the estimated regression coefficients when a predictor variable is added or deleted
Strategies to handle it::
A major limitation of ridge regression is that ordinary inference procedures are not applicable and exact distributional properties are not known. Another limitation is that the choice of the biasing constant k is a judgmental one. While formal methods have been developed foo making this choice, these methods have their own limitations.