Question

My professor asked us if the least square estimated regression equation is indeed the best unique linear combination of predictor variables. I said yes but he asked me if there are other potential linear combination of the predictor variables that may work that could provide insight? he then asked that I give an example. what other linear combinations are there???

Answer 1

Normally, least square estimated regression equation is indeed the best unique linear combination of predictor variables as we minimize the error in this regard to get the equation... But some other combinations can be:

(1) More complex models may include higher powers of one or more predictor variables:

For example:

(2) or, interaction effects of two or more variables can be stated as:

Models of the above mentioned type can be designated as linear regression models as they can be stated as linear combinations of the β parameters in the model.

Confusingly, the model represented in part 1 are also sometimes referred as non-linear regression models or polynomial regression models, as the regression curve is not a line.

Models of Part 2 are usually called linear models with interaction terms.

Others can be of type:

1. Y = A +B(LnX) (Logarithmic Regression)

2. Y = AB^x or Y = Ae^x (Exponential)

and many more...