Question

(1) For linear regression with multiple variables, why do we need to do feature scaling?

(2) How does the learning rate alpha influence the gradient descent algorithm?

Answer 1

1.

Rescaling matters for algorithms that have constraints or penalties based on coefficient sizes or differences. This includes most machine learning and search algorithms.

The only other reason to rescale is to make the estimated coefficients easier to interpret. This makes the most sense when your independent variables have arbitrary scales in the first place. For example, if you regressed college graduation class rank on candidate SAT scores and high school GPAs, the raw coefficients would have little meaning. If you rescaled your independent variables, then the size of the coefficients is directly meaningful. If you rescaled to mean 0 and standard deviation 1, for example, a coefficient of -0.1 on SAT score would tell you someone with a one standard deviation better SAT score is predicted to have a 0.1 standard deviation lower class rank.

2.

alpha must be specified that controls how much the coefficients can change on each update.

coefficient = coefficient – (alpha * delta)

This process is repeated until the cost of the coefficients (cost) is 0.0 or close enough to zero to be good enough.

Please comment in case of doubts, please upvote the solution