Question

We can apply PCA to reduce features in a data set for model construction. But, why do we still
need regularization?
What is the difference between lasso and ridge regression? What is the role of hyper parameter in regularization task?

Answer 1

PCA help with a better estimation of coefficients so it reduces variance, if the variables were highly correlated. Because PCA will modify your features and they can end up as one feature this way.Regularization will most likely give all glory to one of the variables and regularize the other away.

Lasso regression	Ridge regression
1.It uses l1 penalty function	1.It uses l2 penalty function
2.In Lasso regression, the penalty is the sum of the absolute values of the coefficients.	2. In ridge regression, the penalty is the sum of the squares of the coefficients .

In machine learning, hyperparameter is a parameter whose value is used to control the learning process.Different model training algorithms require different hyperparameters, some simple algorithms like ordinary least squares regression requires none. Given these hyperparameters, the training algorithm learns the parameters from the data. For example, Lasso is an algorithm that adds a regularization hyperparameter to ordinary least squares regression, which has to be set before estimating the parameters through the training algorithm.