Question

Suppose your dataset has a large number of features. What effect, if any, would feature selection have on an SVM? And what is the effect of raising or lowering the λ hyper-parameter in an SVM?

Answer 1

Effect of feature selection on SVM:

Feature selection is very important in SVM. This is because it enables the Support Vector classifier to train faster. The model complexity is also minimized at the same time. The model can easily interpret the results. It also helps in improving the accuracy of the model.

Hyper-parameter in SVM:

Hyper-parameters are necessary for the learning process. They also control the training and performance behaviour of the model. Raising/lowering the value of hyperparameter affects the margin.

• If the value is raised, margin will be of smaller width. If the value is significantly lowered, margin will increase.
• In both the cases, one has to keep in mind that value of hyperparameter also has an impact on classification of points due to increase or decrease in the width of margin or hyperplane. However, it depends on the given situation and given training data.
• If the value of hyperparameter is increased, the optimization function of SVM tends to minimize the loss function and over fit the data.