Question

When performing hyperparameter tuning of a neural network, why would random hyper-parameter search be more likely to converge compared to grid-search?

Answer 1

Ans: - Hyperparameters in machine learning are those parameters which remains fixed and controlled according to the learning process of the model. In neural network the number of hidden layers can be the example of hyperparameter.

Hyperparameter tuning of a neural network is the complete designing and shaping of the model by finding the best suitable hyperparameter for the high accuracy and precision of the model. While performing hyperparameter tuning it is not and easy task to find the right set of the hyperparameter. But by finding the right hyperparameter for the learning model we can achieve high accuracy and best performance of the model.

There are two main methods of hyperparameter tunning technique which are as follows: -

Grid Search and Random Search

Both of the above techniques are effective to find the right set of the hyperparameters depending upon the different problems and models.

Grid Search: -

In this technique of finding right set for the model, the model is prepared for the combination of each and every hyperparameter. Every model for the combination of hyperparameter values is analyzed and evaluated. Then the model which is providing the highest accuracy and precision is considered as the best and right set of hyperparameters for the model. This technique is not considered as the best technique for finding the right set of hypermeters as when the problem needs to be evaluated dimensionally then there is large increase in number the hyperparameters. Also, this technique does not give the guarantee to find the perfect solution for the learning model.

Random Search: -

In this method the hyperparameter combinations are formed randomly to find the best set of hyperparameter for the model. Any combination of hyperparameter value is formed and then evaluated. It can give good results as compared to grid search as it picks the random values instead of evaluating each and every combination of hyperparameters.

That’s why the Random search is considered as the better tunning technique for the neural network. There are higher chances of getting the right and optimized set of parameters as model may be fully trained for the high accuracy and precision without aliasing.

In neural network there are large number of parameters that are needed in the training of the model. By performing hyperparameter tuning we can get the hyperparameters with full precision and accuracy. So, the random search is better option to find the best suitable set of parameters from the large number of parameters.