Question

The RMSE on the test set is 51.279 and the RMSE on the Validation Set is 56.455. Compare the two and please comment.

The average error on the test set is 1.017. What does it suggest?

Answer 1

Answer: -

RMSE or Root-Mean-Square-Error is used to measure to measure the difference between sample values predicted by model and values actually observed.

Residuals are a measure of how far from regression line data points are, RMSE is a measure of how spread out these residuals are that means how concentrated the data are around the line of best fit. RMSE is commonly used in forecasting, regression analysis to verify experimental outcomes.

RMSE = SQRT [(f - o)^2)] where f is model forecast and o is observed value.

The test given in the question not much data is given hence it is assumed to be carrying 3 parts of data set in Training set, Validation Set and Test Set. Training set is used to fit the model parameter or adjust the weights, validation set is fine tune the model parameters whereas test set gives the accuracy of model or performance on unseen data in order to confirm the predictive power of the model. With this understanding we can state that the given model in question has been fine tuned to 56.455 whereas the model performance is at 51.279.

As new data is entering the model, test set which is the performance or accuracy of model is carrying an average error of 1.017.