Question

Find the 'least square linear regression line' for the following cases:

(x,y)coordinates (-1,0), (0,2), (1,4), (2,5)

(However, use the gradient descent method and use cost function to get it)

(Explain the changing process of cost functuon, gradient, and intercept together)

Answer 1

In statistics, linear regression is a linear approach to modelling the relationship between a dependent(or regressand) variable and one or more independent (predictor)variables. Let X be the independent variable and Y be the dependent variable. We will define a linear relationship between these two variables as follows:

y = mx +c

This is the equation for a line that you studied already in school life . m is the slope of the line and c is the y intercept. We will use this equation to train our model with a given dataset and predict the value of Y for any given value of X. Our challenge is to determine the value of m and c, such that the line corresponding to those values is the best fitting line or gives the minimum error.

First of all calculate the difference between the actual y and predicted y value(y = mx + c), for a given x and square this difference and then calculate the mean of the squares for every value in X.

error = (1/n) { y -(mx+c)}^2

So we square the error and find the mean which is called  Mean Squared Error. Now that we have defined the loss function, lets get into the interesting part — minimizing it and finding m and c.

The Gradient Descent Algorithm

Gradient descent is an iterative optimization algorithm to find the minimum of a function. Here that function is our loss function.

.D is equivalent to the steepness of the slope and L can be the speed with which he moves. Now the new value of m that we calculate using the above equation will be his next position, and L×D will be the size of the steps he will take. When the slope is more steep (D is more) he takes longer steps and when it is less steep (D is less), he takes smaller steps. Finally he arrives at the bottom of the valley which corresponds to our loss = 0.