Question

Consider two functions
f (x; y) = (x10)2 + (y + 2)2; and
g(x; y) = (x ? 10)2 + (x ? y + 5)4
Please implement a basic gradient descent algorithm using python
Starting with (x; y) = (0; 0), and set up a xed learning rate = 0:5,
run the gradient descent algorithm for each function. Run for 10
iterations, and report both (x; y) and the function value at the end of
each iteration.
Adjust the , and nd out the fastest convergence (smallest function
value) after T steps (try T = 10 or 100).

Answer 1

The Matlab code for your assignment is as GradientDescent1.m for first function and GradientDescent2 for second
*******************************GradientDescent1.m*************************
x0=0;

y0=0;

%alpha is the rate by which we update the x and y values

%at alpha=0.5 is the convergence fastest

alpha=0.5;

tol=0.001;

%the following is the function wrt x and y

f_xy=@(x,y)(x-10).^2+(y+2).^2;
%following are the derivatives wrt x and y

dfdx=@(x) 2*(x-10);
dfdy=@(y) 2*(y+2);
%negative of derivatives give the descent

s1=-dfdx(x0);

s2=-dfdy(y0);
x_next=x0+alpha*s1;

y_next=y0+alpha*s2;
err=inf;

i=0;

while i<10

  

i=i+1;

err=(f_xy(x_next,y_next)-0);

x0=x_next;

y0=y_next;

  

s1=-dfdx(x0);

s2=-dfdy(y0);

%update the x and y values

% using gradient descent

x_next=x0+alpha*s1;

y_next=y0+alpha*s2;

fprintf('x=%f,y=%f, f(x,y)=%f\n',x0,y0,f_xy(x0,y0))

end

*****************************GradientDescent2.m*************************************
x0=0;

y0=0;

%alpha is the rate by which we update the x and y values

%alpha less or equal to 0.01 works

%otherwise it offshoots

alpha=0.01;

tol=0.001;

%the following is the function wrt x and y

f_xy=@(x,y)(x-10).^2+(x-y+5).^4;
%following are the derivatives wrt x and y

dfdx=@(x,y) 2*(x-10)+4*(x-y+5)^3;
dfdy=@(x,y) -1*4*(x-y+5)^3;
%negative of derivatives give the descent

s1=-dfdx(x0,y0);

s2=-dfdy(x0,y0);
x_next=x0+alpha*s1;

y_next=y0+alpha*s2;
err=inf;

i=0;

while i<100

  

i=i+1;

err=(f_xy(x_next,y_next)-0);

x0=x_next;

y0=y_next;

  

s1=-dfdx(x0,y0);

s2=-dfdy(x0,y0);

%update the x and y values

% using gradient descent

x_next=x0+alpha*s1;

y_next=y0+alpha*s2;

fprintf('x=%f,y=%f, f(x,y)=%f\n',x0,y0,f_xy(x0,y0))

end