Question

write a Matlab program to implement the perceptron learning rule to adjust the value of weight w. You can choose your own learning rate and stop criteria. Try different values of the learning rate and stop criteria, what do you find?

Answer 1

Solution:

Create a Perceptron:-

You can create a perceptron with the following:

net = perceptron;
net = configure(net,P,T);

where input arguments are as follows:

• P is an R-by-Q matrix of Q input vectors of R elements each.

• T is an S-by-Q matrix of Q target vectors of S elements each.

Commonly, the hardlim function is used in perceptrons, so it is the default.

The following commands create a perceptron network with a single one-element input vector with the values 0 and 2, and one neuron with outputs that can be either 0 or 1:

P = [0 2];
T = [0 1];
net = perceptron;
net = configure(net,P,T);

You can see what network has been created by executing the following command:

inputweights = net.inputweights{1,1}

which yields

inputweights = 
        delays: 0
       initFcn: 'initzero'
         learn: true
      learnFcn: 'learnp'
    learnParam: (none)
          size: [1 1]
     weightFcn: 'dotprod'
   weightParam: (none)
      userdata: (your custom info)

The default learning function is learnp, which is discussed in Perceptron Learning Rule (learnp). The net input to the hardlim transfer function is dotprod, which generates the product of the input vector and weight matrix and adds the bias to compute the net input.

The default initialization function initzero is used to set the initial values of the weights to zero.

Similarly,

biases = net.biases{1}

gives

biases = 
       initFcn: 'initzero'
         learn: 1
      learnFcn: 'learnp'
    learnParam: []
          size: 1
      userdata: [1x1 struct]

You can see that the default initialization for the bias is also 0.

Perceptron Learning Rule (learnp):-

Perceptrons are trained on examples of desired behavior. The desired behavior can be summarized by a set of input, output pairs

p1t1,p2t1,…,pQtQ

where p is an input to the network and t is the corresponding correct (target) output. The objective is to reduce the error e, which is the difference t − a between the neuron response a and the target vector t. The perceptron learning rule learnp calculates desired changes to the perceptron's weights and biases, given an input vector p and the associated error e. The target vector t must contain values of either 0 or 1, because perceptrons (with hardlim transfer functions) can only output these values.

Each time learnp is executed, the perceptron has a better chance of producing the correct outputs. The perceptron rule is proven to converge on a solution in a finite number of iterations if a solution exists.

If a bias is not used, learnp works to find a solution by altering only the weight vector w to point toward input vectors to be classified as 1 and away from vectors to be classified as 0. This results in a decision boundary that is perpendicular to w and that properly classifies the input vectors.

There are three conditions that can occur for a single neuron once an input vector p is presented and the network's response a is calculated:

CASE 1. If an input vector is presented and the output of the neuron is correct (a = t and e = t – a = 0), then the weight vector w is not altered.

CASE 2. If the neuron output is 0 and should have been 1 (a = 0 and t = 1, and e = t – a = 1), the input vector p is added to the weight vector w. This makes the weight vector point closer to the input vector, increasing the chance that the input vector will be classified as a 1 in the future.

CASE 3. If the neuron output is 1 and should have been 0 (a = 1 and t = 0, and e = t – a = –1), the input vector p is subtracted from the weight vector w. This makes the weight vector point farther away from the input vector, increasing the chance that the input vector will be classified as a 0 in the future.

The perceptron learning rule can be written more succinctly in terms of the error e = t – a and the change to be made to the weight vector Δw:

CASE 1. If e = 0, then make a change Δw equal to 0.

CASE 2. If e = 1, then make a change Δw equal to p^T.

CASE 3. If e = –1, then make a change Δw equal to –p^T.

All three cases can then be written with a single expression:

Δw=(t−α)pT=epT

You can get the expression for changes in a neuron's bias by noting that the bias is simply a weight that always has an input of 1:

Δb=(t−α)(1)=e

For the case of a layer of neurons you have

ΔW=(t−a)(p)T=e(p)T

and

Δb=(t−a)=e

The perceptron learning rule can be summarized as follows:

Wnew=Wold+epT

and

bnew=bold+e

where e = t – a.

Now try a simple example. Start with a single neuron having an input vector with just two elements.

net = perceptron;
net = configure(net,[0;0],0);

To simplify matters, set the bias equal to 0 and the weights to 1 and -0.8:

net.b{1} =  [0];
w = [1 -0.8];
net.IW{1,1} = w;

The input target pair is given by

p = [1; 2];
t = [1];

You can compute the output and error with

a = net(p)
a =
     0
e = t-a
e =
     1

and use the function learnp to find the change in the weights.

dw = learnp(w,p,[],[],[],[],e,[],[],[],[],[])
dw =
     1     2

The new weights, then, are obtained as

w = w + dw
w =
    2.0000    1.2000

The process of finding new weights (and biases) can be repeated until there are no errors. Recall that the perceptron learning rule is guaranteed to converge in a finite number of steps for all problems that can be solved by a perceptron. These include all classification problems that are linearly separable. The objects to be classified in such cases can be separated by a single line.

You might want to try the example nnd4pr. It allows you to pick new input vectors and apply the learning rule to classify them.