Question

Handwritten Digit Recognition using Shallow ANN

The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image of 28-by-28.

Hint: The ANN should have 28*28=784 input nodes.

Data files:

• training_set: contains images of handwritten digits for training. Each picture is 28×28. There are 60,000 images.
• training_label: contains the digit label for each training image in the training set.
• testing_set: contains the images of handwritten digits for testing. There are 10,000 images.
• testing_label: contains the digit label for each testing image in the testing set.

Data input:

The data is stored in a very simple file format designed for storing vectors and multidimensional matrices. All the integers in the files are stored in the MSB first (high endian) format used by most non-Intel processors. You can use the functions loadMNISTImages and loadMNISTLabels to read in the training/testing set and label files respectively.

Data format for ANN:

Function loadMNISTImages will return an array of 784 rows, each column contains pixel values of an image.

Use the following code to convert labels into a 10-row array, each column represents one digit:

labels = loadMNISTLabels('training_label'); % initialize figure

labels = labels';  

labels(labels==0)=10;

labels=dummyvar(labels);      

Create an ANN:

Use patternnet(hiddenLayerSize, trainFcn)to create an ANN for pattern recognition. You may need to try different hiddenLayerSize and training function to obtain good results.

Testing:

After the ANN is trained, use the testing_set/label files to verify the ANN. You must SYSTEMATICALLY test the performance of your system, i.e. use the 20% of data as testing examples and report the overall accuracy, using different number of hidden nodes and training functions. Plotting will be helpful to visualize your results.

NEED TO WRITE CODE IN MATLAB

Answer 1

step1:Loading the MNIST training data

The loading function loads both the images and associated lables at once. However we need to specify first the file with images followed by the label file. It takes a few seconds to load the data and when we done, we will be notified in the command window.

% loading training images and lables imageFile='train-images.idx3-ubyte'; labelFile='train-lables.idx1-ubyte'; //If you get an error above two linesremove them. labels=loadMNISTLabels('training_label'); labels=labels; lables=dummyvar(labels); images=loadMNISTImages('training_label'); images=images; images=dummyvar(images);

loading digit images..

file contains 60000 digit images

done

loading labels..

file contains 60000 image lables

done.

The output from the loading function [images,labels] is the images and labels as the names suggest. The digits images are loaded int he form of 3D matrix with the dimension of 28 by 28 by 60000, where each element in the third dimension is image.

% The dimension (size) of the loaded training images disp(size(images));

28 28 60000

The lables are loaded into a column vector with 60000 elements

% the dimension (size) of loaded training labels disp(size(labels));

60000

we will show a digit image by using the imagesc

% Displaying the three fist images for i=1:3 figure('position',[500 500 200 200]); Img=images(:,:,i); Lbl=labels(i); imagesc(Img); colormap('gray'); colorbar; axis image off title(num2str(Lbl)); end

step 2:structure the training data correctly.

The Neutral network ca't use 3D matrix structure of the digit images.so, we need to transform all the images into 2D matrix(784by60000), where each column contain pixel values from each of training images. This is done by satcking all the 28 columns of the training images so that each image becomes 28by 28 is 784 column vector.

%Reshapeing the 28x28 image in to a column vector col_img=reshape(images(:,:,1),[ ],1);

step3:creating a hidden layer

hid=hidden(numInputs,numLayers,biasConnect,inputConnect,layerConnect,outputConnect)

we will remember always get the documentation for a function of doc followed by the function name in command window.

% open the documentations of the hidden functio doc hidden

% create a custom hidden layer with four layers hid1=hidden(1,4,[1; 1; 1; 1],[1; 0; 0; 0],[0 0 0 0;1 0 0 0;0 1 0 0; 0 0 1 0],[0 0 0 1]); %rename the layers hid1.layers{1}.name='Hidden Layer 1'; hid1.layers{2}.name='Hidden Layer 2'; hid1.layers{3}.name='Hidden Layer 3'; hid1.layers{4}.name='ouput Layer'; %view in the hidden view(hid1);

step4:configuration

% set the number of neuron in hidden layer 1 to 20 hid.layers{1}.size=20; % set the number of neuron in hidden layer 2 to 20 hid.layers{2}.size=20;

we see the current value or setting simply by typing the structure element into the command window.

%get the current function used in hidden layer1 hid.layers{1}.transferFcn; %get the current function used in hidden layer 2 hid.layers{2}.transferFcn;