Question

Subject: Neural network and Pattern recognition (Deep Learning)

Given the following partial network definitioon:

Input - A 28 x 28 RGB image

First Layer - 7, 5 x 5 filters. Activation function: Sigmoid

Second Layer - 23, 3 x 3 filters. Activation function: RelU

Third Layer - 20, 3 x 3 filters. Activation function: RelU

You task is to do design(s) of the next part of the network (so that it uses the output of the third layer).

The purpose of the complete network will be to predict a probability distribution over 5 classes.

You are required to do two designs.

- In the first design you can use fully connected layers.

- In the second design you are cannot use fully connected layers but instead must use convolutional layers.

Your design definition should include

- Any operation you are performing (e.g. flattening, reshaping etc.)

- Any activation function you want to use

- The number of neurons (for fully connected layers)

- The number of filters, their kernel size, and response shape (for convolutional layers)

Answer 1

For the above problem using pytorch to solve the problem

load the requires packages

above mentioned the we can any number size of layers, so i choosed to 512

first i like to solve mlp and fully connected network using python

and the input size in fully connected layers include print so shapes of network will sure the final result

class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 7, kernel_size=5, stride=1, padding=2),
            nn.Sigmoid(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(7, 23, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(23, 20, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        
        self.drop_out = nn.Dropout()
        self.fc1 = nn.Linear(784, 1000)
        self.fc2 = nn.Linear(1000, 10)
     def forward(self, x):
        out = self.layer1(x)
        out = self.layer2(out)
        out = self.layer3(out)
        out = out.reshape(out.size(0), -1)
        out = self.drop_out(out)
        out = self.fc1(out)
        out = self.fc2(out)
    return out
net = ConvNet()
print(net

import torch.nn as nn
import torch.nn.functional as F

# define the NN architecture
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        # number of hidden nodes in each layer (512)
        hidden_1 = 512
        hidden_2 = 512
        # linear layer (784 -> hidden_1)
        self.fc1 = nn.Linear(28 * 28, hidden_1)
        # linear layer (n_hidden -> hidden_2)
        self.fc2 = nn.Linear(hidden_1, hidden_2)
        # linear layer (n_hidden -> 10)
        self.fc3 = nn.Linear(hidden_2, 5)
        # dropout layer (p=0.2)
        # dropout prevents overfitting of data
        self.dropout = nn.Dropout(0.2)

    def forward(self, x):
        # flatten image input
        x = x.view(-1, 28 * 28)
        # add hidden layer, with relu activation function
        x = F.sigmoid(self.fc1(x))
        # add dropout layer
        x = self.dropout(x)
        # add hidden layer, with relu activation function
        x = F.relu(self.fc2(x))
        # add dropout layer
        x = self.dropout(x)
        # add output layer
        x = self.fc3(x)
        return x
network = Net()
print(network)