Computer Vision Neural Network Project: 11/15/2020 summary

import torch

import torchvision

import torchvision.transforms as transforms

transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))]);

trainset = torchvision.datasets.CIFAR10(root = './data', train=True, download=True, transform=transform);

trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, shuffle=True, num_workers=2);

testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform);

testloader = torch.utils.data.DataLoader(testset, batch_size = 4, shuffle = False, num_workers = 2);

classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck');

TorchVision contains all of the important datasets which are used in modern-day computer vision algorithms. I download the training and testing datasets of birds, planes, cars, cats, deer, dog, frog, ship, and trucks. 

The next few lines help to show some training images. 

While running the code, I encountered the following error: 

RuntimeError: 

        An attempt has been made to start a new process before the

        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your

        child processes and you have forgotten to use the proper idiom

        in the main module:

            if __name__ == '__main__':

                freeze_support()

                ...

        The "freeze_support()" line can be omitted if the program

        is not going to be frozen to produce an executable.

This thread shows how to solve the issue, and I have to put if __name__ == '__main__' as the thread in Python. 

My code now looks like this: 

# -*- coding: utf-8 -*-

"""

Spyder Editor

This is a temporary script file.

"""

import torch

import torchvision

import torchvision.transforms as transforms

import matplotlib.pyplot as plt

import numpy as np

def imshow(img):

    img = img / 2 + 0.5     # unnormalize

    npimg = img.numpy()

    plt.imshow(np.transpose(npimg, (1, 2, 0)))

    plt.show()

    

transform = transforms.Compose(

    [transforms.ToTensor(),

     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,

                                        download=True, transform=transform)

trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,

                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,

                                       download=True, transform=transform)

testloader = torch.utils.data.DataLoader(testset, batch_size=4,

                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',

           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

if __name__ == '__main__':

    classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

    # get some random training images

    dataiter = iter(trainloader)

    images, labels = dataiter.next()

    # show images

    imshow(torchvision.utils.make_grid(images))

    # print labels

    print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

and received the following output: 

runfile('C:/Users/cchu3/.spyder-py3/temp.py', wdir='C:/Users/cchu3/.spyder-py3')

Files already downloaded and verified

Files already downloaded and verified

Figures now render in the Plots pane by default. To make them also appear inline in the Console, uncheck "Mute Inline Plotting" under the Plots pane options menu. 

  deer  bird  bird truck

Here is the output so far, when I click the "plots" in Spyder: 

The image is blurred, but should provide sufficient information to figure things out. 

This is the part of the code that loads the CIFAR10 Library. The next step is to define a Convolutional Neural Network. Here, I define a convolution neural network and a pooling network. We use the torch.nn library and torch.nn.functional library.

Now a quick lesson on what max pooling is in a convolutional neural network 

Max pooling reduces the dimensionality of images by reducing the pixels from the previous layer. We define some nxn region as a corresponding filter for the max pooling operation. 

Then we define a stride, how many pixels should the filter move as it moves across the image. Take the first 2x2 image and calculate the maximum value. Carry out this process for the entire image. 

Pool, then rectify, and then put into a linear neural network function.

The rectifier that this function is using is linear.

Printing the net gives the following output: 

Net(

  (conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))

  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)

  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))

  (fc1): Linear(in_features=400, out_features=120, bias=True)

  (fc2): Linear(in_features=120, out_features=84, bias=True)

  (fc3): Linear(in_features=120, out_features=84, bias=True)

)

I define a new class as Net and initialize Net() in the main function and the code is as follows now: 

# -*- coding: utf-8 -*-

"""

Spyder Editor

This is a temporary script file.

"""

import torch

import torchvision

import torchvision.transforms as transforms

import matplotlib.pyplot as plt

import numpy as np

import torch.nn as nn

import torch.nn.functional as F

class Net(nn.Module):

    def __init__(self):

        super(Net, self).__init__()

        self.conv1 = nn.Conv2d(3, 6, 5)

        self.pool = nn.MaxPool2d(2,2)

        self.conv2 = nn.Conv2d(6, 16, 5)

        self.fc1 = nn.Linear(16 * 5 * 5, 120)

        self.fc2 = nn.Linear(120, 84)

        self.fc3 = nn.Linear(120, 84)

        

    def forward(self, x):

        x = self.pool(F.relu(self.conv1(x)))

        x = self.pool(F.relu(self.conv2(x)))

        x = x.view(-1, 16 * 5 * 5)

        x = F.relu(self.fc1(x))

        x = F.relu(self.fc2(x))

        x = self.fc3(x)

        return x

def imshow(img):

    img = img / 2 + 0.5     # unnormalize

    npimg = img.numpy()

    plt.imshow(np.transpose(npimg, (1, 2, 0)))

    plt.show()

    

transform = transforms.Compose(

    [transforms.ToTensor(),

     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,

                                        download=True, transform=transform)

trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,

                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,

                                       download=True, transform=transform)

testloader = torch.utils.data.DataLoader(testset, batch_size=4,

                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',

           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

if __name__ == '__main__':

    classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

    # get some random training images

    dataiter = iter(trainloader)

    images, labels = dataiter.next()

    # show images

    imshow(torchvision.utils.make_grid(images))

    # print labels

    print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

    net = Net()

    

The next step is to define a loss function and optimizer. The loss function that is to be defined here is the cross-entropy loss, and SGD with momentum. The cross-entropy loss measures the performance of a classification model whose output is a probability value between 0 and 1. SGD is stochastic gradient descent, which is an iterative method for maximizing an objective function and achieve faster iterations than normal gradient descent. 

The next step is to train the network, looping over the data iterator, feed the inputs to the network, and optimize.

Then, save the path.

Here is the result:

runfile('C:/Users/cchu3/.spyder-py3/temp.py', wdir='C:/Users/cchu3/.spyder-py3')

Files already downloaded and verified

Files already downloaded and verified

  car plane  frog  bird

[1,  2000] loss: 2.188

[1,  4000] loss: 1.860

[1,  6000] loss: 1.695

[1,  8000] loss: 1.592

[1, 10000] loss: 1.538

[1, 12000] loss: 1.461

[2,  2000] loss: 1.377

[2,  4000] loss: 1.374

[2,  6000] loss: 1.333

[2,  8000] loss: 1.289

[2, 10000] loss: 1.287

[2, 12000] loss: 1.290

[3,  2000] loss: 1.206

[3,  4000] loss: 1.197

[3,  6000] loss: 1.177

[3,  8000] loss: 1.198

[3, 10000] loss: 1.164

[3, 12000] loss: 1.181

[4,  2000] loss: 1.104

[4,  4000] loss: 1.094

[4,  6000] loss: 1.116

[4,  8000] loss: 1.103

[4, 10000] loss: 1.097

[4, 12000] loss: 1.089

[5,  2000] loss: 1.021

[5,  4000] loss: 1.032

[5,  6000] loss: 1.035

[5,  8000] loss: 1.032

[5, 10000] loss: 1.021

[5, 12000] loss: 1.044

Finished Training

As one can see, the error in total was optimized to around 1. 

The next step is to test the network. There needs to be a check in if the network has learned anything at all. It also shows a clever way to join multiple statements in a for loop.

We want to compare the predicted to the ground truth.

Here is the final file output.

runfile('C:/Users/cchu3/.spyder-py3/temp.py', wdir='C:/Users/cchu3/.spyder-py3')

Files already downloaded and verified

Files already downloaded and verified

  cat   car  ship  ship

[1,  2000] loss: 2.240

[1,  4000] loss: 1.869

[1,  6000] loss: 1.652

[1,  8000] loss: 1.573

[1, 10000] loss: 1.509

[1, 12000] loss: 1.464

[2,  2000] loss: 1.397

[2,  4000] loss: 1.395

[2,  6000] loss: 1.367

[2,  8000] loss: 1.320

[2, 10000] loss: 1.313

[2, 12000] loss: 1.269

[3,  2000] loss: 1.223

[3,  4000] loss: 1.212

[3,  6000] loss: 1.210

[3,  8000] loss: 1.199

[3, 10000] loss: 1.182

[3, 12000] loss: 1.189

[4,  2000] loss: 1.104

[4,  4000] loss: 1.114

[4,  6000] loss: 1.092

[4,  8000] loss: 1.093

[4, 10000] loss: 1.095

[4, 12000] loss: 1.114

[5,  2000] loss: 1.023

[5,  4000] loss: 1.021

[5,  6000] loss: 1.047

[5,  8000] loss: 1.026

[5, 10000] loss: 1.028

[5, 12000] loss: 1.044

Finished Training

GroundTruth:    cat  ship  ship plane

Predicted:    cat   car plane plane

Accuracy of the network on the 10000 test images: 61 %

Accuracy of plane : 50 %

Accuracy of   car : 74 %

Accuracy of  bird : 64 %

Accuracy of   cat : 48 %

Accuracy of  deer : 50 %

Accuracy of   dog : 44 %

Accuracy of  frog : 65 %

Accuracy of horse : 64 %

Accuracy of  ship : 74 %

Accuracy of truck : 72 %

cuda:0

How do we finish this? 

We get the outputs and measure the accuracy of each particular class, transfer the neural network to the GPU,  and predict whatever has the highest energy.

Data parallelism can massively speedup the set. Here is the final code rundown:

# -*- coding: utf-8 -*-

"""

Spyder Editor

This is a temporary script file.

"""

import torch

import torchvision

import torchvision.transforms as transforms

import matplotlib.pyplot as plt

import numpy as np

import torch.nn as nn

import torch.nn.functional as F

import torch.optim as optim

class Net(nn.Module):

    def __init__(self):

        super(Net, self).__init__()

        self.conv1 = nn.Conv2d(3, 6, 5)

        self.pool = nn.MaxPool2d(2, 2)

        self.conv2 = nn.Conv2d(6, 16, 5)

        self.fc1 = nn.Linear(16 * 5 * 5, 120)

        self.fc2 = nn.Linear(120, 84)

        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):

        x = self.pool(F.relu(self.conv1(x)))

        x = self.pool(F.relu(self.conv2(x)))

        x = x.view(-1, 16 * 5 * 5)

        x = F.relu(self.fc1(x))

        x = F.relu(self.fc2(x))

        x = self.fc3(x)

        return x

def imshow(img):

    img = img / 2 + 0.5     # unnormalize

    npimg = img.numpy()

    plt.imshow(np.transpose(npimg, (1, 2, 0)))

    plt.show()

    

transform = transforms.Compose(

    [transforms.ToTensor(),

     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,

                                        download=True, transform=transform)

trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,

                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,

                                       download=True, transform=transform)

testloader = torch.utils.data.DataLoader(testset, batch_size=4,

                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',

           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

if __name__ == '__main__':

    classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

    # get some random training images

    dataiter = iter(trainloader)

    images, labels = dataiter.next()

    # show images

    imshow(torchvision.utils.make_grid(images))

    # print labels

    print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

    net = Net()

    criterion = nn.CrossEntropyLoss()

    optimizer = optim.SGD(net.parameters(), lr = 0.001, momentum = 0.9)

    

    for epoch in range(5):

        running_loss = 0.0

        for i, data in enumerate(trainloader, 0):

        # get the inputs; data is a list of [inputs, labels]

            inputs, labels = data

        # zero the parameter gradients

            optimizer.zero_grad()

        # forward + backward + optimize

            outputs = net(inputs)

            loss = criterion(outputs, labels)

            loss.backward()

            optimizer.step()

        # print statistics

            running_loss += loss.item()

            if i % 2000 == 1999:    # print every 2000 mini-batches

                print('[%d, %5d] loss: %.3f' %

                (epoch + 1, i + 1, running_loss / 2000))

                running_loss = 0.0

    print('Finished Training')

    PATH = './cifar_net.pth' 

    torch.save(net.state_dict(), PATH)

    dataiter = iter(testloader)

    images, labels = dataiter.next()

    # print images

    imshow(torchvision.utils.make_grid(images))

    print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))

    net = Net()

    net.load_state_dict(torch.load(PATH))

    outputs = net(images)

    _, predicted = torch.max(outputs, 1)

    print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]

                              for j in range(4)))

    

    #see how the network performs in all classes

    correct = 0

    total = 0

    with torch.no_grad():

        for data in testloader:

            images, labels = data

            outputs = net(images)

            _, predicted = torch.max(outputs.data, 1)

            total += labels.size(0)

            correct += (predicted == labels).sum().item()

    

    print('Accuracy of the network on the 10000 test images: %d %%' % (

        100 * correct / total))

    

    #what classes performed well and what classes didn't

    class_correct = list(0. for i in range(10))

    class_total = list(0. for i in range(10))

    with torch.no_grad():

        for data in testloader:

            images, labels = data

            outputs = net(images)

            _, predicted = torch.max(outputs, 1)

            c = (predicted == labels).squeeze()

            for i in range(4):

                label = labels[i]

                class_correct[label] += c[i].item()

                class_total[label] += 1

    

    

    

    for i in range(10):

        print('Accuracy of %5s : %2d %%' % (

            classes[i], 100 * class_correct[i] / class_total[i]))

        

    #transfer everything to a GPU

    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# Assuming that we are on a CUDA machine, this should print a CUDA device:

    print(device)

    net.to(device)

    inputs, labels = data[0].to(device), data[1].to(device)