Neural Networks are used for all kinds of computer vision tasks, from low-level image processing to high level inference.
They also help to detect faces, and recognize handwritten tasks, or recognize chairs. Neural networks are modelled on the behaviour of the human brain. A neural network architecture is described below.
There is an input layer with some hidden layers.
There is input, output, and hidden layers, and weights between the networks.
The basic building block of a neural network is a neuron, which is known as a function that takes several inputs. A neuron that takes discrete outputs is called a perceptron.
f is a function that takes an input and returns either a 0 or a 1 depending on the sum of the outputs.
The perceptron would subsequently look like a Step Function.
The most distinct activation function is simply to ReLU activation function.
Activation functions can try to be linear as well. What would a neural network with a linear function do? Suppose there is a neural network where each neuron has a linear activation function. The thing is the entire network would just be a linear function and the output is just a linear combination of each inputs, which does not do anything beneficial to the user of the network. These then render the intermediate layers useless.
We look at non-linear functions as a result. The next thing I want to talk about is a rectified linear unit, which is 0 for a negative number and a line for the positive numbers.
If x > 0 reLU(x) = x, otherwise the reLU function is equivalent to 0.
Suppose we want to have a neural network. There will be one input for each pixel in the image, and there is one output for each of the ten digits.
The following page will demonstrate the neural network:
the value with the highest (a) will basically be the output value we choose. The weights and biases together are called the parameters of the network.
All entries are 0 except when there is a correct layer, which is equivalent to 1.
We train the network with images with known labels. For each images, we know its ground truth label, AKA what the correct digit should be. We an subsequently train an network so it outputs its desired results.
I first initialize random weights and biases of the network with random values. Then I compute the network activation value for each image.
Now the next thing I do is compute the cost/difference for the entire training data, compute the difference between the true output and the desired output. We can call this the COST of the training image.
I need to find the difference between the desired activation and the actual activation. The desired activation is ideally binary.
The last step is to update all the weights and biases using gradient descent.
Backpropagation makes the process feasible.
The steps are as follows summarized.
1. Initialize the weights and the biases of the network with random values.
2. Compute network activation with each training image
3. Compute Cost for training data
4. Update weights + biases with Gradient Descent
5. Repeat 2-4 until convergence.
Digit recognition is still important when it comes to a new network to train/test.
Most modern datasets are on a MUCH bigger scale. One set is called imageNet with 1,000 different classifications and approximately 1,000,000 images.
There are 42,000 neurons in ImageNet, but because of the weight fact, we have more than 40 million connections.
To help the neurons run faster, you can drop some neurons, or reduce the number of connections. The idea that people came up with was based on these intuitions, but done in a much more formalized way.
Lines, gradients, corners look similar regardless of their location in the image. Therefore, we call them stationary features.
Parameters are reduced when more neurons are assigned certain weights.
Low and mid-level features are local, and stationary. We don't need to see a whole image to see aline or corner, and a line looks like a line, anywhere in an image.
Subsequent neuron layers extract to higher level features. Initial neurons do not have to see entire file for classification. This discovery accelerated the development of neural networks.
There are similar parameters accross different locations. A small window is slid over the image, and a weighted sum is computed over each image pixel. We did this in a previous lecture, but now to apply to neural networks.
A gaussian convolution filter is used and a bias is added in order to get a result and a feature map.
There are channels using different filters to have different filters for the image.
The idea now is to use filters as the inputs. This would drastically reduce the number of images necessary to convolve. The following illustration demonstrates a convolutional neural network.
One example of pooling is picking the maximum value of some window.
Convolution and pooling interchange from step to subsequent step, and it goes like follows:
Convolution --> Pooling --> convolution 00> Fully Connected --> Output predictions (dog, boat, cat, bird, etc).
First layers are simple, the subsequent layers get way more complicated.
In 2012, a Neural network came along and saw almost a 10 point improvement.
For segmentation, we want to return a label for every pixel, such that the output size is the same as the input size.
One idea is to design a netwrok with only convolutional layyers, such that the sixe of the features remains the same. This does seem reasonable in theory, but the convolution multiple times can be very expensive and have multiple parameters.
Another idea is to design the network as a bunch of convolutional layers, with downsampling and upsampling accross the network. Making the network smaller, and then subsequently larger. The recommended framework for Deep Learning is PyTorch.
If you want to do something like image segmentation, we can do this with a neural network if we have sufficient label data.
Comments
Post a Comment