2D Image Recognition using SIFT

Image matching has applications in Object Recognition, and making panoramas, and 3D imaging, where we need to match features across images. 

SIFT matches images and it is called Scale Invariant Feature Transform. Most of the time, direct template matching doesn't work, especially if an image is partially visible. We try to find a good part of the object to solve this.

A feature has rich image content (color, intensity, gradient variations, etc.) It also has a well-defined representation for matching or comparing with other points. It should have a well defined position, and should be invariant to lighting changes, so we can match images taken under very different conditions.

 Edges are interesting because there is a strong intensity change among edges. However, the main problem with edges is that they are hard to localize. We cannot "localize" an edge-like feature.

"Blobs" have edges of strong gradients, and have a fixed location, as well as a well-defined size. There are clear intensity and edge variations within these blob windows. 

One simple idea is to take a small window and try to use image intensities to try to match across a region. This often doesn't work, due to changes in size, lighting, orientation, etc. Thus, we need to remove the source of variation. This means normalizing an image with size, angle, and lighting.

We are going to use these "blobs" for 2D recognition and image recognition.

We need to locate a blob, and determine its size and orientation, and develop a representation which normalizes for these factors. 

Blob in 2D is 2 consecutive edges in quick succession.

First we smoothen a function using a Gaussian filter until noise is filtered out. Check the edge detection notes for more information. 

Here is the Laplacian Function with 1D blobs. 

and this equals 

And for another blob, we get this:

And normalize the function by multiplying by sigma squared. A narrow filter has a shallower valley, whereas a wider filter has a deeper valley. The magnitude of a filter's deepness corresponds a lot to the blob size. Result achieves minima if Laplacian filter represents side of blob.

If we know the characteristic scale, as a result, we can find the blob size.

When the filter becomes bigger than blob A and blob B, then the values started going back up.

The size of a filter matches with the size of a blob.

For each sigma, we correlate f(x), and then we get a collection of these filtered functions, and then we look for the minimum value for all of these filtered functions, or maximum value at times.

For nLog, we want to take the difference of Laplacians. 

The laplacian can be approximated as a difference of Gaussians, and we can factorize a 2D Gaussian into 2 1D Gaussians, much faster to apply. 

To extract SIFT Interest points, we apply Gaussian filters of different sizes, we minus the filters, and find the extrema of every grid, and then we find interest point minima coordinates. Then we remove weak points by suppression, and then get SIFT interest points. 

Each small features correspond to a green dot that we find in a stack of images, and represent the collection of features as a collection of circles. 

We want to convolve the image using laplacian filters, look for local locations of the stack.

After this, we removes heights lower than a certain threshold. We define the sigmas using an imperative design choice. 

The idea of analyzing an image at multiple scales is called scale space analysis, for example, blurring the image with increasingly larger Gaussians and luring the resolution of the image. 

If s is a constant number and σ is initial scale σk is σ0 s^k. Let H1(k) and H2(k) be two arrays of data of length N. The smaller the distance metric, the more similar the features, the larger the distance metric, the better the correlation.

We can use an analysis to compare the size of 2 blobs. THe plots of the laplacian filters at different scales are different, and the location of the maximus has characteristic scales. σ1/σ2 is the ratio of blob sizes. The ratio of the scales are the ratio of the image sizes as well. You can scale on of the images with this ratio, and directly match features accross them.

 

The next step is to use a histogram of gradient directions and the distance, and then finally the direction that gets the maximum amount of votes is the orientation of a blob.

Suppose there is a feature which is the same across images. We can rotate no so that image principal orientations match, and get invariance into different orientations as well. 

One idea is to define a feature in terms of the gradients of image pixels around the image point. The directions of the gradients are invariant to absolute image brightness, so we use the gradients to define the SIFT feature representations. 

We first rotate the image region, so that the image points towards a vertical direction (the gradient), then look at pixels approximately within an area, then we create a histogram of gradient directions over all of the pixels, dividing the image window into 4 subwindows, and create a histogram for gradient directions for each window. We concatenate all of these histograms into a single vector of numbers, which is a SIFT descriptor, representing a SIFT feature location.

First we want to find a principal orientation, and rotate the feature such that the principle orientation goes into a common direction. We divide everything into 16 subwindows to get a new feature vector.

We compare sift features for both images and fine machine features for the images, even when the orientations are different. We can use feature maps to recognize objects, even if the objects are only partially visible. 

Using SIFT has many different applications. One is making panoramas, collections of images of a scene using different viewpoints. First, find the sift features, match them, and alight and combine the images to create a larger image.

We can also perform a SIFT for 3D objects of our scene, but it is very difficult. Therefore, SIFT is reliable for planar of only small changes in viewpoint. SIFT is finally one of the most robust vision algorithms. We can compute the principal direction for all and determine the differences in orientation.