1

Image Features

Hao Jiang

Sept. 2014

Image Matching

2

Image Mosaicing

3

How can This be Done Automatically?

4

5

Easy to match Hard to match

Image Local Structures

6

Step Ridge

Peak

Valley

Corner Junction

Image Local Structures

7

Step Ridge

Peak

Valley

Corner Junction

Line Structures:“Edge”

Point Structures:“Corners”

Regions

8

An Example

9

edgeRegion

corners

Edge Detection in Matlab

10

>> im = imread('flower.jpg');>> im = im2double(im);>> im = rgb2gray(im);>> ed = edge(im, 'canny', 0.15);

How to Find an Edge?

11

A 1D edge

12

f(x)

f’(x)

f’’(x)

Extend to 2D

13

ab

There is a direction in whichimage f(x,y) increases thefastest. The direction is calledthe gradient direction.

Gradient [df/dx df/dy]Magnitude: sqrt(fx^2 + fy^2)Direction: atan2(fy, fx)

Finite Difference

Approximating derivatives using finite difference.

Finite difference and convolution

14

Noise Reduction

15

0.01 noise

0.03 noise

Noise Reduction

16

Gaussian Filtering in Edge Detection

17

Gaussian Filtering in Edge Detection

18

h * (g * f) = (h * g) * f

Difference Kernel Gaussian Kernel

image

DifferenceGaussian Kernel

18

[1,-1]

Edge Detection in Images

Gaussian smoothed filtering in x and y directions: Ix, Iy Non-maximum suppression for |Ix|+|Iy|

Edge Tracing – double thresholding.

19

Edge Detection Using Matlab

Canny edge detector:

edge(image, ‘canny’, threshold) Sobel edge detector:

edge(image, ‘sobel’, threshold) Prewitt edge detector:

edge(image, ‘prewitt’, threshold)

20

21D. Martin, C. Fowlkes, D. Tal, J. Malik. "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics”, ICCV, 2001

Berkeley Segmentation DataSet [BSDS]

Corner Detection

Corner is a point feature that has large changing rate in all directions.

22

Step LineCorner

Flat region

Harris corner detector

C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988

The Basic Idea

We should easily recognize the point by looking through a small window

Shifting a window in any direction should give a large change in intensity

Harris Detector: Basic Idea

“flat” region:no change in all directions

“edge”:no change along the edge direction

“corner”:significant change in all directions

Find a Corner

26

Compute matrix H =

in each window. If the ratio (Ix2 * Iy2 – Ixy ^2 )R =------------------------ > T (Ix2 + Iy2 + eps)We have a corner

Ix2 IxyIxy Iy2=

Harris Detector: Workflow

Harris Detector: Workflow

Compute corner response R

Harris Detector: Workflow

Find points with large corner response: R>threshold

Harris Detector: Workflow

Take only the points of local maxima of R

Harris Detector: Workflow

Corner Detection Programming

32

Harris Detector: Some Properties Rotation invariance

Ellipse rotates but its shape (i.e. eigenvalues) remains the same

Corner response R is invariant to image rotation

Harris Detector: Some Properties Partial invariance to affine intensity change

Only derivatives are used => invariance to intensity shift I I + b

Intensity scale: I a I

R

x (image coordinate)

threshold

R

x (image coordinate)

Harris Detector: Some Properties

But: non-invariant to image scale!

All points will be classified as edges

Corner !

Scale Invariant Detection

Consider regions (e.g. circles) of different sizes around a point Regions of corresponding sizes will look the same in both

images

Scale Invariant Detection

The problem: how do we choose corresponding circles independently in each image?

Choose the scale of the “best” corner

Descriptors Invariant to Rotation

Find local orientation

Dominant direction of gradient

• Extract image patches relative to this orientation

Descriptor Vector

Orientation = blurred gradient Rotation Invariant Frame

Scale-space position (x, y, s) + orientation ()

Scale and Rotation Invariant Feature

SIFT (D. Lowe, UBC)

40

Stable Feature

41

Stable Feature

42

Local max/min point’s valuesare stable when the scale changes

SIFT

43

Filteringthe imageusing filters at differentscales.(for example using Gaussian filter)

Difference of Gaussian

44

SIFT Feature Points

45

(b) Shows the points at the local max/min of DOG scale space forthe image in (a).

Feature matching

?

Feature matching

• Exhaustive search for each feature in one image, look at all the other features

in the other image(s)

• Hashing compute a short descriptor from each feature vector, or hash

longer descriptors (randomly)

• Nearest neighbor techniques kd-trees and their variants

What about outliers?

?

Feature-space outlier rejection Let’s not match all features, but only these that have

“similar enough” matches? How can we do it?

SSD(patch1,patch2) < threshold How to set threshold?

Feature-space outlier rejection

A better way [Lowe, 1999]: 1-NN: SSD of the closest match 2-NN: SSD of the second-closest match Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN That is, is our best match so much better than the rest?

Feature-space outliner rejection

Can we now compute H from the blue points? No! Still too many outliers… What can we do?