Post on 21-Dec-2015
1MSU CSE 803 Fall 2014
Vectors [and more on masks]
Vector space theory applies directly to several image
processing/representation problems
2MSU CSE 803 Fall 2014
Image as a sum of “basic images”
What if every person’s portrait photo could be expressed as a sum of 20 special images? We would only need 20 numbers to model any photo sparse rep on our Smart card.
3MSU CSE 803 Fall 2014
Efaces
100 x 100 images of faces are approximated by a subspace of only 4 100 x 100 “images”, the mean image plus a linear combination of the 3 most important “eigenimages”
4MSU CSE 803 Fall 2014
The image as an expansion
5MSU CSE 803 Fall 2014
Different bases, different properties revealed
6MSU CSE 803 Fall 2014
Fundamental expansion
7MSU CSE 803 Fall 2014
Basis gives structural parts
8MSU CSE 803 Fall 2014
Vector space review, part 1
9MSU CSE 803 Fall 2014
Vector space review, Part 2
2
10MSU CSE 803 Fall 2014
A space of images in a vector space
M x N image of real intensity values has dimension D = M x N
Can concatenate all M rows to interpret an image as a D dimensional 1D vector
The vector space properties applyThe 2D structure of the image is
NOT lost
11MSU CSE 803 Fall 2014
Orthonormal basis vectors help
12MSU CSE 803 Fall 2014
Represent S = [10, 15, 20]
13MSU CSE 803 Fall 2014
Projection of vector U onto V
14MSU CSE 803 Fall 2014
Normalized dot product
Can now think about the angle between two signals, two faces, two text documents, …
15MSU CSE 803 Fall 2014
Every 2x2 neighborhood has some constant, some edge, and some line component
Confirm that basis vectors are orthonormal
16MSU CSE 803 Fall 2014
Roberts basis cont.
If a neighborhood N has large dot product with a basis vector (image), then N is similar to that basis image.
17MSU CSE 803 Fall 2014
Standard 3x3 image basis
Structureless and relatively useless!
18MSU CSE 803 Fall 2014
Frie-Chen basis
Confirm that bases vectors are orthonormal
19MSU CSE 803 Fall 2014
Structure from Frie-Chen expansion
Expand N using Frie-Chen basis
20MSU CSE 803 Fall 2014
Sinusoids provide a good basis
21MSU CSE 803 Fall 2014
Sinusoids also model well in images
22MSU CSE 803 Fall 2014
Operations using the Fourier basis
23MSU CSE 803 Fall 2014
A few properties of 1D sinusoids
They are orthogonal
Are they orthonormal?
24MSU CSE 803 Fall 2014
F(x,y) as a sum of sinusoids
26MSU CSE 803 Fall 2014
Continuous 2D Fourier Transform
To compute F(u,v) we do a dot product of our image f(x,y) with a specific sinusoid with frequencies u and v
27MSU CSE 803 Fall 2014
Power spectrum from FT
28MSU CSE 803 Fall 2014
Examples from images
Done with HIPS in 1997
29MSU CSE 803 Fall 2014
Descriptions of former spectra
30MSU CSE 803 Fall 2014
Discrete Fourier Transform
Do N x N dot products and determine where the energy is.
High energy in parameters u and v means original image has similarity to those sinusoids.
31MSU CSE 803 Fall 2014
Bandpass filtering
32MSU CSE 803 Fall 2014
Convolution of two functions in the spatial domain is equivalent to pointwise multiplication in the frequency domain
33MSU CSE 803 Fall 2014
LOG or DOG filter
Laplacian of GaussianApprox
Difference of Gaussians
34MSU CSE 803 Fall 2014
LOG filter properties
35MSU CSE 803 Fall 2014
Mathematical model
36MSU CSE 803 Fall 2014
1D model; rotate to create 2D model
37MSU CSE 803 Fall 2014
1D Gaussian and 1st derivative
38MSU CSE 803 Fall 2014
2nd derivative; then all 3 curves
39MSU CSE 803 Fall 2014
Laplacian of Gaussian as 3x3
40MSU CSE 803 Fall 2014
G(x,y): Mexican hat filter
41MSU CSE 803 Fall 2014
Convolving LOG with region boundary creates a zero-crossing
Mask h(x,y)
Input f(x,y) Output f(x,y) * h(x,y)
42MSU CSE 803 Fall 2014
43MSU CSE 803 Fall 2014
LOG relates to animal vision
44MSU CSE 803 Fall 2014
1D EX.
Artificial Neural Network (ANN) for computing
g(x) = f(x) * h(x)
level 1 cells feed 3 level 2 cells
level 2 cells integrate 3 level 1 input cells using weights [-1,2,-1]
45MSU CSE 803 Fall 2014
Experience the Mach band effect
46MSU CSE 803 Fall 2014
Simple model of a neuron
51MSU CSE 803 Fall 2014
Canny edge detector uses LOG filter
53MSU CSE 803 Fall 2014
Summary of LOG filter
Convenient filter shapeBoundaries detected as 0-
crossingsPsychophysical evidence that
animal visual systems might work this way (your testimony)
Physiological evidence that real NNs work as the ANNs