Visionâ€”Feature Extraction Topics Part I â€¢Fourier Transform

Pattern Recognition for VisionFall 2004

Vision—Feature Extraction

Topics

Part I•Fourier Transform •Windowed Fourier Transform•Wavelets

Part II•Principal Component Analysis•Independent Component Analysis


Vision—Feature Extraction I

2 ( )ˆ ( , ) ( , ) x y

y

j x yxf f x y e dxdyπ ω ωω ω

∞ ∞− +

−∞ −∞

= ∫ ∫Fourier Transform

x

yImage(B)(A)

Spectrum

xω

yω

Amplitude of (A) &Phase of (B)

Phase carries Information

Courtesy of Professors Tomaso Poggio and Sayan Mukherjee. Used with permission.



Template Matching' ' '( ) ( ) ( ) ( ) ( )c x f x g x x dx h x g x

∞

−∞

= − = −∫' ' ' ˆ ˆˆ( ) ( ) ( ) ( ) ( ) ( )c x f x h x x dx c f hω ω ω

∞

−∞

= − =∫

Result c

Template g

Image f



Windowed Fourier Transform

2( , ) ( ) ( ) j uf t f u g u t e duπωω∞

−

−∞

= −∫%

2inst( ) cos( ),f t t tπ ω= = ( )g t ( , )f tω%

( )f uu

( )g u t−

t T− t



Wavelet Transform

,1( )s t p

u tus s

ψ ψ − = ,( , ) ( ) ( )s tT s t f u u duψ

∞

−∞

= ∫

2

0.5, 10

1,0

2,15

( ) red

green blue

uu ueψψ

ψ

ψ

−

− −

−

=



Haar Wavelets (Matlab Toolbox)

Screenshot from Matlab Toolbox removed due to copyright reasons.


Vision—Feature Extraction II

Principal and Independent Component Analysis

x1

PCA

ICA

PCA

ICA

x2PCA:•Decorrelated•OrthogonalICA:•Statistically Independent

Image removed due to copyright considerations. See Figure 1 in: Baek, Kyungim, et. al. "PCA vs. ICA: A comparison on the FERET data set."International Conference of Computer Vision, Pattern Recognition, and ImageProcessing, in conjunction with the 6th JCIS. Durham, NC, March 8-14 2002, June 2001.


Pattern Recognition—Classifiers

Topics

Fisher Discriminant Analysis (FDA)•Support Vector Machines (SVM)



Fisher Discriminant Analysis

x2Maximize

21 22 2

1 2

( )m mσ σ

−+

x1



Linear Support Vector Machines

1( )

N

i i ii

f sign y bα=

= ⋅ −

∑x x xMaximize margin d

ix



Non-linear Support Vector Machines

x1

x2

Input Space

x*1

x*2

)(* xx Φ=

Feature Space

1( ) ( , ) , ( , ) ( ) ( )

N

i i i i ii

f sign y K b Kα=

= − = Φ Φ

∑x x x x x x x


Vision—Biological Object Recognition

Topics

•Visual Cortex•Hierarchical Processing•Scale Invariance •MAX Model for Object Recognition



The Visual System Simplified

Ventral stream:What?

Figure 1 in: Ungerleider, L., et. al. "A neural system for human visual working memory." Proceedings of the National Academy of Sciences 95 (February 1998): 883-890. Copyright 1998 National Academy of Sciences, U.S.A.



From Small and Simple to Big and Complex

Max’s stuffV1V4

ITC

modified from Ungerleider and Haxby, 1994

IT cells:Object specific

Hubel & Wiesel, 1959

Simple cells:Orientation of bars

Image removed due to copyright considerations.See: Hubel, and Weisel. "Receptive fields of single neuronsin the cat's striate cortex." Journal of Physiology 148 (1959): 574-951.

Image removed due to copyright considerations. See:Ungerleider, and Haxby. "'What' and 'where' in the human brain." Current Opinion in Neurobiology 4, no. 2 (1994): 157-165.



Hierarchical Processing: From Simple to Complex Features

Figure 1 in: Wersing, H., and E. Korner. "Learning Optimized Features for Hierarchical Models of Invariant Object Recognition." Neural Computation 15, no. 7 (2003): 1559-1558. MIT Press Journals. Courtesy of ________________MIT Press Journals. Used with permission.

http://mitpress.mit.edu/catalog/item/default.asp?ttype=4&tid=31&xid=13&xcid=1620


Applications—Morphable Models

Topics

•2D MMs of facical components•2D MMs for facial animation

•3D MMs of faces•Modifying faces in the MM space



2D Morphable Models for Facial Animation“Air” “Badge”

VideoDatabase

Visual SpeechProcessing

“Badge”

•Video realism•Machine learning

1I

2I 3I

4I

2C 3C

4C

Buildinga 2D MM

From Figure 1 in: Ezzat, Geiger, and Poggio. "Trainable Videorealistic Speech Animation."Proceedings of SIGGRAPH 2002, San Antonio, TX. Courtesy of authors. Used with permission.



2D Morphable Models for Facial Animation

From Figure 10 in: Ezzat, Geiger, and Poggio. "Trainable Videorealistic Speech Animation."Proceedings of SIGGRAPH 2002, San Antonio, TX. Courtesy of authors. Used with permission.



Generating 3D Face Models with MM

FaceImages

3D FaceModel

Morphing Space

Courtesy of Maxmillian Riesen-Huber. Used with permission.



Modifying Faces in the MM Space

NovelImages

Figures removed due to copyright considerations. Please see: Figures from Vetter, T., and V. Blanz. "A Morphable Model for the Synthesis

of 3D Faces." Proceedings of SIGGRAPH (1999).


Applications—Object Detection & Recognition

Topics

•Face detection & recognition•Pedestrian detection•Feature extraction •Classification


Applications—Object Detection & RecognitionObject Detection

Search for faces atdifferent resolutions and locations

Feature vector (x1, x2 ,…, xn)

Classifier

Off-line training

Face examples

Non-face examples

Feature Extraction

Pixel pattern

Classification Result

Photograph by MIT OCW.

Photograph by MIT OCW.



Object Recognition

Feature Extraction

Feature vector

Classifier

Face examples of different persons

A B

C D

Off-line training Identity of Person

Pixel pattern

Single image of a face

A



Face and Pedestrian Detection

Visionâ€”Feature Extraction Topics Part I â€¢Fourier Transform

Documents

Transcript of Visionâ€”Feature Extraction Topics Part I â€¢Fourier Transform