1

Vijayakumar Bhagavatula

Vijayakumar Bhagavatula

Title Goes HereCorrelation Pattern Recognition

December 10, 2003

2

Vijayakumar Bhagavatula

Outline

! Correlation pattern recognition! Pattern recognition examples! Book! Demos

3

Vijayakumar Bhagavatula

18-794 Pattern Recognition Theory

! Speech recognition! Optical character recognition (OCR)! Fingerprint recognition! Face recognition! Automatic target recognition! Biomedical image analysis

Objective: To provide the background and techniques needed for pattern classification

For advanced UG and starting graduate students

Example Applications:

4

Vijayakumar Bhagavatula

Pattern Recognition Methods

Feature ExtractionInput Classifier Class

! Statistical methods (e.g., Bayes decision theory)! Machine learning methods! Artificial neural networks! Correlation filters

Most approaches are based in image domain whereas significant advantages exist in spatial frequency domain.

6

Vijayakumar Bhagavatula

Example Feature-based Matching

Minutiae

Minutiae Extraction

Input Image

Minutiae

Orientation Field

Region of Interest

Thinned Ridges

Extracted RidgesRidge Ending

Ridge Bifurcation

Orientation Estimation

Fingerprint Locator

Ridge Extraction

Thinning f

Minutiae Extraction

! Features based on intuition & experience

! Significant preprocessing needed

! Sensitive to occlusions

7

Vijayakumar Bhagavatula

Correlation Pattern Recognition

! Normalized correlation between r(x) and s(x) between -1 and +1; reaches +1 if and only if r(x) = s(x).

! Problem: Reference patterns rarely have same appearance! Solution: Find the pattern that is consistent (i.e., yields large

correlation) among the observed variations.

( ) ( )

( ) ( )2 21 1

r x s x dx

r x dx s x dx− ≤ ≤∫

∫ ∫

! r(x) test pattern! s(x) reference pattern

8

Vijayakumar Bhagavatula

Pattern Variability

! Facial appearance may change due to illumination! Fingerprint image may change due to plastic deformation

9

Vijayakumar Bhagavatula

Pattern Locations

! Desired Pattern can be anywhere in the input scene.! Multiple patterns can appear in the scene.! Pattern recognition methods must be shift-invariant.

11

Vijayakumar Bhagavatula

Cross-Correlation Function

! Determine the cross-correlation between the reference and test images for all possible shifts

!When the target scene matches the reference image exactly, output is the autocorrelation of the reference image.

! If the input r(x) contains a shifted version s(x-x0) of the reference signal, the correlator will exhibit a peak at x=x0.

! If the input does not contain the reference signal s(x), the correlator output will be low

! If the input contains multiple replicas of the reference signal, resulting cross-correlation contains multiple peaks at locations corresponding to input positions.

( ) ( ) ( )c r x s x dxτ τ= −∫

12

Vijayakumar Bhagavatula

Cross-Correlation Via Fourier Transforms

InputScene

FT

CorrelationFilter

IFTCorrelationOutput

ReferenceIm age s(x)

FilterDesign

r(x)

R(f)

H (f)

c(τ)

! Fourier transforms can be done digitally or optically

13

Vijayakumar Bhagavatula

ToInput SLM

FourierLens

FourierLens

Correlationpeaks for objects

ToFilter SLM

CCD Detector

Laser Beam

FourierTransform

InverseFourierTransform

Optical Correlator

SLM: Spatial Light ModulatorCCD: Charge-Coupled Detector

14

Vijayakumar Bhagavatula

Correlation Filters

M atchNo M atch

DecisionTest Image

IFFT Analyze

Correlation output

FFT

Correlation Filter

Filter Design . . .Training Images

TrainingRecognition

15

Vijayakumar Bhagavatula

Peak to Sidelobe Ratio (PSR)

σmeanPeak

PSR−

=

1. Locate peak1. Locate peak

2. M ask a sm all 2. M ask a sm all pixel regionpixel region

3. Com pute the m ean and 3. Com pute the m ean and σσ in a in a bigger region centered at the peakbigger region centered at the peak

! PSR invariant to constant illumination changes

! Match declared when PSR is large, i.e., peak must not only be large, but sidelobes must be small.

16

Vijayakumar Bhagavatula

Train on 3, 7, 16, Train on 3, 7, 16, --> Test on 10.> Test on 10.

18

Vijayakumar Bhagavatula

Using sam e Filter trained before,

Perform cross-correlation on cropped-face shown on left.

19

Vijayakumar Bhagavatula

••CO RRELATIO N FILTERS ARE SHIFT-INVARIANT

•Correlation output is shifted down by the sam e am ount of the shifted face im age, PSR rem ains SAM E!

20

Vijayakumar Bhagavatula

•Using SO M EO NE ELSE’S Filter,… . Perform cross-correlation on cropped-face shown on left.

•As expected very low PSR.

21

Vijayakumar Bhagavatula

Automatic Target Recognition Example

22

Vijayakumar Bhagavatula

Correlation Plane Contour M ap Correlation Plane Contour M ap

Correlation Plane SurfaceCorrelation Plane Surface

M 1A1 in the open M 1A1 near tree line

SAIP ATR SDF Correlation Perform ance for Extended Operating

Conditions

Courtesy: Northrop Grum m an

Adjacent trees cause some correlation noise

24

Vijayakumar Bhagavatula

Biometric Verification Examples

25

Vijayakumar Bhagavatula

Facial Expression Database

! Facial Expression Database (AMP Lab, CMU)! 13 People! 75 images per person! Varying Expressions! 64x64 pixels! Constant illumination

! 1 filter per person made from 3 training images

26

Vijayakumar Bhagavatula

PSRs for the Filter Trained on 3 Images

Response to Training Images Response to

Faces Images from Person A

M ARGIN OF SEPARATION

Response to 75 face images of the other 12 people=900 PSRs

PSR

28

Vijayakumar Bhagavatula

PIE Database Illumination Variations

! Simulations using 65 people from the Pose, Illumination and Expression (PIE) Database.

! Each person (with and without background lighting) has 21/22 face images respectively at frontal view.

29

Vijayakumar Bhagavatula

49 Faces from PIE Database illustrating the variations in illum ination

30

Vijayakumar Bhagavatula

Training Image selection

! We used three face images to synthesize a correlation filter ! The three selected training images consisted of 3 extreme

cases (dark left half face, normal face illumination, dark righthalf face).

n = 3 n = 7 n = 16

33

Vijayakumar Bhagavatula

Reject Reject

AuthenticateAuthenticateThresholdThreshold

EER using Filter with Background illumination

36

Vijayakumar Bhagavatula

Iris Verification

! High-quality iris images yield low error rates

! Correlation filters yield zero verification errors for the 9 iris images

! Challenge is to acquire high-quality iris images

Source: National Geographic Magazine

Source: Dr. J. Daugman’s web site

37

Vijayakumar Bhagavatula

Features of Correlation Filters

! Shift-invariant; no need for centering the test image! Graceful degradation! Can handle multiple appearances of the reference image in

the test image! Closed-form solutions based on well-defined metrics

48

Vijayakumar Bhagavatula

Motivation for the Book

! Most pattern recognition researchers are not able to take advantage of the power of correlation filters because of the diverse background needed! Signals and systems

! Probability theory and random variables

! Linear algebra! Optical processing

! Digital signal processing

! Detection and estimation theory

! Goal of the book: To provide the background and techniques for correlation pattern recognition and illustrate with applications.

49

Vijayakumar Bhagavatula

Book Chapters

! Introduction! Mathematical background ! Signals and systems! Detection theory! Basic correlation filters! Advanced correlation filters! Optics basics! Optical correlators! Application examples

50

Vijayakumar Bhagavatula

Book Status

! Co-authors! Dr. AbhijitM ahalanobis, Lockheed M artin

! Dr. Richard Juday, NASA Johnson Space Center (Retired)

! All nine chapters written! References and final editing being done! To be published by Cambridge University Press! Should come out in late 2004