Lecture #26 - Case Western Reserve...

EECS490: Digital Image Processing

Lecture #26

• Moments; invariant moments

• Eigenvector, principal component analysis

• Boundary coding

• Image primitives

• Image representation: trees, graphs

• Object recognition and classes

• Minimum distance classifiers

• Correlation

• Statistical pattern recognition; Bayes decision rule

• Neural network classifiers


Moments

mpq = x pyq f x, y( )dxdy++

The calculation of moments of for images andobjects in images is very similar to that used tocalculate statistical moments

mpq = xipyj

q f xi , yj( )j=0

L 1

i=0

L 1

For an LxL discrete image we can write


Moments

We can also define central moments in a similar manner

μpq = xi x( )pyj y( )

qf xi , yj( )

j=0

L 1

i=0

L 1

μ00 = m00 = f xi , yj( )j=0

L 1

i=0

L 1

m10 = xi f xi , yj( )j=0

L 1

i=0

L 1

m01 = yi f xi , yj( )j=0

L 1

i=0

L 1

x =m10

m00

y =m01

m00

Some simple moments and means


Moments

We can use these second moments to define the firstinvariant moment

μ20 = xi xi( )2f xi , yj( )

j=0

L 1

i=0

L 1

= m20 xm10

20 =μ20

μ002

Computing higher moments and means

μ02 = yj yj( )2f xi , yj( )

j=0

L 1

i=0

L 1

= m02 ym01

02 =μ02

μ002

1 = 20 + 02


Invariant Moments

1 = 20 + 02

2 = 20 02( )2+ 4 11

2

3 = 30 3 12( )2+ 21 3 03( )

2

7 = 3 21 03( ) 30 + 12( ) 30 + 12( )23 21 + 03( )

2

+ 3 12 30( ) 21 + 03( ) 3 30 + 12( )2

21 + 03( )2


Moments

Reduced 50% insize

Rotated 2˚

Mirrored

Rotated 45˚

Original


There are seven moments which are invariant totranslation, rotation, and scale.

Each row in this table should remain constant.The constancy shows the utility of the moment

Moments


Moments

© 2002 R. C. Gonzalez & R. E. Woods

Reduced50% insize

Rotated90˚

Translated

Rotated 45˚

Original

Mirrored


Representation and Description


Each row in this table should also remainconstant. The constancy shows the utility of themoment

This example is much better than the examplefrom the 2nd edition


x =

x1x2x3

=

R

G

B

x =

x1x2

xn

Eigenvectors and Eigenvalues

For RGB images (with 3 components)we can write each pixel as x

For registered multi-spectral images(with n components) we can write x as

n=6 components


For these n registered images we can compute themean vector (nx1) and covariance matrix (nxn) forthe set of all x

x =

x1x2

xn

mx = E x{ } =

E x1{ }E x2{ }

E xn{ }

Cx = E x mx( ) x mx( )T{ } =

E x1 m1( ) x1 m1( ){ } E x1 m1( ) x2 m2( ){ } E x1 m1( ) xn mn( ){ }E x2 m2( ) x1 m1( ){ } E x2 m2( ) x2 m2( ){ }

E xn mn( ) x1 m1( ){ } E xn mn( ) xn mn( ){ }



Transform the data by a Hotelling* transformation

y = A x mx( )

A =

e1T

e2T

enT


where the rows of A are the eigenvectors of thecovariance matrix Cx

* Also known as the Karhunen-Loeve transformation


The transformed variable y has the properties that

my = E y{ } = 0

Cy = ACxAT=

1 0 0

0 2

0 n

1 > 2 > > n


Cy is diagonal which indicates that the transformed y vectors areuncorrelated.

where

and


x = AT y + mx

A =

e1T

e2T

ekT


This transformation can be invertedto give back the original imagevectors x

where


The difference (rms error) between this approximation and theoriginal x is given by the sum of the eigenvalues correspondingto the removed eigenvectors.

B =

e1T

e2T

ekT

x̂ = BT y + mx

erms = jj= k+1

n


What if I replace A by B which has only the keigenvectors corresponding to the k largesteigenvalues?

We now have an approximation to x given by



The transformed vectors y are orthogonal and form aorthoginal basis set for describing the original set of images.

Each yi is an eigenvector of Cx and represents a “feature” ofthe original set of images. y1 corresponds to the mostsignificant feature since y1 corresponds to the largesteigenvalue 1. y2 corresponds to the next most significantfeature since y2 corresponds to the next largest eigenvalue 2.

x̂ = ai yii=1

k


Eigenvector Example


Six registered multi-spectral images: blue, green, red, nearinfrared, middle infrared, and far (thermal) infrared bands.


Eigenvector Example


Each pixel in this multi-spectal image can be described by a six-component pixel vector x

These are 384x239 pixels so there are 91776 pixel vectorsdescribing the complete image


Eigenvector Example


We can use a computer program to compute the mean and 6x6covariance matrix Cx for these 91776 pixels. We can thencalculate the eigenvectors e1 to e6 for Cx. For this example theeigenvalues are


Eigenvector Example


This is the mostsignificant imagewith the mostinformation ( 1)

This is the leastsignificant imagewith the leastinformation ( 1)

These are the six principal component images given by yi=eiT(x-mx)

This transformation rearranges the information presented in the multi-spectral image.

y1 y2 y3

y4 y5 y6


Eigenvector Example


Now reconstruct the six original images using only the two mostsignificant eigenvectors.

y = A x mx( )

Compute

B =e1T

e2T

Use only

x̂ = BTy1y2

+ mx

Invert


Eigenvector Example


Original images


Eigenvector Example


Differencesbetween theoriginalimages andthe imagesreconstructedusing only twoeigenvectors.


Eigenface Analysis


Original dataset of registeredfaces

http://www.geop.ubc.ca/CDSST/eigenfaces.html.

Corresponding 15 eigenfaces

Eigenvector vector expansions have beenfinding increasing application in such areasas face recognition.


Eigenvalues & Eigenvectors

The first eigenvector e1 corresponds to the axis oflargest variance, i.e., the principal axis.

Translating the objectto its centroid (mean)and aligning the principalaxes (variances) canoften be used tofacilitate objectanalysis and recognition.


Eigenvector Example #2


1. Representthe boundarypoints of anobject as a setof 2-D vectorsx and computeCx

2. Compute the eigenvectors of Cx

3. Compute the mean mx andtranslate the mean to the origin

4. Rotate the objectso that e1 correspondsto the horizontal axisand e2 to the verticalaxis.


Manual Eigenvector Example


This is asimpleboundaryanalysiswhich you canreplicate byhand.


Relational Descriptors


This is similarto the texturegrammarswhich wediscussedearlier.

These are the re-writing rules whichcan be used to generate abab… fromS

a,b are the elements shown above; S is thestarting symbol; A is a variable

SaAabSabaAabab

etc.




SaAabSabaAabab

The rules can berepeated to describelarger objects. Thenumbers correspond tothe rules which wereapplied.




Object boundaries can be coded using head-to-tail connected directed line segments.




Morecomplexrelationshipscan bedefined fordirected linesegments.


Object Recognition

This shows a clearly defined region for Irissetosa based upon petal length and width.



Object Recognition

Signatures can be used to code objects to berecognized.



Object Recognition

String descriptions can also be used to code objects.



Object Recognition

More complexstructures suchas trees areneeded todescribe theobjects in atypical image




Object Recognition



Minimum Distance Classifiers

The prototype for each class (out of W classes) is the meanvector of that class

mj =1

N j

x jx wj

j = 1,2,...,W

Compute “closeness” using Euclidian distance

Dj x( ) = x mj j = 1,2,...,W

Assign x to class j if Dj(x) is the smallest distance



Minimum Distance Classifiers

More mathematically the distance from a candidate pattern xto the pairs of mean vectors mi and mj can be written as

This describes a plane marking the classification boundarybetween classes i and j, i.e.,

If dij(x) >0 then assign x to class j otherwise assign to class i

dij x( ) = di x( ) dj x( ) = xT mi mj( )1

2mi mj( )

Tmi + mj( )

dij x( ) = 0



Object Recognition

d12 x( ) = 0

Decision classifier



Object Recognition

Check characters areread horizontally by asingle vertical sensorwhich travels from leftto right to generate adistinct waveform foreach character

The characters andsensors are optimized togive distinct responses ina simple x-y grid.



Object Recognition

Correlation can be used tofind similar patterns



Object Recognition



Statistical Pattern Recognition

If the class probabilities are not equal we must use a Bayesianclassifier and use the maximum to classify unknown patterns

The threshold for classifying patterns is where thepdf’s overlap

dj x( ) = p x | j( )P j( ) j = 1,2,...,W



Object Recognition

In moredimensions theclassifier forms aplane (orhyperplane) forGaussian patternclasses

dj x( ) = lnP j( )1

2ln Cj

1

2x mj( )

TCj

1 x mj( ) j = 1,...,W



Object Recognition

We can use the same registered multi-spectralimage data as input to a pattern classifier



Object Recognition

Prototypes

(3)Vegetationclassification

(1) Waterclassification

(2) Urban dev.classification

Errors usinghalf of theprototypes totrain, theother toclassify



Object Recognition

Bayes classification of pixels inthe training regions 1,2,3



Object Recognition

The perceptron is an early neuralnetwork architecture forlearning decision classifiers



Object Recognition



Object Recognition

Neural networks can be used toimplement (and learn)geometrically complex decisionfunctions.



Object Recognition

Multi-layer networks can be used toapproximate (learn) more complexdecision functions.



Shape Number Classification

Objects can besimilar up todifferent ordersof shape number.This can be usedto classifyobjects.



Shape Number Classification

Lecture #26 - Case Western Reserve...

Documents

Transcript of Lecture #26 - Case Western Reserve...