Image Analysis - Lecture 2 › matematiklth › vision › bildanalys › nu › f02.pdf · Review...

Review of Linear algebraFourier transform

Image Analysis - Lecture 2

Fredrik Kahl

3 September 2010

Fredrik Kahl Image Analysis - Lecture 2


Lecture 2

Contents◮ Linear Algebra

◮ Linear spaces, bases◮ Scalar product, orthogonality, ON-bases◮ Sub-spaces, projections

◮ The Fourier transform◮ Definition, examples, 1D and 2D◮ Discrete Fourier Transform, Fast Fourier Transform◮ Application to Images

◮ Master thesis proposal of the day



Linear space, basisScalar productOrthogonal projection

Linear algebra: Linear spaces

The following linear spaces are well-known:

◮ Rn : all n × 1-matrices, x =

x1...

xn

, xi ∈ R

◮ Cn : all n × 1-matrices, x =

x1...

xn

, xi ∈ C




Basis

Definitione1, . . . en ∈ Rn is a basis in Rn if

◮ they are linearly independent◮ they span Rn.

Example (3D space)e1, e2, e3 ∈ R3 is a basis in R3 if they are not located in thesame plane.




Canonical Basis

Example (canonical basis)

e1 =

10...0

, e2 =

01...0

, . . . en =

00...1

is called the canonical basis in Rn.

x =

x1...

xn

= x1e1 . . . + xnen .




Coordinates

Let e1, e2, . . . , en be a basis. Then for every x there is a uniqueset of scalars ξi such that

x =

n∑

i=1

ξiei .

These scalars are called the coordinates for x in the basise1, e2, . . . , en.




Image matrix

An M × N image, f , is described by the matrix

f =

f (0, 0) . . . f (0, N − 1)...

. . ....

f (M − 1, 0) . . . f (M − 1, N − 1)

, f (i , j) ∈ C

f (i , ·)=i :th row, f (·, j)=j :th column.




Row-stacking

Row-stacking:

f =

f T (0, ·)f T (1, ·)

...f T (M − 1, ·)

∈ RMN(CMN)

Properties:f + g = f + g

λf = λf , λ ∈ R

A linear space!




Canonical basis

χ(i , j) =

0 . . . 0 . . . 0... 1

...0 . . . 0 . . . 0

,

with the 1 at position (i , j).Using this canonical basis we can write

f =∑

i ,j

f (i , j)χ(i , j) .

Idea for image transform:Choose another basis that is more suitable in some sense.Image matrices can thus be seen as vectors in a linear space.




Scalar product

DefinitionLet A be a (complex) matrix. Introduce

A∗ = (A)T .

DefinitionLet x and y be two vectors in Rn (Cn). The scalar product of xand y is defined as

x · y =∑

xiyi = x∗y .




Orthogonality

DefinitionThe scalar product of two matrices (images) is defined as

f · g =

M−1∑

i=0

N−1∑

j=0

f (i , j)g(i , j) .

x , y ∈ R(C) are orthogonal if x · y = 0. This is often written

x⊥y ⇔ x · y = 0 .

The length or the norm of x is defined as

||x || =√∑

|xi |2 = (x∗x)1/2 .




Coordinates:

e1

e2

x

ξ1

ξ2

Norm:

x = (x1, x2)

||x ||

x1

x2




Orthogonal basis (ON-basis)

Definition{e1, . . . , en} is an orthonormal (ON-) basis in Rn (Cn) if

ei · ej =

{0 i 6= j

1 i = j




Coordinates in ON-basis

TheoremAssume that {e1, . . . , en} is orthonormal (ON) basis and

x =n∑

i=1

ξiei .

Then

ξi = ei · x = e∗

i x , ||x ||2 =

n∑

i=1

|ξi |2




Illustration

x

e1

e2

e1 · x

e2 · x




Orthogonal projection

DefinitionLet {a1, . . . , ak} ∈ Rn, k ≤ n, span a linear subspace, π, in Rn,i.e.:

π = {w |w =k∑

i=1

xiai , xi ∈ R} .

The orthogonal projection of u ∈ Rn on π is a linear mappingP, such that uπ = Pu and defined by

minw∈π

||u − w || = ||u − uπ|| .




Orthogonal projection (ctd.)

The orthogonal projection is characterized by

1. uπ ∈ π

2. u − uπ⊥w for every w ∈ π

π

uu − uπ

uπ




Example: Matrix-basis

What is the orthogonal projection of f

f =

1 2 34 5 67 7 7

onto the space spanned by (e1, e2, e3)

e1 =13

1 1 11 1 11 1 1

, e2 =

1√6

1 1 10 0 0−1 −1 −1

, e3 =

1√6

1 0 −11 0 −11 0 −1




Example: Matrix-basis (ctd.)

Since (e1, e2, e3) is orthonormal the coordinates arex1 = f · e1 = 14, x2 = f · e2 = −15/

√6, x3 = f · e3 = −4/

√6.

The orthogonal projection is thenf = 14e1 − 15/

√6e2 − 4/

√6e3

f =

1 2 34 5 67 7 7

, f =

1.5 216 25

64 42

3 513

6.5 716 75

6

,




Example: ’Face’-basis

What is the orthogonal projection of f

onto the space spanned by (e1, e2, e3)




Example: ’Face’-basis (ctd.)

Since (e1, e2, e3) is orthonormal, the coordinates arex1 = f · e1 = −2457, x2 = f · e2 = 303, x3 = f · e3 = −603.The orthogonal projection is then f = −2457e1 + 303e2 −603e3




Uniqueness of the projection

Let a ∈ π and b ∈ π be two solutions to the minimisationproblem. Set

f (t) = ‖u − ta − (1 − t)b‖2 = . . .

= ‖u − b‖2 + t2‖a − b‖2 − 2t(a − b) · (u − b), t ∈ R .

This is a second degree polynomial with minimum in t = 0 andt = 1 ⇒ f (t) is a constant function and thus ⇒ a = b.




Characterization of the projection

Let f (t) = ‖u − uπ + ta‖2, where a ∈ π. It follows thatf ′(0) = 2(u − uπ) · a = 0, i.e. (u − uπ) ⊥ a.Conversely: Assume w ∈ π. The property that (u − uπ) ⊥ a, forevery a ∈ π gives that

‖u − w‖2 = ‖u − uπ + uπ − w‖2 =

‖u − uπ‖2 + ‖uπ − w‖2 ≥ ‖u − uπ‖2,

i.e. uπ solves the minimization problem.




An important result

Let A = [a1 . . . ak ] be a n × k matrix and

π = {w |w = Ax , xi ∈ Rn}

LemmaIf {a1, . . . , ak} are linearly independent Rn then A∗A isinvertible.

Proof: Do it on your own. (Use SVD if you are familiar with it.)




Projection onto the subspace spanned by A

Theoremif the columns of A are linearly independent, then the projectionof u on π is given by

uπ = x1a1 + . . . + xkak , x = (A∗A)−1A∗u .

Proof: Use the characterization of the projection (above).

a∗

i (u − uπ) = 0 ⇒

A∗(u − Ax) = 0 ⇒A∗u = A∗Ax ⇒ x = (A∗A)−1A∗u




The pseudo-inverse

DefinitionA+ = (A∗A)−1A∗ is called the pseudo-inverse of A.

Observe that if A is quadratic and invertible then A+ = A−1.

TheoremIf {a1, . . . , ak} are orthonormal, then the projection of u on π isgiven by

uπ = y1a1 + . . . + yk ak , yi = a∗

i u .

Proof: This follows from A∗A = I.




Illustration

uπ

u

y1a1

π

y2a2



Continuous Fourier transformDiscrete Fourier Transform (DFT,FFT)Two-dimensional Fourier Transform

Fourier transform

DefinitionLet f be a function from R to R. The Fourier transformen of f isdefined as

(F f )(u) = F (u) =

∫ +∞

−∞

e−i2πxuf (x)dx .

TheoremUnder the right assumptions on f , the following inversionformula

f (x) =

∫ +∞

−∞

ei2πuxF (u)du

holds.




Examples

Example

δ(x) 7→ 1(u)

rect(x) 7→ 2sin(2πu)

2πu= 2 sinc(2πu)




Illustrations

x

x

u

u−1 1

1

1

F

F2




Properties

c1f1(x) + c2f2(x) 7→ c1F1(u) + c2F2(u) (linearity)

f (λx) 7→ 1|λ|F (

uλ

) (scaling)

f (x − a) 7→ e−i2πuaF (u) (translation)

e−i2πxaf (x) 7→ F (u + a) (modulation)

f (x) 7→ F (−u) (conjugation)

dfdx

7→ 2πiuF (u) (differentiation I)

−2πixf (x) 7→ dFdu

(differentiation II)

Example: δ(x − 1) 7→ e−i2πu




The discrete Fourier transform (DFT)

Sample f (x) and F (u).

0∆x

N − 1 0x u

N − 1∆u

F (n∆u) ∼N−1∑

k=0

e−i2πk∆xn∆u f (k∆x )∆x ,

f (k∆x ) ∼N−1∑

n=0

ei2πk∆x n∆u F (n∆u)∆u.




The discrete Fourier transform (DFT) (ctd.)

This works particularly well if ∆x∆u = 1N :

1∆x

F (n∆u) ∼N−1∑

k=0

e−i2πkn/N f (k∆x ),

f (k∆x ) ∼ 1N

N−1∑

n=0

ei2πkn/N 1∆x

F (n∆u).

F and f are extended to periodic functions with periods N∆u

and N∆x respectively.




Definition of the discrete Fourier transform

Let the vector(f (0), f (1), . . . , f (N − 1)) .

represent the discretized version of f (x).

DefinitionThe discrete Fourier Transformen (DFT) of f is

F (u) =

N−1∑

k=0

f (k)ωkuN , u = 0, . . . , N − 1 ,

where ωN = e−i2π/N .

Represent the sequence F (u) with the vector

(F (0), F (1), . . . , F (N − 1)) .




Important assumption

All sequences are assumed to be period with period N.

01 N − 123




Properties of DFT

There are similar formulas for the discrete Fourier Transform(as compared to that of the continuous Fourier Transform), e.g.

(f (−k0), f (1 − k0), . . . , f (N − 1 − k0)) 7→

7→N−1∑

k=0

f (k − k0)ωku = [l = k − k0]

=

N−1−k0∑

l=−k0

f (l)ω(l+k0)u =

= ωk0uN−1−k0∑

l=−k0

f (l)ωlu = [f periodic] = ωk0uN−1∑

l=0

f (l)ωlu =

= ωk0uF (u) = e−i2πk0/NF (u)




DFT in matrix form

Let

f =

f (0)...

f (N − 1)

, F =

F (0)...

F (N − 1)

.

DefinitionThe Fourier Matrix FN is given by

FN =

1 1 1 . . . 11 ωN ω2

N . . . ωN−1N

1 ω2N ω4

N . . . ω2(N−1)N

......

.... . .

...

1 ωN−1N ω

2(N−1)N . . . ω

(N−1)(N−1)N

.




Computation of DFT

Theorem

F = FN f

Proof: Use the definition of DFT and of the Fourier MatrixConsequence: DFT can be computed using matrixmultiplication.




Properties of the Fourier matrix

Lemma

FNFN = NI ⇐⇒ F−1N =

1NFN

Proof: Multiply FN with FN and use

ωNωN = 1,N−1∑

j=0

(ωpN)j =

1 − ωNpN

1 − ωN= 0 .




The inversion formula

This lemma gives us the following inversion formula

Theorem

f =1NFF ⇐⇒ f (k) =

1N

N−1∑

u=0

F (u)ω−kuN , k = 0, . . . , N − 1

Proof:

F = F f ⇒ 1NFF =

1NFF f = If = f .




Example

ExampleN = 2, ω = −1:

F2 =

(1 11 −1

).

f =

(13

)⇒ F =

(4−2

).




Example

ExampleN = 4, ω = −i :

F4 =

1 1 1 11 −i −1 i1 −1 1 −11 i −1 −i

.




Factorization of the Fourier matrix

N = 22 = 4, ω = ω4, τ = ω2

F4 =

1 1 1 11 ω ω2 ω3

1 ω2 ω4 ω6

1 ω3 ω6 ω9

=

=

1 1 1 11 ω2 ω ω3

1 ω4 ω2 ω6

1 ω6 ω3 ω9

1 0 0 00 0 1 00 1 0 00 0 0 1

=

=

(1 11 τ

) (1 1ω ωτ

)

(1 11 τ

) (−1 −1−ω −ωτ

)

P4




Factorization of the Fourier matrix (ctd.)

P4 denotes a 4 × 4 permutation matrix (a matrix with zeros andones, where each row and each column only contains one one).

F4 =

F2

(1 00 ω

)F2

F2 −(

1 00 ω

)F2

P4 =

(I D2

I −D2

)(F2 00 F2

)P4

whereD2 = diag(1, ω) .




The Fast Fourier Transform (FFT)

TheoremThe Fourier Matrix can be factorized as

F2N =

(I DN

I −DN

)(FN 00 FN

)P2N ,

whereDN = diag(1, ω2N , ω2

2N , . . . , ωN−12N ) .

and P2N is a permutation matrix of order 2N × 2N that maps

(x(0), x(1), . . . , x(2N − 1)) −→(x(0), x(2), . . . , x(2N − 2), x(1), x(3), . . . , x(2N − 1)) .




The Fast Fourier Transform (FFT) (ctd.)

CorollaryCalculation of FN f thus involves two calculations of FN/2f ,which involves 4 calculations of FN/4f , etc.

This algorithm is called the Fast Fourier Transform .




Calculational complexity

Let µn be the number of multiplications needed for calculatingDFT of order 2n. Factorization gives

µn = 2µn−1 + 2n−1 .

A solution to this recursion formula is

µn =n2n

2=

N log2 N2

om N = 2n .

ExampleN = 1024 = 210

◮ FFT gives µ ∼ 104 multiplications.◮ DFT gives N2 ∼ 106 multiplications.




Two-dimensional Fourier Transform

DefinitionLet f (x , y) be a function from R2 to R. The Fourier transform off is defined as

F f (u, v) = F (u, v) =

∫ +∞

−∞

e−i2π(ux+vy)f (x , y)dxdy .

This can be written (using u = (u, v), x = (x , y)):

F (u) =

∫ +∞

−∞

e−i2πu·xf (x)dx .




The inversion formula in 2D

TheoremUnder certain conditions on f , the following inversion formula

f (x , y) =

∫ +∞

−∞

ei2π(ux+vy)F (u, v)dudv

holds.




Properties of the 2D Fourier transformation

Properties: (in addition to those for the 1-D Fourier Transform)

f1(x)f2(y) 7→ F1(u)F2(v) (separability)

f (Qx) 7→ F (Qu) (rotation)

where Q denotes an orthogonal matrix.

Example

rect(x) rect(y) 7→ 4 sinc(2πu) sinc(2πv)

δ(x)1(y) 7→ 1(u)δ(v)

δ(x) rect(y) 7→ 1(u)2 sinc(2πv)

f (x − 1) + f (x + 1) 7→ (e−i2πu + ei2πu)F (u) =

= 2 cos(2πu)F (u)




A useful fact

If f real (usual case for images):◮ even f 7→ real F◮ odd f 7→ imaginary F◮ F (u) = F (−u)

Observe: F (u, v) is in general complex valued. It is common toillustrate the transform with |F (u, v)|.




DFT and FFT in two dimensions

The discrete Fourier Transform (DFT) of f is defined as

F (u, v) =M−1∑

x=0

N−1∑

y=0

f (x , y)e−i2π( uxM + vy

N ) =

=M−1∑

x=0

N−1∑

y=0

f (x , y)ωuxM ωvy

N =

=(

1 ωuM ω2u

M . . . ω(M−1)uM

)f

1ωv

Nω2v

N...

ω(N−1)vN

,

x = 0, . . . , M − 1, y = 0, . . . N − 1 .




DFT in Matrix form

Let the matrix F represent the Fourier transform of the imagef (x , y):

F = FM fFN

orF = FM(FN f T )T .

i.e. the DFT in two dimensions can be calculated by repeateduse of the one-dimensional DFT, first for the rows, then for thecolumns.




f DFT

DFT

FFredrik Kahl Image Analysis - Lecture 2



FFT on images

Let the M × N-matrix f represent an image f (x , y).Extend the image periodically




FFT on images (ctd.)

FFT gives a double periodic function

fftshift

It is common to move the origin to the middle of the image forillustration purposes.




Interpretation of the Fourier Transform

◮ Usually, the gray-levels of the Fourier Transform imagesare scaled using c log(1 + |F (u, v)|).

◮ The middle of the Fourier image (after fftshift) correspondsto low frequencies.

◮ Outside the middle high components in F corresponds tohigher frequencies and the direction corresponds to"edges"in the images with opposite orientation.




Example

What does the original image look like if this is the Fouriertransform?

Left: Magnitude, Right: Phase




Answer




Masters thesis suggestion of the day: Crosswordreader/solver

Construct a system for automatic crossword scanning andsolution. Idea: Take an image, find squares with text andwithout, interpret the text. Is it possible to solve crosswordsautomatically?




Recommended reading

◮ Forsyth & Ponce: 1. Cameras . Lecture 1.◮ Szeliski: 1. Introduction and 3.1 Point operators .

Lecture 1.◮ Forsyth & Ponce: 7. Linear filters . Lectures 2-3.◮ Szeliski: 3. Image processing , sections 3.2-3.4.

Lectures 2-3.




Review - Lecture 2

◮ Linear Algebra◮ Subspaces◮ Projections, Pseudo-inverse◮ Image matrix◮ Fourier Transform in 1 and 2 dimensions◮ Discrete Fourier Transform in 1 and 2 dimensions◮ Fast Fourier Transform (FFT)


Image Analysis - Lecture 2 › matematiklth › vision › bildanalys › nu › f02.pdf · Review...

Documents

Transcript of Image Analysis - Lecture 2 › matematiklth › vision › bildanalys › nu › f02.pdf · Review...