Athens Week 1

Introduction to image processing and linear filtering

F. Tupin

Athens Week 2

Introduction to image processing

What is an image ?Spatial resolution and Fourier transformQuantification and histogramLinear filtering

Athens Week 3

Introduction to image processing

What is an image ?Spatial resolution and Fourier transformQuantification and histogramLinear filtering

Athens Week 4

What is an image ?Message support Signal (continuous 2D signal) of a physical measure

Passive imagery (Colour intensity) Active imagery (X-ray transmission,

electromagnetic radiation, …)Other dimensions:

Volumetric measures (3D medical images) 2D ½ (stereo images + intensity) Video (2D+t) 3D+t (images sequences)

Athens Week 5

What is a numerical image ?A digital image (2D) is a matrix defined by its resolution (amount of pixels) its depth (amount of potential values for each

pixel) its palette (colour look up table (CLUT))

Athens Week 6

Digital Image

Athens Week 7

Different kind of imagesImage Application Size Channels

Video VisiophonyTVHD TV

256 x 256720 x 6251920 x 1150

133

Bio-medical Tomography (IRM)Radiography

512x512256x256x2561024 x 1024

1-3

Remote Sensing

1970-19801985-19901990-

2000 x 30006000 x 600015000 x 15000

3-73-203-256

Robot vision Quality controlAutomatic driving

256 x 256512 x 512

12-3

Defense Surveillance, tracking

256 x 256512 x 512

12-3

Athens Week 8

From the continuous world to the numerical one Image digitization Sampling (number of pixels) Quantification (number of bits per

pixel)

Athens Week 9

From the continuous world to the numerical one Image digitization Sampling (number of pixels) Quantification (number of bits per

pixel)

Athens Week 10

Introduction to image processing

What is an image ?Spatial resolution and Fourier transformQuantification and histogramLinear filtering

Athens Week 11

Resolution : Vision (physical Image) vs. Digital Image

Resolution Human vision is able to look (in stereo) at from

very small objects to huge ones The digital image has a fixed resolution

determined by the amount of pixels of the image.Sampling Preserving the image frequencies (Shannon

theory, 2D Fourier transform) Preserving the image content: depends of each

application (psycho-visual tests, size of the smallest element to detect, …)

Athens Week 12

Resolution of an imageNl : amounts of lines per picture height

Maximum frequency : Nl/2 cycles per picture heightIf D is given (in TV, D=6H), there is a direct relationship between cycles per degree and cycles per picture height

ViewerViewerHH

DD

Athens Week 13

Sampling and informationintuitive approach with example

Aliasing 1D : false frequency

Athens Week 14

Sampling and informationintuitive approach with example

Aliasing 2D : false frequency and false direction of periodical structures !

Athens Week 15

1-D Fourier transform

Sampling at Fe spectrum periodization at Fe Avoid aliasing: low pass filtering in [-Fe/2;Fe/2]All information in [-Fe/2;Fe/2]: normalized frequency

dfefGtg

dtetgfG

tfj

tfj

c

tc

2

2

)()(

)()(

Athens Week 16

1D discrete Fourier transform

Signal sampling Fourier periodizationFrequency sampling signal periodizationNumerical signals = implicit periodization in both temporal and frequency domains !

0.5;0.5ou1;0)()( 2

ffengfG nfj

n

Athens Week 17

2D Fourier transform

Fourier transform = signal projection on 2D complex sinusoidFrequency domain = new visualization of image informationFourier transform = complex signal (phase + modulus)

Athens Week 18

2D Discrete Fourier Transform

2D DFT

Number of frequencies = number of pixels f(i,j) is the image in the spatial domain. Exponential term is the basis function

corresponding to each point, F(k,l), in the Fourier space.

1

0

1

0

)(2),(1),(

N

i

N

j

Nlj

Nkii

ejifN

lkF

Athens Week 19

2D DFT - Properties

1

0

1

0

)(2),(1),(

N

k

N

l

Nlj

Nkii

elkFN

jif

F(0,0) corresponds to the average brightnessF(N-1, N-1) represents the highest frequency

Inverse Fourier transform:

Athens Week 20

2D DFT computation

Separation of DFT Formula:

Expressing two-dimensional Fourier transform in terms of series of two N one-dimensional transforms decreases the number of required computations.

1

0

2),(1),(

N

i

Nkii

ejifN

jkP

21

0),(1),(

NljiN

jejkP

NlkF

1

0

1

0

)(2),(1),(

N

i

N

j

Nlj

Nkii

ejifN

lkF

Athens Week 21

FFTFast Fourier Transform(FFT)

N2 complexity is reduced to Nlog2N. Limited to image size N = 2n.

FFT is complex Real / imaginary or magnitude /phase Magnitude linked to geometric information Phase linked to spatial position To re-transform we need to use both magnitude and phase. If real image, symmetry vs 0

Athens Week 22

2-D discrete Fourier transformSimple examples

2 pixel wide vertical stripes Fourier transform

Athens Week 23

DFT : simple examplesVertical lines : No variation along y axis Variation along x axis linked to the spatial

frequency of the lines

Fourier peak P(k,l) interpretation: |P(k,l)| related to signal frequency (O,P(K,l)) perpendicular to the pattern

direction

Athens Week 24

DFT simple examples

Athens Week 25

DFT Real examples

Original image Fourier transform Logarithmic operator applied

Athens Week 26

DFT Real examples

Athens Week 27

DFT Real examples

Athens Week 28

DFT Phase information

The value of each point determines the phase of the corresponding frequency.

The phase information is crucial to reconstruct the correct image in the spatial domain.

Phase image Re-transform using only magnitude

Athens Week 29

DFT properties

),(),()),(),((

),(),()),(),((

),()),((

),()),((

),()),((),(),(),(),(

),(),(),(

2

2

002

**

00

HFmnhmnf

HFmnhmnf

Femnf

Fmnf

FmnfFFmnfmnf

qpFqpF

D

D

mnj

TFD

TFD

TFDTFDTFDTFD

ZxZ

Athens Week 30

Back to the sampling problem…

Must be achieved at a frequency superior to 2 times the highest frequency in the signal (Nyquist frequency)

Or the original image has to be filtered at a frequency below the half of the sampling frequency

Athens Week 31

Original and spectrum

Athens Week 32

Sub-sampled image and spectrum

Athens Week 33

Image comparison

Original image Zoom of the sub-sampled image

Athens Week 34

Spectrum comparison

Original spectrum Zoom of the sub-sampled image spectrum

Athens Week 35

ExplanationOriginal image spectrum is defined in [-0.5,0.5] The 2-D pattern spectrum [-0.5,0.5] x [-0.5,0.5] is

periodically repeated in (0,1), (O,2),…(p,q),…When doing the sub-sampling: The 2-D pattern spectrum [-0.5,0.5] x [-0.5,0.5] is

periodically repeated in (0,0.5), (O,1),…(p/2,q/2),…

The different patterns overlap in the Fourier domain

The new spectrum corresponds to the [0.25,0.25] x [-0.25,0.25] area

Athens Week 36

Initial spectrum and pattern of the sub-sampled spectrum

Athens Week 37

Athens Week 38

(0,0) (1,0)(-1,0)

Periodization on (p,q)

Athens Week 39

(0,0) (1,0)(-1,0)

Periodization on (p/2,q/2)

Athens Week 40

(0,0) (1,0)(-1,0)

Selection of [-0.25,0.25]x [-0.25,0.25]

Athens Week 41

Athens Week 42

Spectrum comparison

Original spectrum Zoom of the sub-sampled image spectrum

Athens Week 43

Sub-samplingTo avoid aliasing when sub-sampling Low-pass filtering (suppression of all the

frequencies higher than Fe’/2) Blurring effect but no artefacts are

introduced

Athens Week 44

Athens Week 45

Introduction to image processing

What is an image ?Spatial resolution and Fourier transformQuantification and histogramLinear filtering

Athens Week 46

Depth of an imageBinary representation: b bits, 2b levels

whitewhite

blackblack

11

00 0000

0101

1010

1111

Athens Week 47

The depth : Vision vs. Digital Image

Vision sensitivity limited to 6 to 8 bits per colour component i.e.

max 24 bits Vision works in the (350nm) to red (700nm) range

Digital images Much larger depth (in medical imaging and in

remote sensing) Different wavelengths than vision: NMR imaging

(0,001nm) to microwaves images (100000nm) Image visualization: 8 bits = 256 grey-levels(0 for black and 255 for white)

Athens Week 48

Visible domainrayons

cosmiquesrayons

gammasrayons

Xultra violet visible infra

rougemicroondes TV radio 50 Hz

10-5 nm 10-3 nm 10 nm 500 nm 1500 nm 5 m 1000 m

VIOLET BLUE GREEN YELLOW RED

380 nm 500 nm 555 nm 600 nm 720 nm

Puech William Université Montpellier II - Nîmes

Athens Week 49

Image Histogram

Histogram: grey level distribution F(x) = number of pixels having x as grey-

level If normalized = probability of the grey-level

Used to Contrast enhancement Classification

Athens Week 50

Histogram: example

Histogram modes correspond to region of interests (clouds, parts of the boats, etc.)

Athens Week 51

Histogram …. be careful !

Athens Week 52

Histogram …. be careful !

Athens Week 53

Histogram Stretch Linear Stretch

Histogram Equalised Stretch

Athens Week 54

Athens Week 55

Colour spacesThe RGB space (3 types of phosphors for colour excitation on a screen)

BB

GG

RR

blackblack

whitewhite

Athens Week 56

Colours (cont.)Printing industry : CYM(K) Cyan Yellow Magenta (can be translated into RGB)Hue Saturation Intensity (HSI): artists, vision

II

SSHH

blackblack

whitewhiteredred

greengreen

blueblue

Athens Week 57

Colours(cont.)YUV : luminance, chrominances in TV black and white TV backward compatibility Decorrelation of the components Y is containing most of the information

Y=0.299R+0.587G+0.114BU=-0.147R-0.289G+0.437BV=0.615R-0.515G-0.100B

Athens Week 58

Video formatsCCIR 601: 576 lines of 720 pixels, 25 images per second, 2 interlaced fields per frame, format 4:2:2 (YUYVYUYV) = 165 Mbit/sHDTV: 16/9 format, 2*576 lignes, 720*2(*4/3) pixels (non interlaced and even higher for digital cinema)TV 16/9CIF: CCIR 601 /2 /2 /2QCIF: CIF /2 /2

Athens Week 59

Colour CompositesA colour composite is a colour image produced through optical combination of multiband images by projection through filters.True Colour Composite: A colour imaging process whereby the colour of the image is the same as the colour of the object imaged.False Colour Composite: A colour imaging process which produces an image of a colour that does not correspond to the true colour of the scene (as seen by our eyes).

Athens Week 60

TM7,4,1

TM5,7,2

TM5,4,3

TM4,3,2

Athens Week 61

Introduction to image processing

What is an image ?Spatial resolution and Fourier transformQuantification and histogramLinear filtering

Athens Week 62

Linear filteringProperty :F(I+aJ)=F(I)+aF(J)

Image domain: convolutiony=x*hFourier domain : multiplicationY=XH

Athens Week 63

Convolution: involves moving a 'window' of a few pixels in dimension (e.g. 3x3, 5x5, etc.) over each pixel in the image, applying a mathematical calculation using the pixel values under that window, and replacing the central pixel with the new value.

Athens Week 64

Linear filtering

Athens Week 65

Linear Filtering

),(),(),( lmknxlkhmnylk

),(),(),( yxyxyx ffXffHffY

Spatial domain (h convolution kernel)

Frequency domain

Athens Week 66

Linear FilteringLow pass filters:noise suppression (= high frequencies)

Athens Week 67

Linear filteringHigh or band pass filters: Selection of frequencies of interest

High frequencies (edges) Specific frequencies (texture analysis)

Athens Week 68

High pass filter = edge detector

Athens Week 69

Stripes of the zebra create high energy waves generally along the u-axis; grass pattern is fairly random causing scattered low frequency energy

y

x

v

u

Athens Week 70

Non-linear filtering

Simple non-linear filters = median filters: the output of the filter is the median in a given windowMin / Max : mathematical morphology