Digital Image Processing Images © 2002 …bcmi.sjtu.edu.cn/~luhongtao/dip/res/dip04.pdf1 Gianni...

1

Gianni RamponiUniversity of Triestehttp://www.units.it/ramponi

Images © 2002 Gonzalez & WoodsDigital Image Processing

Chapter 4Enhancement in the Frequency Domain:

Fourier transform fundamentals



1-D case:

Three parameters for the sinusoidal functions:-Frequency-Amplitude-Phase

Zero-frequency sine:DC value



∫+∞

∞−−==ℑ dxuxjxfuFxf ]2exp[)()()}({ π

Let f(x) be a continuous (and aperiodic) function of a real variable x. ItsFourier transform is

It is a complex function of the real variable u:

∫+∞

∞−

− −==ℑ duuxjuFxfuF ]2exp[)()()}({1 π

The inverse Fourier transform is given by

)(|)(|)()()( ujeuFujIuRuF Φ=+=







∑−

=

−=1

0

]/2exp[)(1)(N

x

NxujxfN

uF π

For a discrete (sampled), finite length signal, when x takes integer valuese.g. in the range [0,N-1], the Discrete Fourier Transform (DFT) is

It is a complex function of the discrete variable u = 0, 1, 2, … N-1 . Periodic repetition of the input sequence is implied.

The Inverse Discrete Fourier Transform (IDFT) is

∑−

=

=1

0

]/2exp[)()(N

u

NxujuFxf π











2



dxdyvyuxjyxfvuFyxf )](2exp[),(),()},({ +−==ℑ ∫∫+∞

∞−π

If f(x,y) is a continuous function of two real variables x,y, the Fourier pair is

In the discrete, finite-size, supposed periodic case, the 2-D DFT is:

dudvvyuxjvuFyxfvuF )](2exp[),(),()},({1 +==ℑ ∫∫+∞

∞−

− π

∑∑−

=

−

=

+−=1

0

1

0)]//(2exp[),(1),(

M

x

N

yNvyMxujyxf

MNvuF π

∑∑−

=

−

=

+=1

0

1

0

)]//(2exp[),(),(M

u

N

v

NvyMxujvuFyxf π







),(),(),(

),(),(tan),(

)],(),([),(

222

1

2/122

vuIvuRvuF

vuRvuIvu

vuIvuRvuF

+=

⎥⎦

⎤⎢⎣

⎡=

+=

−φ

As a function of the spatial frequenciesu,v we define the- Fourier spectrum, - phase angle, - power spectrum:

DC component (image average): ∑∑−

=

−

=

=1

0

1

0),(1)0,0(

M

x

N

yyxf

MNF





Symmetry: if f is real, F is conjugate symmetric the spectrum is symmetric:

|),(||),(|),(),( * vuFvuFvuFvuF −−=→−−=



Define the 2-D impulse, unit step, exponential, complex sinusoid:

elsewhere0,

0),(

elsewhere0,

01

),(elsewhere

001

),( 10

≥

⎩⎨⎧

=≥

⎩⎨⎧

===

⎩⎨⎧

=yxba

yxfyx

yxuyx

yxuyx

(Chapter 4)Enhancement in the Frequency Domain:


(Chapter 4)Enhancement in the Frequency Domain:


u=.4*pi; v=.1*pi;

[x,y] = meshgrid([0:.1:10],[0:.1:10]); [expo.m]

Real: Imag:

∞≤≤∞−=+= yxjvyjuxvyuxjyxf ,)exp()exp()](exp[),(



It is common practice to multiply f(x,y) by (-1)^(x+y), which movesthe origin of the Fourier transform at u=M/2, v=N/2:





{ } )2/,2/()1)(,( NvMuFyxf yx −−=−ℑ +

3









Separability:

∑

∑∑

∑∑

−

=

−

=

−

=

−

=

−

=

−=

−−=

+−=

1

0

1

0

1

0

1

0

1

0

]/2exp[),(1

]/2exp[),(1]/2exp[1

)]//(2exp[),(1),(

M

x

N

y

M

x

M

x

N

y

MxujvxFM

NvyjyxfN

MxujM

NvyMxujyxfMN

vuF

π

ππ

π











1-D data of size 2^n

2-D FFT: sequence of two 1-D transforms







)()(]exp[)(]exp[)(

)](exp[)()(]exp[)()(

]exp[)()()]()([

;)()()()(;]exp[)()(

uHuFmujmhnujnf

nmujmhnfxujnxhnf

xujnxhnfxhxf

nxhnfxhxfxujxfuF

mn

mnxn

nx

nx

=−−=

+−=−−=

=−−=∗ℑ

−=∗−=

∑∑

∑∑∑∑

∑∑

∑∑

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

Indeed (1-D case, infinite-length sequences, DTFT):

4







Hence, we can perform the convolution as a product in the Fourierdomain, and then take the inverse transform.

Note:for finite-length sequences the product of the DFTs is used, which isequivalent to circular convolution. Zero padding is required to makeit equivalent to linear convolution.

In other words:the Fourier transform assumes that the f and h functions are periodic, with period equal to their length. This should be taken intoaccount, otherwise a wrong result is obtained. The error source isalready visible in the data domain:









Solution: zero padding

10

10)()(

010)(

)(

−+≥⎩⎨⎧

≤≤−≤≤

=

⎩⎨⎧

≤≤−≤≤

=

BAPPxBBxxh

xh

PxAAxxf

xf

e

e











5













An example







When dealing with real images, f=f* : the correlation operationcoincides with the convolution (apart from the + sign in the shifteddata: no mirroring of the h function).

The correlation can be performed in the Fourier domain, after suitablezero padding.







Correlation application: template matching

6







Summary pf properties



















7















Chapter 4(Enhancement in the Frequency Domain)

the 2-D Discrete Cosine Transform

Chapter 4(Enhancement in the Frequency Domain)

the 2-D Discrete Cosine Transform

⎩⎨⎧

>=

=⎩⎨⎧

>=

=

++=

++=

∑∑

∑∑−

=

−

=

−

=

−

=

0if/20if/1)(

0if/20if/1)(

]2

)12(cos[]2

)12(cos[)()(),(),(

]2

)12(cos[]2

)12(cos[)()(),(),(

1

0

1

0

1

0

1

0

vNvNv

uMuMu

Nvy

MuxvuvuFyxf

Nvy

MuxvuyxfvuF

M

u

N

v

M

x

N

y

αα

ππαα

ππαα

With respect to the DFT:• The transform is real. No symmetries.• Better energy compaction performance, especially for highly correlated data.• Reduced artifacts from periodic repetition.• Easily truncated to perform compression.(See dftdct.m)



Chapter 4Image Enhancement in the

Frequency Domain


Frequency Domain

8




Frequency Domain


Frequency Domain

- (Perform zero-padding)

- Multiply f(x,y) by (-1)^(x+y)

- Take g = Realpart of the inverse transform

- Multiply g(x,y) by(-1)^(x+y)

- (Crop the image)




Frequency Domain


Frequency Domain

The displayed image has beenshifted upwards by 128 levels




Frequency Domain


Frequency Domain




Frequency Domain


Frequency Domain

9




Frequency Domain


Frequency Domain

i.e.: unsharp masking




Gaussian lowpass filter



If the filter has a (real) Gaussian shape in the frequency domain, its coefficients in the data domain are real and have a Gaussian shape too:

)2exp(2)()2/exp()( 22222 xAxhuAuH σπσπσ −=⇔−=ℑ

When σ is large (wide passband in the frequency domain), the coefficients profile isnarrow, and viceversa.

Typical 3x3 lowpass filter masks in the data domain can be considered as very roughapproximations of the Gaussian shape.




Difference-of-Gaussians (DoG) filter


Difference-of-Gaussians (DoG) filter

( )

2121

221

222

221

211

22

22

21

21

;

)2exp()2exp(2)(

)2/exp()2/exp()(

σσ

σπσσπσπ

σσ

>≥

−−−=

−−−=

AA

xAxAxh

uAuAuH




Ideal lowpass filter



^ cutoff frequency

It will be a sinc function in the coefficients domain ringing

10
















Ideal lowpass filter example









11



\\\\\\\\








Butterworth lowpass filter



nDvuDvuH 2

0 )/),((11),(

+=

n: filter order

D(u,v): Euclidean distance from center of transform

D0: cutoff frequency ( H(D0)=0.5 )













Fig.4.15 (a) original; (b-d) Butterworth filterof order 2, cutoff freqs. 5, 15, 30.

12







[ ]20

2 2/),(exp),( DvuDvuH −=D(u,v): Euclidean distance from center of

transform

D0: cutoff frequency (σ in slide 32)

H(D0)=0.607

0.607

==







Fig.4.18 (a) original; (b-d) Gaussian filterwith cutoff freqs. 5, 15, 30.













13







National Oceanic and Athmospheric Administration



Chapter 4Highpass filters in the Frequency Domain:

Ideal, Butterworth, Gaussian



In general, Hhp = 1 - Hlp

[ ]20

2 2/),(exp1),( DvuDvuH −−=

nvuDDvuH 2

0 )),(/(11),(

+=

⎩⎨⎧ >

=elsewhere

DvuDifvuH

0),(1

),( 0










Ideal


Ideal

14




Butterworth


Butterworth




Gaussian


Gaussian




Laplacian


Laplacian

),()(),()(),()(),(),(

)()()(

22222

2

2

2

vuFvuvuFjvvuFjudy

yxfddx

yxfd

uFjudx

xfd nn

n

+−=+=⎥⎦

⎤⎢⎣

⎡+ℑ

→=⎥⎦

⎤⎢⎣

⎡ℑ

Hence, a Laplacian filter in the Fourier domain is described as:

Or, when shifted:

)(),( 22 vuvuH +−=

))2/()2/((),( 22 NvMuvuH −+−−=



(value at the top of the dome is zero)


Laplacian


Laplacian

15




Laplacian


Laplacian



Chapter 4Image Enhancement

via subtraction of the Laplacian



),(),(),(

2 yxfyxfyxg

∇

−=

Small details are enhanced; smoothareas become grayish

))2/()2/((1),(

22 NvMuvuH

−+−+

=

Or, equivalently:



High-boost:

),(),()1(),( 2 yxfyxfAyxg ∇−−=

))2/()2/(()1(),(

22 NvMuAvuH

−+−

+−=

(with A>1)











… followed by histogramequalization

16



We restrict ourselves to FIR filters with real coefficients and zero (or linear) phase. Nonlinear-phase filters distort the image: differentfrequency components that make up edges and details lose properregistration.

Zero phase implies symmetry wrt the origin for the coefficients:

Additional symmetries may be expedient to obtain more isotropicresponses:

(Chapter 4)Design of linear filters


),(),(*),(),(*),( yxhyxhyxhvuHvuH −−=−−=⇔=

),(),(),(),(),( xyhyxhyxhyxhyxh =−=−=−−=

Independent coefficientsfor twofold (origin), fourfold, eightfoldsymmetries are:

[J.S. Lim][dipum_4_5.m]



2-D filters can be designed extending methods known for the 1-D case:

Windowing

The desired frequency response Hd(u,v) is assumed known; by IDFT the desired impulse response hd(x,y) is determined, which in general hasinfinite extent; it is truncated via multiplication by a suitable window function: h(x,y)=hd(x,y)w(x,y). In the Fourier domain one obtains: H(u,v)=conv[Hd(u,v),W(u,v)].

The window is derived from a 1-D window (rectangular, Hamming, Kaiser,…) via a separable approach (square window) or by rotation.

0000000000000000x000000000x1x0000000x111x00000x11111x00000x111x0000000x1x000000000x0000000000000000

Frequency sampling

Hd is sampled at equally spaced point on the Cartesian grid, and an IDFT is performed. It isexpedient to avoid sharp changes in Hd: define a transition region to do this. E.g., for a lowpass:





A 2-D function h(x,y) is said to be separable if h1, h2 exist such that

h(x,y)=h1(x)h2(y)

e.g., the delta and unit step sequences are separable

In general, a 2-D function is NOT separable; note indeed that an NxMnonseparable function has NxM degrees of freedom, N+M if separable.

Separable filters are convenient in terms of implementation of the 2-D convolution:

i.e., one can perform a sequence of two 1-D convolutions, first by imagerows (columns) then by columns (rows)

∑∑∑∑∞

−∞=

∞

−∞=

∞

−∞=

∞

−∞=

−−=−−=tsts

tyhtsfsxhtysxhtsfyxg )(),()(),(),(),( 21





The simplest way to design a 2-D filter is to start from a suitable pair of 1-D filters and realize a separable 2-D filter.

Examples…

Such a solution may be not satisfactory wrt isotropy. In this case, the McClellan transformation can be used.

Let us start from a zero-phase 1-D FIR filter, whose frequency responseis expressed as:

( Remember the Chebyshev polynomials of the first kind Tn, defined as

, and which can be built iteratively:

)

nN

n

N

n

N

Nn

N

nnbnnannhhnjnhH )(cos)(cos)(cos)(2)0()exp()()(

001ωωωωω ∑∑∑ ∑

==−= =

==+=−=

)(cos)cos( ωω nTn =)()(2)(;)(;1)( 2110 xTxxTxTxxTxT nnn −− −===



17



Determine a mapping for the frequency domain such that:

The function G maps each frequency into a 2-D contour. McClellan chosethe mapping with

),(|)(),( vuGHvuH == ωω

),(cos 1 vuQ−=ω)cos()cos()cos()cos(),( 4

141

21

21

21 vuvuvuvuQ −+++++−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

121242121

81q

which when the inverse Fourier transform iscomputed yields the set of space-domain values:

The resultingmapping showsgood circularsymmetry, especially in the low-frequencyregion

[mcclellan.m]





We shall not deal with IIR 2-D filters:

the fundamental theorem of algebra does not extend to 2-D!

It is not possible in general to factorize a 2-D polynomial

The pole-zero stability analysis cannot be performed

Simple cascade realizations of low-order sections cannot be obtained

Theorems exist however for determining stability of an IIR filter basedon its 2-D z transform

It is also possible to limit oneself to filters with separable denominator, which however constitute a very restricted class





S. Agaian

Trans. IP March 07

Transform-domain nonlinear filters:Alpha rooting




Smaller transform coefficients (= tipically, higher frequencies) are less attenuated.

-- The signal power is in fact reduced. It hasto be recovered via a suitable scaling; e.g.:

-- The actual scaling factor dependson the input image content

-- The output image is darker than the original, since the most strongly attenuatedcomponent is the largest: the dc one



x = 0:1:255;for alpha=.5:.1:1;

y = x .^ alpha;plot(x,y); hold on; grid on;

end;

18



An objective quality metric can sometimesbe used to select the ideal alpha value:





F. Arslan

Trans.IPNov. 06










via Homomorphic filtering



Image formation model based on illumination and reflectance:

)ln()ln()ln(),(),(),( rifyxryxiyxf +=→=illumination and reflectance can be separated in the log domain;

the former issupposed to varysmoothly and isattenuated, the latter is supposedto vary sharplyand is amplified

(γL<1<γH)

19









• Problem: image enhancement– Poorly illuminated images – Backlights– Light sources within the image

• Solution:– Retinex-based approach improve

the lightness in dark areas and emphasize the details

Retinex-based algorithmsRetinex-based algorithms

Courtesy Dr. Stefano Marsi



• Retinex = Retina + Cortex• We perceive an object in the

same way, independently of the illumination conditions

• A perceived image I can be considered as a product of the scene illumination L and theobjects reflectance R

L

I

R

I



),(),(),( yxRyxLyxI =



• Through the estimation of the illumination it is possible to obtain an approximation of the reflectance.

• The components are separately processed and then recombined– The Reflectance (details) is suitably emphasized– The Illumination is improved in dark areas

• In the case of logarithmic sensors:– The multiplication can be replaced with an addition block– The division can be replaced with a subtraction block– The Reflectance can be suitably emphasized through a

multiplicative coefficient

: *

Luminanceestimation

In Out

L

R~

~

…

…IlluminationEstimation



20



• Illumination characteristics– Illumination typically changes very smoothly between

contiguous pixels – Abrupt transitions can also be present

• In presence of light sources in the image• In presence of different illumination systems (backlight)

• The Illumination must be estimated using a narrow band (edge-preserving) lowpass filter– In the literature these filters are realized:

– via very large masks– via recursive filtering– via multi-resolution decomposition

Retinex-based algorithms:Estimation of the illuminationRetinex-based algorithms:

Estimation of the illumination

Courtesy Dr. Stefano Marsi Gianni RamponiUniversity of Triestehttp://www.units.it/ramponi


• We propose:the use of the Recursive Rational Filter (RRF)

A simple first order IIR filter is

a can be used to trim the bandwidth:-- a 1 : very narrow band-- a = 0 : the filter is switched off

)1()()1()( −+−= naynxany






1),(]1)1)([(])1,(),1([),(

+++−++−+−

=ov

ovov

SSmnxSSSmnySmnymny αα

δ+⎟⎟⎠

⎞⎜⎜⎝

⎛++−+

= 2

)1,(1)1,(1log

mnxmnx

HSo

δ+⎟⎟⎠

⎞⎜⎜⎝

⎛++−+

= 2

),1(1),1(1log

mnxmnx

HSv

Where

So and Sv respectively are the horizontal and vertical edge sensors







Courtesy Dr. Stefano MarsiCourtesy Dr. Stefano Marsi

21



• A modified “gamma” function is used …– To increase the lightness in dark zones– To avoid dynamic compression in bright areas

• … followed by a histogram stretching– To better exploit the system dynamics

( ) KLKLL +=Γ )255~

( 255~

255)~(

Retinex-based algorithms:Processing the illuminationRetinex-based algorithms:Processing the illumination



• The reflectance must be processed in a multiplicative domain

• A sigmoid-like function is used to:– emphasize the details when these are poorly defined.– limit the emphasis when the details are already well

defined to avoid artifacts generation– reduce the signal when it is extremely weak

(the information is superseded by the noise)

log expR~



2R1R



• The adopted function is:

where c is a suitable coefficient which controls the slope of the sigmoid and creates the central dead zone

⎟⎟⎠

⎞⎜⎜⎝

⎛−

−+=

21

))(exp(112

112 RRc

KR




Originaloutput

IlluminationEstimation

ProcessedIllumination

ReflectanceEstimation

ProcessedReflectance

Retinex-based algorithms:Processing example

Retinex-based algorithms:Processing example


22



1. Original2. Histogram equalization3. Gamma correction4. Retinex-based method

Retinex-based algorithms:a comparison

Retinex-based algorithms:a comparison



(a) (b)

without (a) and with (b) the dead-zone

Retinex-based algorithms:Noise control

Retinex-based algorithms:Noise control




Retinex-based algorithms:More images…







Digital Image Processing Images © 2002 …bcmi.sjtu.edu.cn/~luhongtao/dip/res/dip04.pdf1 Gianni...

Documents

Transcript of Digital Image Processing Images © 2002 …bcmi.sjtu.edu.cn/~luhongtao/dip/res/dip04.pdf1 Gianni...