Introduction to DeconvolutionIntroduction to DeconvolutionImage ProcessingImage Processing

Introduction to Light and Electron Introduction to Light and Electron MicroscopyMicroscopy

Neu259Neu259 Spring 2006Spring 2006

James BouwerJames BouwerUCSDUCSD

OutlineOutline

Convolution:Convolution:2D:2D:• Airy Disk (Light Point Spread Function)Airy Disk (Light Point Spread Function)• Convolution with an ImageConvolution with an Image

3D:3D:• 3D Point Spread Function3D Point Spread Function• Convolution with a VolumeConvolution with a Volume

Deconvolution:Deconvolution:• Frequency space decompositionFrequency space decomposition• Fourier TransformsFourier Transforms• Application of the Deconvolution TheoremApplication of the Deconvolution Theorem

Some Examples:Some Examples:

The Microscope Optical Train is ComplexThe Microscope Optical Train is Complex

• Every lens element alters Every lens element alters the image in some waythe image in some way

• We call this the image We call this the image transfer function: h(x,y,z)transfer function: h(x,y,z)

• The image transfer functionThe image transfer function of the system is a of the system is a convolutionconvolution of all of the of all of the image transfer functions ofimage transfer functions of all the lens elements and all the lens elements and apertures in the optical apertures in the optical traintrain

The Definition of Convolution:The Definition of Convolution:

Con-vo-lut-ed: adj.Con-vo-lut-ed: adj.1.Having numerous 1.Having numerous overlapping coils or folds: overlapping coils or folds: a convoluted a convoluted seashell.seashell.2.Intricate; complicated: 2.Intricate; complicated: convoluted legal language; convoluted convoluted legal language; convoluted reasoning.reasoning.

The American Heritage® Dictionary of the English Language, Fourth Edition

-1 -0.5 0 0.5 1

x 10-6

-1

-0.5

0

0.5

1

x 10-6

-6 -4 -2 0 2 4 6

x 10-6

-6

-4

-2

0

2

4

6

x 10-6

Formation of an Airy Disk PatternFormation of an Airy Disk Pattern

Impulse Response Function Impulse Response Function [ Point Spread Function (psf)][ Point Spread Function (psf)]

A point in the object spaceA point in the object space A point convolved with the transfer A point convolved with the transfer Microscope functionMicroscope function

Impulse Response FunctionImpulse Response FunctionApplied to a lineApplied to a line

A great advantage is afforded by the ability to express A great advantage is afforded by the ability to express the response of the optical system to an arbitrary the response of the optical system to an arbitrary input in terms of the response to certain “elementary” input in terms of the response to certain “elementary” functions into which the input has been decomposedfunctions into which the input has been decomposed

-6 -4 -2 0 2 4 6

x 10-6

-6

-4

-2

0

2

4

6

x 10-6

Line Transfer FunctionLine Transfer Function

slide courtesy of Edgar Gardunoslide courtesy of Edgar Garduno

How the How the psfpsf (Impulse Response) (Impulse Response) Effects ResolutionEffects Resolution

Image Transfer FunctionImage Transfer Function

-6 -4 -2 0 2 4 6

x 10-6

-6

-4

-2

0

2

4

6

x 10-6

ObjectObject ImageImage

iimage (x, y) iobject (xi , y j ) h(x xi , y y j )j

i

h(x,y)h(x,y)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

x 10-6

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

h r; NA,

2 J1

r r

2 (NA)

wherewhere

Theoretical Model of the Impulse Transfer Function, h(x,y)Theoretical Model of the Impulse Transfer Function, h(x,y)(PSF) (PSF)

JJ11=Bessel function of the first kind=Bessel function of the first kind

The Definition of Convolution:The Definition of Convolution:

Con-vo-lut-ed: adj.Con-vo-lut-ed: adj.1.Having numerous 1.Having numerous overlapping coils or folds: overlapping coils or folds: a convoluted a convoluted seashell.seashell.2.Intricate; complicated: 2.Intricate; complicated: convoluted legal language; convoluted convoluted legal language; convoluted reasoning.reasoning.

The American Heritage® Dictionary of the English Language, Fourth Edition

Convoluted Mess!Convoluted Mess!

Spatial Frequency DecompositionSpatial Frequency DecompositionFourier TransformFourier Transform

-50 0 50 100 150 200 250 300

Pixel

I(x) ai(coskix) ibi(sinkix)i

• Any image can be Any image can be decomposed into a series of decomposed into a series of sines and cosines added sines and cosines added together to give the imagetogether to give the image

0.25µm myelin0.25µm myelin

AmplitudesAmplitudes PhasePhase

Fourier TransformFourier Transform

Low frequencyLow frequency

High frequencyHigh frequency

Fourier Transform Fourier Transform of the Myelin Image of the Myelin Image

Mathematical Formulation of the Mathematical Formulation of the Fourier TransformFourier Transform

(g) g(x, y)e 2 i( fx x fy y)dxdy

2-D Fourier Transform:2-D Fourier Transform:

Real Space to..Frequency SpaceReal Space to..Frequency Space

Inverse 2-D Fourier Transform:Inverse 2-D Fourier Transform:

1(G) G( fx , fy )e2 i( fx x fy y)dfxdfy

Frequency Space to.. Real SpaceFrequency Space to.. Real Space

Inverse Fourier Transform of the FourierInverse Fourier Transform of the FourierTransform Returns the original ImageTransform Returns the original Image

==

Fourier transform of myelinFourier transform of myelin

FF -1 -1

Very Powerful Tool !Very Powerful Tool !

The Wonderful, Great and AmazingThe Wonderful, Great and Amazing Convolution TheoremConvolution TheoremIf If {g(x, y)} G( fx , fy )

{h(x, y)} H ( fx , fy )andand

Then:Then:

{g(x, y)h(x, y)} G( fx , fy )H ( fx , fy )

iimage (x, y) iobject (xi , y j ) h(x xi , y y j )j

i

Remembering that the image intensity is a convolutionRemembering that the image intensity is a convolutionof the impulse function of the impulse function h(x,y)h(x,y) and the object I and the object Iobjectobject(x,y)(x,y)

The object intensity can be easily de-convolved from theThe object intensity can be easily de-convolved from theSmear of the impulse function (PSF)Smear of the impulse function (PSF)

iimage (x, y) iobject (xi , y j ) h(x xi , y y j )j

i

Deconvolution using the Convolution TheoremDeconvolution using the Convolution Theorem

The image is a convolution of the impulse functionThe image is a convolution of the impulse functionh(x,y) (PSF) and the object (the sample): h(x,y) (PSF) and the object (the sample):

{iimage} {iobject (x, y) h(x, y)} Iobj ( fx , fy )H ( fx , fy )

Therefore, the Fourier transform of the image is just theTherefore, the Fourier transform of the image is just theFourier transform of object times the Fourier transformFourier transform of object times the Fourier transformof the impulse function (PSF) in frequency spaceof the impulse function (PSF) in frequency space

So, to obtain the object, we simply divide by the FourierSo, to obtain the object, we simply divide by the Fouriertransform of the impulse function (PSF)transform of the impulse function (PSF)

Iobj ( fx , fy ) Iobj ( fx , fy )H ( fx , fy )

H ( fx , fy )

Iobj ( fx , fy ) Iobj ( fx , fy )H ( fx , fy )

H ( fx , fy )

Finally, we obtain the deconvolved object (sample)Finally, we obtain the deconvolved object (sample)by applying an inverse Fourier transformby applying an inverse Fourier transform

Transform back to real spaceTransform back to real space

iobject (x, y) 1{Iobj ( fx , fy )} 1 Iobj ( fx , fy )H ( fx , fy )

H ( fx , fy )

A simple division in frequency space yields theA simple division in frequency space yields theObject intensity !Object intensity !

Let’s Run though it…Let’s Run though it…

Step 1: Acquire the image and it’s Fourier transformStep 1: Acquire the image and it’s Fourier transform

Convoluted ImageConvoluted Image Fourier Transform Fourier Transform

Step 2: Obtain the impulse function (PSF) and the FourierStep 2: Obtain the impulse function (PSF) and the Fouriertransform of the impulse function (PSF)transform of the impulse function (PSF)

-6 -4 -2 0 2 4 6

x 10-6

-6

-4

-2

0

2

4

6

x 10-6

Steps to Deconvolution…Steps to Deconvolution…

Point spread functionPoint spread function Fourier transformFourier transform

Step 3: Divide and inverse Fourier transformStep 3: Divide and inverse Fourier transform

FF -1 -1

Steps in the Deconvolution…Steps in the Deconvolution…

3D Out-of-focus Point Spread Function3D Out-of-focus Point Spread Function3D Impulse Response3D Impulse Response

xx

zz

yy2-D PSFs vs. z-height2-D PSFs vs. z-height

QuickTime™ and a decompressor

are needed to see this picture.

A Real 3-D PSFA Real 3-D PSFFrom the RTS-2000From the RTS-2000

100nm Diameter Fluorescent Latex Bead100nm Diameter Fluorescent Latex BeadImaged with 50nm Steps in z Imaged with 50nm Steps in z

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

… … Plenty more to this storyPlenty more to this story… … Many commercial software packagesMany commercial software packages

are available in at the NCMIRare available in at the NCMIR