ThinBlinDe: Blind deconvolution for confocal microscopy
-
Upload
inriaariana -
Category
Documents
-
view
231 -
download
0
Transcript of ThinBlinDe: Blind deconvolution for confocal microscopy
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
1/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
ThinBlinDe: Blind deconvolution for confocalmicroscopy
Praveen Pankajakshan,
Laure Blanc-Fraud, Josiane Zerubia.
Ariana joint research group,INRIA/CNRS/UNS,
Sophia Antipolis, France.
This research work was partially funded byP2R Franco-Israeli project and ANR DIAMOND project. Access to
the Applied optics paper is here.
October 2010
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 1 of 42
http://www-sop.inria.fr/ariana/Projets/P2R/http://www-syscom.univ-mlv.fr/ANRDIAMOND/http://www.opticsinfobase.org/abstract.cfm?uri=ao-48-22-4437http://www.opticsinfobase.org/abstract.cfm?uri=ao-48-22-4437http://www-syscom.univ-mlv.fr/ANRDIAMOND/http://www-sop.inria.fr/ariana/Projets/P2R/ -
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
2/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Road map of presentation
1 IntroductionOptical sectioning fluorescence microscopyLimitations of imaging
2
ThinBlinDe software for deconvolutionBreaking the limit-computationallyBlind deconvolution
3 Results
Phantom dataMicrospheresPlant samples
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 2 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
3/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Holy grail of biological imaging
The holy grail for most cell
biologists
E. Betzig et al. Imaging intracellular fluorescent proteins at nanometer resolution, Sc. Exp., Aug. 2006.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 3 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
4/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Holy grail of biological imaging
The holy grail for most cell
biologists
dynamically image living cellsnoninvasively,
E. Betzig et al. Imaging intracellular fluorescent proteins at nanometer resolution, Sc. Exp., Aug. 2006.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 3 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
5/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Holy grail of biological imaging
The holy grail for most cell
biologists
dynamically image living cellsnoninvasively,
to see at the level of anindividual protein molecule,and
E. Betzig et al. Imaging intracellular fluorescent proteins at nanometer resolution, Sc. Exp., Aug. 2006.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 3 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
6/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Holy grail of biological imaging
The holy grail for most cell
biologists
dynamically image living cellsnoninvasively,
to see at the level of anindividual protein molecule,and
to see how these molecules
interact in a cell.
E. Betzig et al. Imaging intracellular fluorescent proteins at nanometer resolution, Sc. Exp., Aug. 2006.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 3 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
7/105
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
8/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Confocal laser scanning microscope
Figure 1: Schematic of a confocal laserscanning microscope (Minsky (1988)).
A- Laser
B- Beam expander
C- Dichroic mirror
D- ObjectiveE- In-focus plane
F- Confocal pinhole
G- Photomultiplier tube
(PMT)
Minsky (1988) Memoir on Inventing the Confocal Scanning Microscope. Scanning (10): 128138.P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 4 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
9/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
3D imaging by optical sectioning
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 5 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
10/105
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
11/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Point-spread function
Figure 2: Experimental PSFdetermination by imaging point sources.
Figure 3: 2D Airy disk function.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 6 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
12/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Illustration: Diffraction effect
Figure 4: Single green fluorescence protein (GFP) if imaged using a widefieldmicroscope. (NA = 1.4, ex = 500nm)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 7 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
13/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Illustration: Effect of circular pinhole
Airy Unit (AU); 1AU= 1.22ex/NA
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 8 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
14/105
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
15/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Illustration: Effect of circular pinhole
1AU 3AU
Airy Unit (AU); 1AU= 1.22ex/NA
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 8 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
16/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Illustration: Effect of circular pinhole
1AU 3AU Fully open
Figure 5: Effect of changing confocal pinhole size on the out-of-focus fluorescence,contrast and SNR.
Airy Unit (AU); 1AU= 1.22ex/NA
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 8 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
17/105
ThinBlinDe
P. Panka-jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
"The triangle of trade-off"
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 9 of 42
Ill S h l b
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
18/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Illustration: Spherical aberration
Figure 6: Schematic of spherical aberration. i is the azimuthal angle in the lensimmersion medium, s is the angle in the specimen and d (NFP) is the depthunder the cover slip and into the specimen. The refractive index of the cover slip is
assumed to be either equal to the index of the lens ni or the specimen ns.P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 10 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
19/105
Th l i
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
20/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
The eternal question
Figure 8: How to get the best of your microscope?
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 12 of 42
P bl t t t
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
21/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
P bl t t t
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
22/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by blurring, due to diffraction limits and aberrations,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
Problem statement
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
23/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by blurring, due to diffraction limits and aberrations,
when pinhole size is 1 airy units (AU), 30% of photonscollected are from out-of-focus sections.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
Problem statement
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
24/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by blurring, due to diffraction limits and aberrations,
when pinhole size is 1 airy units (AU), 30% of photonscollected are from out-of-focus sections.
shot noise, occurring as detectable statistical fluctuations,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
Problem statement
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
25/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by blurring, due to diffraction limits and aberrations,
when pinhole size is 1 airy units (AU), 30% of photonscollected are from out-of-focus sections.
shot noise, occurring as detectable statistical fluctuations, spherical aberrations due to mismatch in medium,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
Problem statement
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
26/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by blurring, due to diffraction limits and aberrations,
when pinhole size is 1 airy units (AU), 30% of photonscollected are from out-of-focus sections.
shot noise, occurring as detectable statistical fluctuations, spherical aberrations due to mismatch in medium,
exact point-spread function (PSF) is unknown and varieswith experiment.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
Problem statement
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
27/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Problem statement
Images from fluorescent optical sectioning microscopes (OSM)
are affected by blurring, due to diffraction limits and aberrations,
when pinhole size is 1 airy units (AU), 30% of photonscollected are from out-of-focus sections.
shot noise, occurring as detectable statistical fluctuations, spherical aberrations due to mismatch in medium,
exact point-spread function (PSF) is unknown and varieswith experiment.
Often only a single observation of the object is given (to avoidphotobleaching) for restoration!
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 13 of 42
Breaking the diffraction barrier
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
28/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Breaking the diffraction barrier
Ernst Karl Abbe [1840-1905]
S. Hell et al. (1994) Breaking the diffraction resolution limit by stimulated emission:stimulated-emission-depletion fluorescence microscopy. OL 19 (11): 780782.
M. J. Rust et al. (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy
(STORM). NM 3 (10): 793
796.P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 14 of 42
Breaking the diffraction barrier
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
29/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Breaking the diffraction barrier
Ernst Karl Abbe [1840-1905]
Abbes diffraction limitapproximation:
d=ex
2nosinmax
S. Hell et al. (1994) Breaking the diffraction resolution limit by stimulated emission:stimulated-emission-depletion fluorescence microscopy. OL 19 (11): 780782.
M. J. Rust et al. (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy
(STORM). NM 3 (10): 793
796.P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 14 of 42
Breaking the diffraction barrier
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
30/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSMLimitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Breaking the diffraction barrier
Ernst Karl Abbe [1840-1905]
Abbes diffraction limitapproximation:
d=ex
2nosinmax
Recent fluorescence lightmicroscopy techniques are aimed atsurpassing the Abbe resolutionlimit (Hell, et al.(1994), Betzig, etal. (2006), Rust, et al.(2006)).
S. Hell et al. (1994) Breaking the diffraction resolution limit by stimulated emission:stimulated-emission-depletion fluorescence microscopy. OL 19 (11): 780782.
M. J. Rust et al. (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy
(STORM). NM3
(10
):793796
.P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 14 of 42
Breaking the diffraction barrier
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
31/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Breaking the diffraction barrier
Ernst Karl Abbe [1840-1905]
Abbes diffraction limitapproximation:
d=ex
2nosinmax
Recent fluorescence lightmicroscopy techniques are aimed atsurpassing the Abbe resolutionlimit (Hell, et al.(1994), Betzig, etal. (2006), Rust, et al.(2006)).
Resulting images are quite similarand differences in the operationaldetails can make these techniquesmore or less suitable for specific
types of biological studies.
S. Hell et al. (1994) Breaking the diffraction resolution limit by stimulated emission:stimulated-emission-depletion fluorescence microscopy. OL 19 (11): 780782.
M. J. Rust et al. (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy
(STORM). NM3
(10
):793796
.P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 14 of 42
Imaging Model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
32/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Imaging Model
Image formation statistics is Poissonian, and
i(x) = P((ho+ b)(x)),x s
where denotes 3D deconvolution,
O(s = {o = oxyz : s N3
R}),
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 15 of 42
Imaging Model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
33/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
g g
Image formation statistics is Poissonian, and
i(x) = P((ho+ b)(x)),x s
where denotes 3D deconvolution,
O(s = {o = oxyz : s N3
R}), h : s R models the PSF,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 15 of 42
Imaging Model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
34/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
g g
Image formation statistics is Poissonian, and
i(x) = P((ho+ b)(x)),x s
where denotes 3D deconvolution,
O(s = {o = oxyz : s N3
R}), h : s R models the PSF,
b is the low frequency background fluorescence signal,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 15 of 42
Imaging Model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
35/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
g g
Image formation statistics is Poissonian, and
i(x) = P((ho+ b)(x)),x s
where denotes 3D deconvolution,
O(s = {o = oxyz : s N3
R}), h : s R models the PSF,
b is the low frequency background fluorescence signal,
1/ is the photon conversion factor, and i(x) is the
observed photon at the detector.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 15 of 42
Trends: Deconvolution and CLSM
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
36/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Jan 2004 Jun 2006 Jan 200910
20
30
40
50
60
70
80
90
100
Time
Normalizedn
umberofsearches
Confocal Microscopy
Deconvolution
Figure 9: Annual search trends forCLSM and deconvolution between2004-2009 (Source: Google).
0 20 40 60 80 100
10
20
30
40
50
60
70
80
90
Relative number of searches for "confocal microscopy"
Relativenumberofse
archesfor"deconvolution"
Figure 10: Scatter plot between thekeyword hits deconvolution andconfocal microscopy (Source:Google). Dotted line is the line of leastsquare (LS) fit.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 16 of 42
Trends: scientific interest
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
37/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
1970 1975 1980 1985 1990 1995 2000 20050
1
2
3
4
5
6
7
8
9
10
Year
NormalizedReferenceVolume
Deconvolution
Blind Deconvolution
Figure 11: Recent scientific trends on deconvolution and blind deconvolution (Datasource: ISI web of knowledge).
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 17 of 42
Breaking the limitations-ThinBlinDe software
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
38/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 12: Observed volume section of a convallaria and restoration using the
ThinBlinDe software. cINRA and Ariana-INRIA/CNRS/UNSP. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 18 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
39/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
40/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
41/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
The equivalent energy function is
J(o|i)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
42/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
The equivalent energy function is
J(o|i) = log(Pr(o|i))
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
43/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
The equivalent energy function is
J(o|i) = log(Pr(o|i)) = Jobs(i|o) +Jreg(o)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
44/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
The equivalent energy function is
J(o|i) = log(Pr(o|i)) = Jobs(i|o) +Jreg(o)
Maximum a posteriori (MAP) estimate is obtained by
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
45/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
The equivalent energy function is
J(o|i) = log(Pr(o|i)) = Jobs(i|o) +Jreg(o)
Maximum a posteriori (MAP) estimate is obtained by
o(x) = arg maxo(x)0
Pr(o|i)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
Deconvolution as Bayesian inference
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
46/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
MicrospheresPlant samples
From the Bayes theorem, the posterior probability is
Pr(o|i) Pr(i|o)Pr(o)
Pr(i|o) is the likelihood, Pr(o) is the belief of the object.
The equivalent energy function is
J(o|i) = log(Pr(o|i)) = Jobs(i|o) +Jreg(o)
Maximum a posteriori (MAP) estimate is obtained by
o(x) = arg maxo(x)0
Pr(o|i) = arg mino(x)0
( log(Pr(o|i)))
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 19 of 42
State of the art: deconvolution
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
47/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Assumption Method References
No Noise Nearest neighbors [Agard 84],Inverse filter [Erhardt et al. 85]
Additive Gaussian Reg. lin. least square [Preza et al. 92],white noise Wiener filter [Tommasi et al. 93],
JVC [Agard 84,
Carrington et al. 90],APEX [Carasso 99],
MAP estimate [Levin et al. 09]Poisson noise ML estimate [Holmes 88],
MAP estimate [Verveer et al. 99,
Dey et al. 06,Pankajakshan et al. 08]
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 20 of 42
Maximum likelihood object estimation
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
48/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Likelihood of the observation, i(x), knowing the specimen
o(x) is given as
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 21 of 42
Maximum likelihood object estimation
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
49/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Likelihood of the observation, i(x), knowing the specimen
o(x) is given as
Pr(i|o) (ho+ b)(x)i(x) exp((ho+ b)(x))
i(x)!
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 21 of 42
Maximum likelihood object estimation
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
50/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Likelihood of the observation, i(x), knowing the specimen
o(x
) is given as
Pr(i|o) (ho+ b)(x)i(x) exp((ho+ b)(x))
i(x)!
Find o(x) such that
o(x) = arg maxo(x)0
Pr(i|o)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 21 of 42
Maximum likelihood object estimation
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
51/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Likelihood of the observation, i(x), knowing the specimen
o(x
) is given as
Pr(i|o) (ho+ b)(x)i(x) exp((ho+ b)(x))
i(x)!
Find o(x) such that
o(x) = arg maxo(x)0
Pr(i|o)
Approach: Maximum likelihood expectation maximization(MLEM) algorithm [Richardson 72, Lucy 74, Dempster 77,Celeux 85].
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 21 of 42
Maximum likelihood object estimation
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
52/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-
Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Likelihood of the observation, i(x), knowing the specimen
o(x
) is given as
Pr(i|o) (ho+ b)(x)i(x) exp((ho+ b)(x))
i(x)!
Find o(x) such that
o(x) = arg maxo(x)0
Pr(i|o)
Approach: Maximum likelihood expectation maximization(MLEM) algorithm [Richardson 72, Lucy 74, Dempster 77,Celeux 85].
Assumption: h(x) is available!
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 21 of 42
Prior as statistical information
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
53/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Gibbsian distribution with total variation (TV) function is used
as prior on the object [Demoment 1989]
Figure 13: Markov random fieldover a six member neighborhood
x
for a voxel sitex
s.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 22 of 42
Prior as statistical information
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
54/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Gibbsian distribution with total variation (TV) function is used
as prior on the object [Demoment 1989]
Figure 13: Markov random fieldover a six member neighborhood
x
for a voxel sitex
s.
Pr(o) = Z1o exp(ox
s
|o(x)|),
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 22 of 42
Prior as statistical information
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
55/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Gibbsian distribution with total variation (TV) function is used
as prior on the object [Demoment 1989]
Figure 13: Markov random fieldover a six member neighborhoodx
for a voxel sitex
s.
Pr(o) = Z1o exp(ox
s
|o(x)|),
Zo =oO(s)
exp(oxs
|o(x)|) is
the partition function,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 22 of 42
Prior as statistical information
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
56/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Gibbsian distribution with total variation (TV) function is used
as prior on the object [Demoment 1989]
Figure 13: Markov random fieldover a six member neighborhoodx
for a voxel sitex
s.
Pr(o) = Z1o exp(ox
s
|o(x)|),
Zo =oO(s)
exp(oxs
|o(x)|) is
the partition function, with o 0.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 22 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
57/105
Maximum a posteriori object estimate
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
58/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Combining the likelihood and the prior terms
Pr(o,h|i) Pr(i|o,h)Pr(o)
this posterior probability can be written as
Pr(o,h|i) xs
((ho+ b)(x))i(x) exp((ho)(x))
i(x)!
exp(oxs
|o(x)|)
o
exp(oxs
|o(x)|)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 23 of 42
Gradient vector field
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
59/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples X
Y
50
100
150
200
250
Lateral direction
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 24 of 42
Gradient vector field
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
60/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples X
Y
50
100
150
200
250
X
Z
50
100
150
200
250
Lateral direction Axial direction
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 24 of 42
Gradient vector field
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
61/105
ThinBlinDe
P. Panka-
jakshan, L.Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples X
Y
50
100
150
200
250
X
Z
50
100
150
200
250
Lateral direction Axial direction
Figure 14: The gradient vector field is overlapped over a synthetic object. Thefield flow is more prominent along the borders and less along the homogeneousinteriors. The effect of the noise is negligible. cAriana-INRIA/CNRS/UNS.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 24 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
62/105
Incoherent scalar PSF model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
63/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 26 of 42
Incoherent scalar PSF model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
64/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
For the case when the acquisition parameters of the
experiments are exactly known, deconvolution can beachieved by using theoretically calculated PSFs.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 26 of 42
Incoherent scalar PSF model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
65/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
For the case when the acquisition parameters of the
experiments are exactly known, deconvolution can beachieved by using theoretically calculated PSFs.
Computation of the incoherent PSF can be reduced to 2Nz2D Fourier transform of the pupil function
hTh(x;ex,em) = C|hA(x;ex)| |AR(x,y)hA(x,y,z;em)|
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 26 of 42
Incoherent scalar PSF model
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
66/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
For the case when the acquisition parameters of the
experiments are exactly known, deconvolution can beachieved by using theoretically calculated PSFs.
Computation of the incoherent PSF can be reduced to 2Nz2D Fourier transform of the pupil function
hTh(x;ex,em) = C|hA(x;ex)| |AR(x,y)hA(x,y,z;em)|
IfP(kx,ky,z) is the 2D complex pupil function and is thewavelength, the amplitude PSF is
hA(x,y,z;) =
kx
ky
P(kx,ky,z)exp(j(kxx+ kyy))dkydkx
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 26 of 42
Numerically computed PSF
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
67/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 16: Numerically calculated PSF for a C-Apochromat water immersionobjective. NA 1.2, 63 magnification. cAriana-INRIA/CNRS/UNS.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 27 of 42
State of the art: Blind deconvolution
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
68/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Method References
A priori PSF [Boutet de Monvel 2001],identification
Marginalization [Jalobeanu 2007,
Levin et al. 2009],Joint maximum likelihood [Holmes 1992,
Michailovich & Adam 2007],
Parametric blind deconvolution [Markham& Conchello 1999].
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 28 of 42
Thi Bli D
Blind deconvolution by alternate minimization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
69/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
When the imaging parameters are not known, it isnecessary to estimate o(x) and h(x) from i(x)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 29 of 42
Thi Bli D
Blind deconvolution by alternate minimization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
70/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
When the imaging parameters are not known, it isnecessary to estimate o(x) and h(x) from i(x)
Minimizing the cost function w.r.t o(x)
o(x) = arg mino(x)0
J(o(x)|o,h(x))
= arg mino(x)0Jobs(i(x
)|o(x
),h(x
)) +Jreg(o(x
)|o)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 29 of 42
ThinBlinDe
Blind deconvolution by alternate minimization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
71/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
When the imaging parameters are not known, it isnecessary to estimate o(x) and h(x) from i(x)
Minimizing the cost function w.r.t o(x)
o(x) = arg mino(x)0
J(o(x)|o,h(x))
= arg mino(x)0Jobs(i(x
)|o(x
),h(x
)) +Jreg(o(x
)|o)
Minimizing the cost function w.r.t h(x)
h(x) = arg minh(x)0
J(h(x)|o,o)
= arg minh(x)0
Jobs(i(x)|o,h(x)) +Jreg(h(x))
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 29 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
72/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 30 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
73/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
The estimation of the complete PSF from the observationis a difficult problem as the number of unknowns are large.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 30 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
74/105
ThinBlinDe
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
The estimation of the complete PSF from the observationis a difficult problem as the number of unknowns are large.
Since the CLSM PSF is theoretically the Bessel functionraised to the fourth power, an approximation in the spatialdomain is a 3D Gaussian approximation,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 30 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
Th f h l PSF f h b
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
75/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
The estimation of the complete PSF from the observationis a difficult problem as the number of unknowns are large.
Since the CLSM PSF is theoretically the Bessel functionraised to the fourth power, an approximation in the spatialdomain is a 3D Gaussian approximation,
Diffraction-limited PSF in the LS sense (up to 3AU pinhole
diameter)
h(x;h) = (2) 3
2 ||1
2 exp(1
2(x)T1(x))
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 30 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
Th i i f h l PSF f h b i
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
76/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
The estimation of the complete PSF from the observationis a difficult problem as the number of unknowns are large.
Since the CLSM PSF is theoretically the Bessel functionraised to the fourth power, an approximation in the spatialdomain is a 3D Gaussian approximation,
Diffraction-limited PSF in the LS sense (up to 3AU pinhole
diameter)
h(x;h) = (2) 3
2 ||1
2 exp(1
2(x)T1(x))
BD is reduced to estimation of spatial parameters,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 30 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
Th ti ti f th l t PSF f th b ti
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
77/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
The estimation of the complete PSF from the observationis a difficult problem as the number of unknowns are large.
Since the CLSM PSF is theoretically the Bessel functionraised to the fourth power, an approximation in the spatialdomain is a 3D Gaussian approximation,
Diffraction-limited PSF in the LS sense (up to 3AU pinhole
diameter)
h(x;h) = (2) 3
2 ||1
2 exp(1
2(x)T1(x))
BD is reduced to estimation of spatial parameters,
advantages: few parameters and no DFT necessary.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 30 of 42
ThinBlinDe
Why 3D Gaussian?
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
78/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Properties of the PSF
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 31 of 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
79/105
ThinBlinDe
Why 3D Gaussian?
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
80/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Properties of the PSF positivity:
h(x) 0, x s
circular symmetry:
h(x,y,z) = h(x,y,z)
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 31 of 42
ThinBlinDe
Why 3D Gaussian?
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
81/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Properties of the PSF
positivity:
h(x) 0, x s
circular symmetry:
h(x,y,z) = h(x,y,z)
normalization:
xs
h(x) = 1
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 31 of 42
ThinBlinDe
Why 3D Gaussian?
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
82/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Properties of the PSF
positivity:
h(x) 0, x s
circular symmetry:
h(x,y,z) = h(x,y,z)
normalization:
xs
h(x) = 1
CLSM PSF is theoretically the Besselfunction raised to the fourth power.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 31 of 42
ThinBlinDe
Why 3D Gaussian?
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
83/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Properties of the PSF
positivity:
h(x) 0, x s
circular symmetry:
h(x,y,z) = h(x,y,z)
normalization:
xs
h(x) = 1
CLSM PSF is theoretically the Besselfunction raised to the fourth power.
Figure 17: The Besselfunction raised to the fourthpower loses its side lobes.
cAriana-INRIA/CNRS/UNS.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 31 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
84/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 32 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
In the presence of aberrations, spatial approximation leads
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
85/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
In the presence of aberrations, spatial approximation leadsto a large number of parameters,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 32 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
In the presence of aberrations, spatial approximation leads
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
86/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
t e p ese ce o abe at o s, spat a app o at o eadsto a large number of parameters,
a way of handling it is estimating the parameters of thepupil function h = d,ni,ns
Pa(;x) = Pd(;x)exp(j(d,ni,ns))
P Pankajakshan L Blanc-Fraud J Zerubia November 5 2010 page 32 of 42
ThinBlinDe
Blind deconvolution by PSF parametrization
In the presence of aberrations, spatial approximation leads
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
87/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
p , p ppto a large number of parameters,
a way of handling it is estimating the parameters of thepupil function h = d,ni,ns
Pa(;x) = Pd(;x)exp(j(d,ni,ns))
regularization of the PSF is achieved by using a functionalform of phase from geometrical optics
(d,ni,ns) = d(nicosins coss) dns secs
P Pankajakshan L Blanc-Fraud J Zerubia November 5 2010 page 32 of 42
ThinBlinDe
PSF estimation
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
88/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
P Pankajakshan L Blanc-Fraud J Zerubia November 5 2010 page 33 of 42
ThinBlinDe
PSF estimation
Deconvolution requires the PSF h(x) or h(x, h)
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
89/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
( ) ( )
h,MAP = argmaxh>0
Pr(i(x)|o(x),h)Pr(h)
P Pankajakshan L Blanc-Fraud J Zerubia November 5 2010 page 33 of 42
ThinBlinDe
PSF estimation
Deconvolution requires the PSF h(x) or h(x, h)
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
90/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
h,MAP = argmaxh>0
Pr(i(x)|o(x),h)Pr(h)
Pr(h) is the parameter prior. We assume the parametersto be uniformly distributed in a set: [h,LB,h,UB],
P Pankajakshan L Blanc-Fraud J Zerubia November 5 2010 page 33 of 42
ThinBlinDe
P P k
PSF estimation
Deconvolution requires the PSF h(x) or h(x, h)
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
91/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
h,MAP = argmaxh>0
Pr(i(x)|o(x),h)Pr(h)
Pr(h) is the parameter prior. We assume the parametersto be uniformly distributed in a set: [h,LB,h,UB],
and the cost function is
J(o,h(h)) =x
i(x)log((h(h)o+ b)(x))
x
(h(h)o+ b)(x)
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 33 of 42
ThinBlinDe
P P k
PSF estimation
Deconvolution requires the PSF h(x) or h(x, h)
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
92/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
h,MAP = argmaxh>0
Pr(i(x)|o(x),h)Pr(h)
Pr(h) is the parameter prior. We assume the parametersto be uniformly distributed in a set: [h,LB,h,UB],
and the cost function is
J(o,h(h)) =x
i(x)log((h(h)o+ b)(x))
x
(h(h)o+ b)(x)
The cost function, J(o,h(h)), could be minimized byusing a gradient-descent type algorithm.
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 33 of 42
ThinBlinDe
P Panka
Simulation results
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
93/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Phantom object Observation = 100,xy = 25nm z = 50nm
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 34 of 42
ThinBlinDe
P Panka
Simulation results
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
94/105
P. Panka-jakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Sans regularization (Naive MLEM) Proposed approach
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 35 of 42
ThinBlinDe
P Panka-
Microsphere deconvolution
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
95/105
P. Pankajakshan, L.
Blanc-Fraud, J.
Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 18: A 15m microsphere has a thin layer of fluorescence of thickness500700nm.
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 36 of 42
ThinBlinDe
P. Panka-
Microsphere deconvolution
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
96/105
P. Pankajakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Observed section BD restored (535.58 nm)
Figure 19: Measured thickness of a microsphere, after using our approach, was535.58nm, and was 260nm with the naive MLEM algorithm.cAriana-INRIA/CNRS/UNS.
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 37 of 42
ThinBlinDe
P. Panka-
Deconvolution on plant samples
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
97/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 20: Observed section of arabidopsis thaliana in water and observed with aLSM 510 microscope and pinhole 2AU. Lateral sampling is 285.64nm and axialsampling is 845.62nm. cINRA.
P Pankajakshan L Blanc Fraud J Zerubia November 5 2010 page 38 of 42
ThinBlinDe
P. Panka-
Deconvolution on plant samples
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
98/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 21: Restoration sans regularization (Naive MLEM).cAriana-INRIA/CNRS/UNS.
P Pankajakshan L Blanc Fra d J Zer bia November 5 2010 page 39 of 42
ThinBlinDe
P. Panka-
Deconvolution on plant samples
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
99/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 22: Restoration using the proposed approach after 10 iterations.cAriana-INRIA/CNRS/UNS.
P P k j k h L Bl F d J Z bi N b 5 2010 40 f 42
ThinBlinDe
P. Panka-
Deconvolution on plant samples
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
100/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
Figure 23: Comparison of observed and restored lateral section of convallariamajalis. Lateral sampling is 53.6nm and axial sampling is 129.4nm. The samplewas observed with a Zeiss LSM 510TMmicroscope, and 1.3NA oil immersionobjective. cINRA, Ariana-INRIA/CNRS/UNS.
P P k j k h L Bl F d J Z bi N b 5 2010 41 f 42
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
101/105
ThinBlinDe
P. Panka-j k h L
Conclusions and discussions
Developed Bayesian framework for blind deconvolution forthin specimens
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
102/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
thin specimens,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 42 of 42
ThinBlinDe
P. Panka-jakshan L
Conclusions and discussions
Developed Bayesian framework for blind deconvolution forthin specimens
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
103/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
thin specimens,
experiments on simulated data, microsphere images andreal data show promising results,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 42 of 42
ThinBlinDe
P. Panka-jakshan L
Conclusions and discussions
Developed Bayesian framework for blind deconvolution forthin specimens
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
104/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
thin specimens,
experiments on simulated data, microsphere images andreal data show promising results,
PSF model chosen initially is a 3D separable Gaussianfunction for thin specimens,
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 42 of 42
ThinBlinDe
P. Panka-jakshan L
Conclusions and discussions
Developed Bayesian framework for blind deconvolution forthin specimens,
-
8/8/2019 ThinBlinDe: Blind deconvolution for confocal microscopy
105/105
jakshan, L.
Blanc-Fraud, J.Zerubia
Road map
Introduction
OSM
Limitations
ThinBlinDe
Break the limit
Blind
deconvolution
Results
Phantom data
Microspheres
Plant samples
thin specimens,
experiments on simulated data, microsphere images andreal data show promising results,
PSF model chosen initially is a 3D separable Gaussianfunction for thin specimens,
Total variation regularization functional sometimes leads toloss in contrast in restoration but there are methods toovercome this.
P. Pankajakshan, L. Blanc-Fraud, J. Zerubia November 5, 2010 page 42 of 42