Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf ·...
Transcript of Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf ·...
![Page 1: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/1.jpg)
Martin Rumpf
lecture course at the summerschool
july 3rd – 7th, Evolène
Partial Differential Equationsin Image and Surface Processing
![Page 2: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/2.jpg)
motivation
imaging is geometry processing
geometry processing can benefit from imaging
![Page 3: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/3.jpg)
ReferencesSapiro, Geometric partial differential equations and image analysisCambridge University Press, 2001
Aubert, Kornprobst, Mathematical Problems in Image ProcessingSpringer, 2002
Osher, Fedkiw, Level Set Methods and dynamic implicit surfacesSpringer, 2003
Sethian, Level set methods and fast marching methods,Cambridge University Press, 1999
Alvarez, Guichard, Lions, Morel, Coll,Axioms and Fundamental Equations of Image ProcessingArch. Ration. Mech. Anal. 123, 199-257, 1993
![Page 4: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/4.jpg)
geometry images
[Gu, Gortler, Hoppe ´02]
parametrization
![Page 5: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/5.jpg)
surface fairing
isotropic denoising
anisotropic denoising
![Page 6: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/6.jpg)
3d ultra sound
denoising images
noisy initial data denoised image
anisotropicdenoising
![Page 7: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/7.jpg)
cartoon extraction based on anisotropic functionals
aerial imagesisotropic anisotropic
orientation
![Page 8: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/8.jpg)
surface / image restoration based on anisotropic area functionals
noise corruptedMR angiography
![Page 9: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/9.jpg)
surface restoration
![Page 10: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/10.jpg)
image restoration (inpainting)
![Page 11: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/11.jpg)
surface matching
?
![Page 12: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/12.jpg)
![Page 13: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/13.jpg)
![Page 14: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/14.jpg)
image matching
a variety of image modalities:
CT, MRI (T1,T2), FLAIR, PET, ....
CTMRT
Matching image morphologies
= Matching of image contours(edge surfaces and regular contours)
![Page 15: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/15.jpg)
matchedinitial mismatchwith edge set
PETCT
matching CT and PET images
![Page 16: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/16.jpg)
numerical relaxation
matching brains and cortical surfaces of two different patients
![Page 17: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/17.jpg)
explicit surfaces (notation)
![Page 18: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/18.jpg)
implicit surfaces (notation)
![Page 19: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/19.jpg)
Finite elements in
![Page 20: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/20.jpg)
Finite elements on explicit surfaces
![Page 21: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/21.jpg)
Finite elements on ensemble of level sets
![Page 22: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/22.jpg)
An axiomatic approach to scale space
[Alvarez, Guichard, Lions, Morel, Coll ´92]
[Rec][Trans][Comp][Loc]
[Reg]
[GS][G]
[Iso]
![Page 23: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/23.jpg)
surface fairing
anisotropic denoising
![Page 24: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/24.jpg)
noisy data timestep 1 timestep 2 timestep 7
denoising 3D images
![Page 25: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/25.jpg)
original image
anisotropic diffusion(Cf. [Weickert ´98])
anisotropic geometric diffusion
Perona Malikmodel
MCM
denoising 3D images
![Page 26: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/26.jpg)
3d ultra sound
denoising 3D images
noisy initial data denoised image
anisotropicdenoising
![Page 27: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/27.jpg)
morphological image denoising in 4D
![Page 28: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/28.jpg)
gradient descent1D
>1D
![Page 29: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/29.jpg)
infinite dimensional problems
![Page 30: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/30.jpg)
restoration of surfacesgradient descent:
Willmore flow
cf. [Ballester et al. 01],[Masnou, Morel ´98],[Mumford, Nitzberg ´96]
![Page 31: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/31.jpg)
![Page 32: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/32.jpg)
image restoration (inpainting)
![Page 33: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/33.jpg)
anisotropic energy functionals and gradient descent
[Belletini, Paolini´96]
![Page 34: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/34.jpg)
classification:
cf. [Esedoglu, Osher ´03]
curve smoothing
![Page 35: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/35.jpg)
anisotropic isotropic
classifikation:
surface fairing
![Page 36: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/36.jpg)
surface / image restoration based on anisotropic area functionals
noise corruptedMR angiography
![Page 37: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/37.jpg)
cartoon extraction based on anisotropic functionals
aerial imagesisotropic anisotropic
orientation
![Page 38: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/38.jpg)
matching surfaces
![Page 39: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/39.jpg)
physical interpretation
(tangential distortion) (normal bending)
(matching of feature sets)
![Page 40: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/40.jpg)
![Page 41: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/41.jpg)
deformation on the parameter domain and on the surface
![Page 42: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/42.jpg)
![Page 43: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/43.jpg)
matching regular contour surfaces
level sets
![Page 44: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/44.jpg)
[Droske, R.´03]
![Page 45: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/45.jpg)
mismatch
FLAIR
drawback
MR
![Page 46: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/46.jpg)
matching image morphologies
matching singular edge surfacesand regular contour surfaces
in explicit:
![Page 47: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/47.jpg)
matching edge sets via a level set approach
[Droske, Ring ´05]
![Page 48: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/48.jpg)
[Mumford, Shah ´86 ]
recall (free discontinuity problem):
phase field approximation [Ambrosio, Tortorelli ´91]
![Page 49: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/49.jpg)
singular morphology matching via a phase field approach
[Droske´05], [Droske, Ring, R.´05]
![Page 50: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/50.jpg)
matchedinitial mismatchwith edge set
PETCT
matching shapes of CT and PET images
![Page 51: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/51.jpg)
[Berkels, Droske, Han, Hornegger, R. ´06]
matching cortical surfaces
MRI
![Page 52: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/52.jpg)
numerical relaxation
matching brains and cortical surfaces of two different patients
![Page 53: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/53.jpg)
matching edge sets and regular contour sets
= matching singular and regular morphology
[Droske, R. ´05]
![Page 54: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/54.jpg)
initial mismatch
![Page 55: Partial Differential Equations in Image and Surface Processingjhorak/evolene/Rumpf_pres.pdf · Osher, Fedkiw, Level Set Methods and dynamic implicit surfaces Springer, 2003 Sethian,](https://reader034.fdocuments.us/reader034/viewer/2022050104/5f42e2052bf2eb6bc1250bbc/html5/thumbnails/55.jpg)
MRI T1 FLAIR initial mismatch
initial final