TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global...
Transcript of TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global...
![Page 1: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/1.jpg)
TVL1 Models for Imaging:Global Optimization & Geometric Properties
Part I
Tony F. Chan
Math Dept, UCLA
S. Esedoglu
Math Dept, Univ. Michigan
Other Collaborators:
J.F. Aujol & M. Nikolova (ENS Cachan), F. Park, X. Bresson (UCLA)
Research supported by NSF, ONR, and NIH.
Papers: www.math.ucla.edu/applied/cam/index.html
Research group: www.math.ucla.edu/~imagers
![Page 2: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/2.jpg)
2
![Page 3: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/3.jpg)
3
Total Variation & Geometric Regularization
∫Ω
∇= dxuuTV ||)(
• Measures “variation” of u, w/o penalizing discontinuities.
•1D: If u is monotonic in [a,b], then TV(u) = |u(b) – u(a)|, regardless of whether u is discontinuous or not.
• nD: If u(D) = c χ(D), then TV(u) = c |∂D|.
• (Coarea formula)
•Thus TV controls both size of jumps and geometry of boundaries.
drdsfdxufnR
ru
)(||
∫ ∫ ∫+∞
∞− =
=∇
![Page 4: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/4.jpg)
4
Total Variation Restoration
2||||2
1)()(min zKuuTVuf
u−+= α
0=∂∂n
uGradient flow:
)(||
)( *zKKuKu
uugut −−
∇∇⋅∇=−= ∗α
anisotropic diffusion data fidelity
∫Ω
∇= dxuuTV ||)(
* First proposed by Rudin-Osher-Fatemi ’92.
* Allows for edge capturing (discontinuities along curves).
* TVD schemes popular for shock capturing.
Regularization:
Variational Model:
![Page 5: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/5.jpg)
5
TV-L2 and TV-L1 Image Models
Rudin-Osher-Fatemi: Minimize for a given image :),( yxf
TV-L1 Model:
Discrete versions previously studied by: Alliney’96 in 1-D and Nikolova’02 in higher dimensions, and E. Cheon, A. Paranjpye, and L. Vese’02.
Is this a big deal?
Other successful uses of L1: Robust statistics; l1 as convexification of l0 (Donoho), TV Wavelet Inpainting (C-Shen-Zhou), Compressive Sensing (Candes, Donoho, Romberg, Tao),
Model is convex, with unique global minimizer.
Model is non-strictly convex; global minimizer not unique.
![Page 6: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/6.jpg)
6
Surprising Features of TV+L1 Model
• Contrast preservation
• Data driven scale selection
• Cleaner multiscale decompositions
• Intrinsic geometric properties provide a way to solve non-convex shape optimization problems via convex optimization methods.
![Page 7: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/7.jpg)
7
Contrast Loss of ROF Model• Theorem (Strong-C 96): If f = 1Br(0), ∆ = BR(0), 0 < r < R;
• Locates edges exactly (robust to small noise).
• Contrast loss proportional to scale-1:
• Theorem (Bellettini, Caselles, Novaga 02): If f = 1Ω, Ω convex, ∂ Ω is C1,1, and for every p on ∂ Ω
Then
![Page 8: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/8.jpg)
8
TV-L1: Contrast & Geometry Preservation
Contrast invariance: If u(x) is the solution for given image f(x), then cu(x) is the solution for cf(x).
Contrast & Geometry Preservation: Let , where is a bounded domain with smooth boundary. Then, for large enough λ, the unique minimizer of E1(¢,λ) is exactly .
The model recovers such images exactly. Not true for standard ROF.
Ω
u=1
u=0
)(1)( xxf Ω= Ω
(Other method to recover contrast loss: Bregman iteration (Osher et al))
)(xf
![Page 9: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/9.jpg)
9
![Page 10: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/10.jpg)
10
![Page 11: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/11.jpg)
11
“Scale-space” generated by the original ROF model
Decreasing λ
![Page 12: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/12.jpg)
12
“Scale-space” generated by the model1L-TV
![Page 13: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/13.jpg)
13
Plots of vs. )()( xfxu −λ1−λ
Discontinuities of fidelity correspond to removal of a feature (one of the squares).
TVL2 TVL1
Data Dependent Scale Selection
![Page 14: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/14.jpg)
14
Multiscale Image Decomposition Example
TVL1 decomposition gives well separated & contrast preserving features at different scales. E.g. boat masts, foreground boat appear mostly in only 1 scale.
(related: Tadmor, Nezzar, Vese 03; Kunisch-Scherzer 03)
![Page 15: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/15.jpg)
15
Given a binary observed image find a denoised (regularized) version.
Applications:
Denoising of fax documents (Osher, Kang).
Understanding many important image models: ROF, Mumford-Shah, Chan-Vese, etc.
Motivating Problem: Denoising of Binary Images
Convexification of Shape Optimization
![Page 16: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/16.jpg)
16
Restriction of ROF to Binary Images
Take f(x)=1Ω(x) and restrict minimization to set of binary images:
Considered previously by Osher & Kang, Osher & Vese. Equivalent
to the following non-convex geometry problem:
where denotes the symmetric difference of the sets S1 and S2. 21 SS ∆
Existence of solution for any bounded measurable Ω.
Global minimizer not unique in general. Many local minimizers possible.
![Page 17: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/17.jpg)
17
Example of Local Minima for Geometry Problem
Illustration of how algorithms get stuck:
![Page 18: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/18.jpg)
18
Global Minimum via TVL1 (C-Esedoglu-Nikolova ’04)
To find a solution (i.e. a global minimizer) u(x) of the non-convex variational problem (same as ROF for binary images):
it is sufficient to carry out the following steps:
• Find any minimizer of the convex TVL1 energy
Call the solution found v(x).
• Let for some .
Then Σ is a global minimizer of the original non-convex problem for almost every choice of µ.
µ>∈=Σ )(:R xvx N )1,0(∈µ
![Page 19: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/19.jpg)
19
For each upper level set of u(x), we have the same geometry problem:
)(1)( xxf Ω=
Connection of TVL1 Model to Shape Denoising
• Coarea formula:
• “Layer Cake” theorem:
When
![Page 20: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/20.jpg)
20 µ>)(: xux
µ>)(: xvx
µ
∫ ∫−>∆>=−
D
dxvxxuxdxvu µµµ1
1)(:)(:
Illustration of Layer-Cake Formula
![Page 21: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/21.jpg)
21
IllustrationEnergy
Solution u Space
Geometry Problem TVL1 Problem
)(xuλ
Global Minimizer
Function values agree on u binary
Non-convex Convex
u binary
![Page 22: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/22.jpg)
22
Intermediates, showing the evolution:Noisy Image
Intermediates non-binary! The convex TV-L1 model opens up new pathways to global minimizer in the energy landscape.
![Page 23: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/23.jpg)
23
What about other Lp norms?
Integrand depends on µ explicitly, and not only on the super level sets of f; so these terms are not purely geometric: Solving different geometric problems at different levels.
![Page 24: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/24.jpg)
24
Generalization to Image Segmentation
Chan-Vese Model (2001): Simplified Mumford-Shah: Best approximation of f(x) by two-valued functions:
Variational CV Segmentation Model:
Similar arguments as for shape denoising show CV is equivalent to:
Theorem: If (c1,c2,u(x)) is a solution of above formulation, then for a.e. µ in (0,1) the triplet:
is a global minimizer of the Chan-Vese model.
![Page 25: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/25.jpg)
25
UPSHOT: For fixed c1, c2 the inner minimization (i.e. the shape optimization) in our formulation is convex. Also, the constraint on u can be incorporated via exact penalty formulation into an unconstrained optimization problem:
where z(ξ) looks like:
Turns out: For γ large enough, minimizer u satisfies u(x) in [0,1] for all x in D.
Solve via gradient descent on Euler-Lagrange equation.
Incorporating the Constraint
![Page 26: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/26.jpg)
26
![Page 27: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/27.jpg)
27
Sample Computation:
Given image f(x)
u(x) computed
![Page 28: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/28.jpg)
Related and Further Works
![Page 29: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/29.jpg)
29
![Page 30: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/30.jpg)
30
![Page 31: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/31.jpg)
31
![Page 32: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/32.jpg)
32
![Page 33: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/33.jpg)
33
Continuous Max Flow/Min Cut (Bresson-C)
Application #1: [Strang] F=1, f=0
If C is a cut then the CMF is given by minimizing the isoparametric ratio:
[Strang 83] defined the continuous analogue to the discrete max flow. He replaced a flow on a discrete network by a vector field p. The continuous max flow (CMF) problem can be formulated as follows:
F,f are the sources and sinks, F,f are the sources and sinks, and w is the capacity constraintand w is the capacity constraint
Isoparametric ratioIsoparametric ratio
![Page 34: TVL1 Models for Imaging: Global Optimization & Geometric ...popov/L12008/talks/Chan.pdfis a global minimizer of the Chan-Vese model. 25 UPSHOT: For fixed c1, c2 the inner minimization](https://reader034.fdocuments.us/reader034/viewer/2022042100/5e7c26122667384d762b5820/html5/thumbnails/34.jpg)
34
Application #3: [Bresson-Chan] F=gr, f=0Optimizing the CEN model corresponds to solve the CMF problem:
The previous CMF problem can be solved by the system of PDEs:
which comes from this energy:
which is the CEN energy!
Application #2: [Appleton-Talbot 06] F=0, f=0 (geodesic active contour)
The CMF is
It is a conservation flow (Kirchhoff’s law, flow in=flow out).
[AT] proposed to solve the CMF solving this system of PDEs:
Continuous Max Flow/Min Cut
We may notice that these PDEs We may notice that these PDEs come from this energy:come from this energy:
(Weighted TV Norm)(Weighted TV Norm)