Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation...
Transcript of Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation...
![Page 1: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/1.jpg)
Snakes, Strings, Balloons and Other Active Contour Models
![Page 2: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/2.jpg)
Goal
• Start with image and initial closed curve
• Evolve curve to lie along “important” features – Edges
– Corners
– Detected features
– User input
![Page 3: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/3.jpg)
Applications
• Region selection in Photoshop
• Segmentation of medical images
• Tracking
![Page 4: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/4.jpg)
Corpus Callosum
[Davatzikos and Prince]
![Page 5: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/5.jpg)
Corpus Callosum
[Davatzikos and Prince]
![Page 6: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/6.jpg)
User-Visible Options
• Initialization: user-specified, automatic
• Curve properties: continuity, smoothness
• Image features: intensity, edges, corners, …
• Other forces: hard constraints, springs, attractors, repulsors, …
• Scale: local, multiresolution, global
![Page 7: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/7.jpg)
Behind-the-Scenes Options
• Framework: energy minimization, forces acting on curve
• Curve representation: ideal curve, sampled, spline, implicit function
• Evolution method: calculus of variations, numerical differential equations, local search
![Page 8: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/8.jpg)
Snakes: Active Contour Models
• Introduced by Kass, Witkin, and Terzopoulos
• Framework: energy minimization – Bending and stretching curve = more energy
– Good features = less energy
– Curve evolves to minimize energy
• Also “Deformable Contours”
![Page 9: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/9.jpg)
Snakes Energy Equation
• Parametric representation of curve
• Energy functional consists of three terms
( ))(),()( sysxs =v
( ) ( ) ( )[ ]dssss conimg∫ ++= )()()(int vvv εεεε
![Page 10: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/10.jpg)
Internal Energy
• First term is “membrane” term – minimum energy when curve minimizes length (“soap bubble”)
• Second term is “thin plate” term – minimum energy when curve is smooth
( ) ( ) 2)()()()()( 22int sssss sss vvv βαε +=
![Page 11: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/11.jpg)
Internal Energy
• Control α and β to vary between extremes
• Set β to 0 at a point to allow corner
• Set β to 0 everywhere to let curve follow sharp creases – “strings”
( ) ( ) 2)()()()()( 22int sssss sss vvv βαε +=
![Page 12: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/12.jpg)
Image Energy
• Variety of terms give different effects
• For example, minimizes energy at intensity Idesired
desiredimg IyxIw −⋅= ),(ε
![Page 13: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/13.jpg)
Edge Attraction
• Gradient-based:
• Laplacian-based:
• In both cases, can smooth with Gaussian
2),( yxIwimg ∇⋅−=ε
22 ),( yxIwimg ∇⋅=ε
![Page 14: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/14.jpg)
Corner Attraction
• Can use corner detector we saw last week
• Alternatively, let θ = tan-1 Iy / Ix and let n⊥be a unit vector perpendicular to the gradient. Then
⊥∂∂
⋅=nθε wimg
![Page 15: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/15.jpg)
Constraint Forces
• Spring
• Repulsion
2xv −⋅= kconε
2xv −=
kconε
![Page 16: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/16.jpg)
Evolving Curve
• Computing forces on v that locally minimize energy gives differential equation for v – Euler-Lagrange formula
• Discretize v: samples (xi, yi) – Approximate derivatives with finite differences
• Iterative numerical solver
02
2
=∂∂
+
∂∂
+
∂∂
vvdsd
vdsd εεε
![Page 17: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/17.jpg)
Other Curve Evolution Options
• Exact solution: calculus of variations
• Write equations directly in terms of forces, not energy
• Implicit equation solver
• Search neighborhood of each (xi, yi) for pixel that minimizes energy – Shah & Williams paper
![Page 18: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/18.jpg)
Variants on Snakes
• Balloons [Cohen 91] – Add inflation force
– Helps avoid getting stuck on small features
)(skFinfl n=
![Page 19: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/19.jpg)
Balloons
[Cohen 91] Snakes
Balloons
![Page 20: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/20.jpg)
Balloons
[Cohen 91]
![Page 21: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/21.jpg)
Other Energy or Force Terms
• Results of previously-run local algorithms – e.g., Canny edge detector output convolved
with Gaussian
• Automatically-evolved control points
• Others…
![Page 22: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/22.jpg)
Brain Cortex Segmentation
Add energy term for constant-color regions of a single color
Davatzikos and Prince
![Page 23: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/23.jpg)
Brain Cortex Segmentation
Davatzikos and Prince
![Page 24: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/24.jpg)
Brain Cortex Segmentation
Davatzikos and Prince
Find features and add constraints
![Page 25: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/25.jpg)
Brain Cortex Segmentation
Davatzikos and Prince
![Page 26: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/26.jpg)
Scale
• In the simplest snakes algorithm, image features only attract locally
• Greater region of attraction: smooth image – Curve might not follow high-frequency detail
• Multiresolution processing – Start with smoothed image to attract curve
– Finish with unsmoothed image to get details
• Heuristic for global minimum vs. local minima
![Page 27: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/27.jpg)
Diffusion-Based Methods
• Another way to attract curve to localized features: vector flow or diffusion methods
• Example: – Find edges using Canny
– For each point in entire image, compute distance to nearest edge
– Push curve along gradient of distance field
![Page 28: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/28.jpg)
Gradient Vector Fields
Xu and Prince
![Page 29: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/29.jpg)
Gradient Vector Fields
Xu and Prince
Simple Snake With Gradient Vector Field
![Page 30: Snakes, Strings, Balloons and Other Active Contour Models€¦ · Snakes Energy Equation •Parametric representation of curve •Energy functional consists of three terms v(s) =](https://reader033.fdocuments.us/reader033/viewer/2022051807/60040726aaa2f7004753c00c/html5/thumbnails/30.jpg)
Gradient Vector Fields
Xu and Prince