Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer...
-
Upload
julian-green -
Category
Documents
-
view
224 -
download
0
Transcript of Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer...
![Page 1: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/1.jpg)
Active Contours / Planes
Sebastian Thrun, Gary Bradski, Daniel RussakoffStanford CS223B Computer Vision
http://robots.stanford.edu/cs223b
Some slides taken from: Julien Jomier
![Page 2: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/2.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Outline
Snakes Planes
![Page 3: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/3.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Introduction (1)
The active contour model, or snake, is defined as an energy-minimizing spline.
Active contours results from work of Kass et.al. in 1987.
Active contour models may be used in image segmentation and understanding.
The snake’s energy depends on its shape and location within the image.
Snakes can be closed or open
![Page 4: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/4.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Example
![Page 5: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/5.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Introduction (2)
Aorta segmentation using active contours
![Page 6: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/6.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Introduction (3)
First an initial spline (snake) is placed on the image, and then its energy is minimized.
Local minima of this energy correspond to desired image properties.
Unlike most other image models, the snake is active, always minimizing its energy functional, therefore exhibiting dynamic behavior.
Also suitable for analysis of dynamic data or 3D image data.
![Page 7: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/7.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Kass Algorithm
The snake is defined parametrically as v(s)=[x(s),y(s)], where s[0,1] is the normalized arc length along the contour. The energy functional to be minimized may be written as
Econt = snake continuity
Ecurv = snake curvature
Eimage = image forces (e.g., edge attraction)
10
10
10
10
*
))((
))(())((
))((
dssvE
dssvEdssvE
dssvEE
image
curvcont
snakesnake
![Page 8: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/8.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Internal Energy
Continuity:
Curvature:
2
))((ds
dvsvEcont
2
2
2
))((ds
vdsvEcurv
![Page 9: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/9.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Image Forces
Dark/Bright Lines
Edges
))(())(( svIsvEimage
))(())(( svIsvEimage
![Page 10: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/10.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Trade-offs
determine trade-off
10
10
10
10
*
))((
))(())((
))((
dssvE
dssvEdssvE
dssvEE
image
curvcont
snakesnake
![Page 11: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/11.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Numerical Algorithm
Select N initial locations p1,…, pN Update until convergence
|)(|
|2|
||
1.0 211
21
k
kkk
kk
kiii
pI
ppp
pp
ppp
|)(||2|||2
112
1*
kk
kkkkksnake pIpppppE
![Page 12: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/12.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Snakes, Done Right
Define Spline over p1,…, pN Optimize criterion for all points on spline Allow for corners
(optimization becomes tricky, fills entire literature)
![Page 13: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/13.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Applications
Main Applications are: Segmentation Tracking Registration
http://www.markschulze.net/snakes/
![Page 14: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/14.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Examples (1)
Julien Jomier
![Page 15: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/15.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Examples (2)
Julien Jomier
![Page 16: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/16.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Examples (3)
Heart Julien Jomier
![Page 17: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/17.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Examples (4)
3D Segmentation of the Hippocampus
Julien Jomier
![Page 18: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/18.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Examples (5)
Julien Jomier
![Page 19: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/19.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Example (6)
Julien Jomier
![Page 20: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/20.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Example (7)
Julien Jomier
![Page 21: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/21.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Problems with snakes
Snakes sometimes degenerate in shape by shrinking and flattening.
Stability and convergence of the contour deformation process unpredictable.
Solution: Add some constraints
Initialization is not straightforward.
Solution: Manual, Learned, Exhaustive
![Page 22: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/22.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
References
M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. In Proc. 1st ICCV, pages 259-268, June 1987. London, UK.
Yongjik Kim. A summary of Implicit Snake Formulation.
Jorgen Ahlberg. Active Contours in Three Dimensions.
M. Bertalmio, G. Sapiro and G. Randall. Morphing Active Contours. IEEE PAMI, Vol 22, No 7, July 2000
![Page 23: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/23.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Outline
Snakes Planes
![Page 24: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/24.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Finding Planes
Lines in Image (e.g., Hough transform) Planes in 3D space (e.g., stereo reconstruction,
SFM) Problems:
– Number of planes– Parameters of planes– Data association: which point belongs to which plane– Outlier removal (noise, non-flat surfaces)
![Page 25: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/25.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Range Data in Multi-Planar Environment
Taken with Laser Range Finder, but similar for
![Page 26: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/26.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Basic Idea
******
![Page 27: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/27.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Mathematical Model: Expectation Maximization
3D Model: },,,{ 21 J
3, jjj Planar surface in 3D
ijij zz ),dist(
Distance point-surfacesurface
surface normal
y
x
z
displacement
![Page 28: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/28.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Mixture Measurement Model
Case 1: Measurement zi caused by plane j
2
2)(
2
1
22
1)|(
jij z
ji ezp
2
2max
2ln
2
1
2max
*2
11)|(
z
i ez
zp
Case 2: Measurement zi caused by something else
![Page 29: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/29.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Measurement Model with Correspondences
J
j
jijj
zc
zc
Ji eccczp 12
2
2
2max
*
)(
2ln
2
1
2*12
1),,,,|(
correspondence variables C:
}
J
jj
j
cc
cc
1*
*
1
}1,0{,
1
)(
2ln
2
1
2
12
2
2
2max
*
2
1),|(
i
zc
zc
J
j
jijiji
eCZp
![Page 30: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/30.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Expected Log-Likelihood Function
1
)(
2ln
2
1
2
12
2
2
2max
*
2
1),|(
i
zc
zc
J
j
jijiji
eCZp
i
J
j
jijij
ic
zcE
zcE
J
CZpE
12
2
2
2max
*
2
)(][
2
1
2ln][
2
1
2)1(
1ln
)|,(ln
…after some simple math
mapping with known data association
probabilistic data association
![Page 31: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/31.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
E-Step
Calculate expectations with fixed model :
(normalize so that )
2
2
2
2max
)(
2
1
2ln
2
1
*
][
][
jij z
ij
z
i
ecE
ecE
J
jiji cEcE
1* 1][][
![Page 32: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/32.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
M-Step
Maximize
i
jijij
ic
zcE
zcE
J
CZpE
2
2
2
2max
*
2
)(][
2
1
2ln][
2
1
2)1(
1ln
)|,(ln
1 jj subject to
Is equivalent to minimizing
i
J
jjijij zcE
1
2)]([ 1 jj subject to
![Page 33: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/33.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
M-Step: Solve via Lagrange Multipliers
i
J
jjijij zcE
1
2 min)]([ 1: jjj
i
J
j
J
jjjjjijij zcEL
1 1
2)]([
Define
0)]([!
jijijj
zcEL
0)]([!
jjijijijj
zzcEL
1 jj
And observe that
![Page 34: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/34.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
M-Step: Solve via Lagrange Multipliers
Solve for :
Substitute back:
Is of the linear form:
Solution: two Eigenvectors of A with smallest Eigenvalues.
kjkj
kkjkj
j cE
zcE
][
][0)]([
!
jijijj
zcEL
0)][
][]([
!
jji
kjkj
kkjkj
ijijj
zcE
zcEzcE
L
0)]([!
jjijijijj
zzcEL
jjjA
![Page 35: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/35.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Determining Number of Surfaces
J =1
First model component
*
*
J =1
E-Step
*
*
J =1
M-Step
*
*
J =3
Add model components
J =3
E-Step
J =3
M-step
J =1
Prune model
J =3
Add model components
J =3
E/M Steps
*J =2
Prune model
![Page 36: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/36.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Choosing the “Right” Number of Planes: AIC
J=2 J=3 J=5J=0 J=1 J=4
increased data likelihood
increased prior probability
)(log)|(log)|(log JpJdpconstdJp
![Page 37: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/37.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Model Selection
Approximately every 20 iterations of EM: Start new surfaces
– Near any set of collinear measurements Terminate unsupported surfaces
– If not supported by enough measurements– If density of measurements too low– If two planes are too close to each other
![Page 38: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/38.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Results
![Page 39: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/39.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Online Robotic Mapping @ CMU
![Page 40: Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision Some slides.](https://reader030.fdocuments.us/reader030/viewer/2022032723/56649d1c5503460f949f1850/html5/thumbnails/40.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Online Robotic Mapping @ Stanford