Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... •...
Transcript of Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... •...
![Page 1: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/1.jpg)
Spatio-Temporal Shape Modeling and Analysis
Guido Gerig
James Fishbaugh, Marcel Prastawa
Scientific Computing and Imaging Institute, University of Utah
Martin Styner, Stephen Pizer, UNC
Stanley Durrleman, Xavier Pennec, INRIA
![Page 2: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/2.jpg)
![Page 3: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/3.jpg)
Content
• Motivation Longitudinal Modeling
• Overview Correspondence
• Currents: Correspondence-free Modeling
• Driving Applications:
• Early Brain Development in Autism
• Huntington’s Disease (HD)
• Mandibular Growth
• Shape Analysis via Grid Deformations
• Growth Trajectories via Time Warp
![Page 4: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/4.jpg)
WM Maturation
Longitudinal/Serial Image Data
Neo 1yr 2yr
Pediatrics: Brain Growth Aging / Neurodegeneration
Trauma: Baseline – Follow-up
Tumor Growth
• Image analysis technology for 4D data is lagging behind acquisition
• Often: individual time-point analysis, ignores causality
Jaw Growth 15-22 yrs
![Page 5: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/5.jpg)
Spatiotemporal Modeling: Natural Task in Clinical Reasoning
Motivation:
Development, degeneration, effects of therapeutic intervention are dynamic processes.
Personalized health care: Individual trajectories compared to expected “norm”.
Clinical terminology: Atypical, Monitoring
Departure from typical development, deviation from healthy
Typical but delayed growth patterns, catch-up, atypical development
Analysis of recovery for each patient
Predict onset of clinical symptoms
Monitor efficacy of treatment
→ Focus on longitudinal design & longitudinal analysis
![Page 6: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/6.jpg)
Shape and Registration
D’Arcy Wentworth Thompson, On Growth and Form (1917, mathematics and biology)
![Page 7: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/7.jpg)
Why Shape?
![Page 9: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/9.jpg)
Shape Similarity in Twins
Upper row: identical twin pairs
Lower row: non-identical twin pairs
Styner/Gerig PNAS 2005
![Page 10: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/10.jpg)
Shape >> Volume
![Page 11: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/11.jpg)
Example Infant Study: Cross-sectional vs. Longitudinal
A: age 2 A: age 4 B: age 2 B: age 4
Cross-sectional: Huge changes between sets of shapes
Longitudinal: Subtle changes of sets of shapes with time
![Page 12: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/12.jpg)
Content
• Motivation Longitudinal Modeling
• Overview Correspondence
• Currents: Correspondence-free modeling
• Driving Applications: • Early Brain Development in Autism
• Huntington’s Disease (HD)
• Mandibular Growth
• Shape Analysis via Grid Deformations
• Growth Trajectories via Time Warp
![Page 13: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/13.jpg)
Shape and Registration
Ch. G. Small, The Statistical Theory of Shape
Homology:
Corresponding
(homologous)
features in all
skull images.
![Page 14: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/14.jpg)
Geometric Correspondence: Shapes
PDM Model
(Cootes/Taylor):
• Point to point
correspondence for
shape modeling
• PCA analysis
• Major eigenmodes
of shape variability
![Page 15: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/15.jpg)
Correspondence through spherical
harmonic surface representations
Normalization using first order ellipsoid:
•Rotation of parameter space to align major axis
•Spatial alignment to major axes
Parameters rotated to
first order ellipsoids
![Page 16: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/16.jpg)
Optimizing 3D Correspondences
Rhodri Davies, Carole Twining, Tim Cootes, Danny Allen, Chris Taylor: MDL Model Szekely/Brechbuehler/Styner/Gerig, SPHARM
![Page 17: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/17.jpg)
Optimization of Correspondence in 3D and over time: Particle Systems
2 entropy terms:
• Surface entropy -> distribute the particles “evenly” across surfaces.
• Ensemble entropy -> how are particles similar across a set of surfaces.
Particle system correspondence (Cates, Whitaker et al., Utah, Styner et al., UNC): Minimization of Entropy:
![Page 18: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/18.jpg)
Particle Correspondence Model C
onfigura
tion S
pace
S
hape S
pace
Accurate Representation (in Configuration Space)
vs.
Compact Model (in Shape Space)
Surface Entropy Ensemble Entropy
[Cates et al. IPMI 2007]
![Page 19: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/19.jpg)
Modeling Head Shape Change
Link to Movie
Changes in head size with age
Changes in head shape with age
Datar, Cates, Fletcher, Gouttard, Gerig, Whitaker, Particle-based Shape Regression, MICCAI 2009
![Page 20: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/20.jpg)
Medial Axis / Skeletal Representations: Intrinsic Shape Model
M-rep: Pizer et al.
(discrete)
CM-rep: Yushkevich
(continuous, parametric)
Gorczowski et al., T-PMI 2010,
Stats on deformations vs. thickness
![Page 21: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/21.jpg)
Graph Spectra/Laplacian
Courtesy Hervé Lombaert, Jon Sporring, Kaleem Siddiqi, Mc Gill, IPMI 2013
![Page 22: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/22.jpg)
Graph Spectra/Laplacian
Courtesy Hervé Lombaert, Jon Sporring, Kaleem Siddiqi, Mc Gill, IPMI 2013
![Page 23: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/23.jpg)
Discussion Correspondence
• Automatic optimization of correspondences efficient and robust for simple 3D shapes.
• PDM and Entropy Methods require fixed topology: Time-consuming pre-processing.
• Challenges: – thin, spiky objects
– gaps, missing parts
![Page 24: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/24.jpg)
Content
• Motivation Longitudinal Modeling
• Overview Correspondence
• Currents: Correspondence-free Modeling
• Driving Applications: • Early Brain Development in Autism
• Huntington’s Disease (HD)
• Mandibular Growth
• Shape Analysis via Grid Deformations
• Growth Trajectories via Time Warp
![Page 25: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/25.jpg)
Problem: Correspondences between Shapes with Variable Topology
Brain ventricles for infants 6mo to 2yrs DTI Fiber Tracts from two subjects
movie
Durrleman, Pennec, Ayache et al.
![Page 26: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/26.jpg)
“Correspondence-free” Registration: Currents
Topology and shape differences and noise can make point-to-point correspondence hard:
Currents: Objects that integrate vector fields
Shape: Oriented points = Set of normals (tangents)
Distance between curves:
[Glaunes2004] Glaunes, J., Trouve, A., Younes, L. Diffeomorphic matching of distributions: a new approach, … CVPR 2004.
[Durrleman2008] S. Durrleman, X. Pennec, A. Trouvé, P. Thompson, N. Ayache, Inferring Brain Variability from Diffeomorphic
Deformations of Currents: an integrative approach, Medical Image Analysis 2008
![Page 27: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/27.jpg)
Currents integrate vector field:
•W: test space of vector fields (Hilbert space)
•W*: the space of continuous maps W->R
•W* includes smooth curves, polygonal lines, surfaces,
meshes.
[Vaillant and Glaunès IPMI’05, Glaunès PhD’06]
![Page 28: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/28.jpg)
Durrleman, PhD thesis, 2010
![Page 29: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/29.jpg)
Durrleman, PhD thesis, 2010
![Page 30: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/30.jpg)
Durrleman, PhD thesis, 2010
![Page 31: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/31.jpg)
Durrleman, PhD thesis, 2010
![Page 32: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/32.jpg)
Distance between shapes:
• No point correspondence
• No individual line correspondence
• Robust to line interruption
• Need consistent orientation of lines/surfaces
• Is a norm
The space of currents: a vector space:
• Addition = union
• Scaling = weighting different structures
• Sign = orientation
Durrleman, PhD thesis, 2010
![Page 33: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/33.jpg)
Shape Regression w/o Correspondence
• Regression model:
• Variational problem, minimize:
d: modeling shapes as “currents” → distance metric
• → From sampled to continuous shape model:
shape baseline :
ndeformatio varyingtime:
)()(
0
0
M
MtSS
t
tii
Vaillant, M., Glaunes, J.: Surface matching via currents (IPMI 2005). Springer LNCS Vol. 3565) 381–392 Durrleman, S., Pennec, X., Trouve, A., Thompson, P., Ayache, N.: Inferring brain variability from diffeomorphic deformations of currents: an integrative approach. Medical Image Analysis 12/5(12) (2008) 626–637
ii MtS )(
![Page 34: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/34.jpg)
• Concept: Given a set of discrete shapes, interpolate a continuous 4D
growth model via shape regression.
• Assumption: Growth/degeneration of biological tissue is inherently
smooth in space and time & nonlinear, locally varying process.
• Method: Continuous flow of diffeomorphisms via correspondence-free
“currents”. Cost function = Data Matching + Regularity.
Durrleman, Pennec, Ayache, Trouve, Gerig, MICCAI ‘09
Fishbaugh, Durrleman, Gerig, MICCAI ’11, SPIE’12, MICCAI’12, IPMI’13
4D Shape Modeling from Time-Discrete Data
![Page 35: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/35.jpg)
Acceleration Controlled Shape Regression
We define the acceleration field a(x(t)) as a vector field of the form
the shape points carrying a point force vector αi
a Gaussian
kernel with standard deviation λV
Time varying deformation given by:
initial position
initial velocity
![Page 36: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/36.jpg)
Regression Criterion
![Page 37: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/37.jpg)
Acceleration Controlled Shape Regression
Point forces α Acceleration Velocity
Evolution of cerebellum from 6 to 24 months
![Page 38: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/38.jpg)
Piecewise Geodesic vs Acceleration Controlled
Synthetic experiment comparing piecewise geodesic and acceleration controlled shape regression
Piecewise geodesic Acceleration controlled
![Page 39: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/39.jpg)
Interpolation Properties
![Page 40: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/40.jpg)
Summary
Benefits:
● More biologically realistic trajectories ● Nice interpolation properties
Drawbacks:
● Not compact or generative
![Page 41: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/41.jpg)
Content
• Motivation Longitudinal Modeling
• Overview Correspondence
• Currents: Correspondence-free modeling
• Driving Applications:
• Early Brain Development in Autism
• Huntington’s Disease (HD)
• Mandibular Growth
• Shape Analysis via Grid Deformations
• Growth Trajectories via Time Warp
![Page 42: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/42.jpg)
Autism: Longitudinal Infant Neuroimaging Study
ACE-IBIS NIH Study: UNC (PI), McGill, Seattle, WU, CHOP, Utah (Image Analysis)
Brain enlargement in autism starts at year 1.
Why? What? Effect?: Longitudinal MRI/DTI study w. >1250 MRI/DTI
Better understanding → Early intervention to improve outcome
30
35
40
45
50
55
0 3 6 9 12 15 18 21 24 27 30 33 36
Age (months)
Hea
d C
irc
um
fere
nc
e
Combined Controls
Autism
N= 113, ADI, ADOS N= 189 community controls
Brain Enlargement on MRI
Longitudinal Head Circumference (Cody
Hazlett et al., ArchGen Psyc Dec. 2005)
![Page 43: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/43.jpg)
Longitudinal Shape Regression
Durrleman, Fishbaugh, Gerig, MICCAI 2011, MICCAI 2012
movie
![Page 44: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/44.jpg)
4D Shape Regression
From discrete 3D shapes (6m,12m,24m) to continuous 4D shape model
Fishbaugh, Durrleman, Gerig, MICCAI 2011, 2012
6 12 24 Time (months)
![Page 45: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/45.jpg)
Individual 4D Growth Profiles
HR+
HR-
LR-
![Page 46: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/46.jpg)
Longitudinal Shape Modeling
Individual 4D model
![Page 47: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/47.jpg)
Work in progress: Stats of 4D growth profiles
Autism Research Collaboration UNC (Piven, Hazlett)
HR+: High risk infant ADOS pos.
HR-: High risk infants ADOS neg.
LR-: Low risk healthy infants
Autism Research Collaboration ACE-IBIS (PI J. Piven, UNC)
![Page 48: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/48.jpg)
Motivation: Longitudinal Imaging in HD Searching for noninvasive biomarker with imaging…
• Symptomatic HD imaging findings
– Atrophied caudate and putamen
– Disproportionate loss of white matter
• Prodromal HD imaging findings
– Striatal atrophy correlates with:
• Neurological impairment
• Poorer performance on cognitive assessments
• Years to motor symptom onset
– Decreased white matter volume
• Seen >15 years before symptom onset
• Morphological study: cognitive deficits had stronger correlation with cerebral white matter atrophy than striatal atrophy Courtesy Jane Paulsen, Hans Johnson, U-Iowa
![Page 49: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/49.jpg)
Motivation: Huntington’s Disease
Courtesy Prof. Hans Johnsen, IOWA
NIH PREDICT Study: Longitudinal imaging study (3-5 scans over 2yr intervals).
Relationship between estimated years to diagnosis of Huntington's disease and
motor exam score and striatal volumes. Distribution of age of onset for individuals
with 36-56 CAG repeats based on the parametric model. Red indicates most likely
time of diagnosis. Blue line is proposed time period when interventional therapies
would have greatest impact.
![Page 50: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/50.jpg)
PREDICT-HD: Longitudinal Imaging Study
58yrs 59yrs 60yrs
Huntington’s Disease: • Neurodegenerative, progressive disease Visual assessment: Subtle changes over time: • Requires well-calibrated image data • Precise, robust segmentations
![Page 51: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/51.jpg)
Huntington’s Disease: Joint analysis of sets of anatomical structures
• Data: Iowa Huntington Disease (HD) study (NAMIC)
• Goal: Prediction of onset of HD from longitudinal preclinical imaging
![Page 52: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/52.jpg)
Clinical Application: Neurodegeneration
in Huntington’s Disease
Continuous individual
subjects’ growth models
Colllaboration Hans Johnson, U-Iowa
Quantitative information
derived from 4D shapes
![Page 53: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/53.jpg)
Muralidharan, Fletcher, Gerig, 2013
![Page 54: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/54.jpg)
Problem for Personalized Profiles: Variability in 3D Segmentation
Volumes from one subject
Volumes from multiple subjects
with varying disease burden
![Page 55: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/55.jpg)
HD: Joint 4D Modeling of subcortical structures
![Page 56: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/56.jpg)
Subject-Specific Shape Modeling
Caudate volume for 32 subjects (3 time pts) extracted after shape regression.
Observed volumes are shown as circles, which highlight the noise in segmentation.
We estimate consistent shape trajectories by considering all shapes simultaneously
which respects the interplay between shape boundaries and locations.
Muralidharan et al, MICCAI 2014
![Page 57: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/57.jpg)
National Alliance for Medical Image Computing
http://na-mic.org
Longitudinal Shape Modeling
Example mandibular surgery case (L. Cevidanes, NA-MIC ancillary
grant).
• V2- 16 years of age before jaw surgery
in the upper jaw maxilla
• V6- 16 years of age 2 months post
surgery
• V10- 18 years of age, 2 years post-
surgery
• V12- 22 years of age, 6 years post-
surgery
![Page 58: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/58.jpg)
National Alliance for Medical Image Computing
http://na-mic.org
Longitudinal Shape Modeling
Example mandibular surgery case (L. Cevidanes, NA-MIC ancillary
grant).
EXOSHAPE ACCEL Tool: Correspondence-free, controlled
acceleration, no tuning, 20’ computing time
Color: speed:
blue = slow and red = fast
![Page 59: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/59.jpg)
Application: Craniosynostosis
4D atlas from 350 full brain shapes (6 – 825 days)
Paniagua, B. et al. 3D of brain shape and volume after cranial vault remodeling surgery for Craniosynostosis correction in infants. SPIE Medical Imaging 2013
velocity
![Page 60: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/60.jpg)
Content
• Motivation Longitudinal Modeling
• Overview Correspondence
• Currents: Correspondence-free modeling
• Driving Applications:
• Early Brain Development in Autism
• Huntington’s Disease (HD)
• Mandibular Growth
• Shape Analysis via Grid Deformations
• Growth Trajectories via Time Warp
![Page 61: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/61.jpg)
Shape Analysis via Transformations
D’Arcy Wentworth Thompson, On Growth and Form (1917, mathematics and biology)
![Page 62: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/62.jpg)
GEODESIC REGRESSION OF IMAGE AND SHAPE DATA
Fishbaugh et al, ISBI 2014
![Page 63: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/63.jpg)
GEODESIC REGRESSION OF IMAGE AND SHAPE DATA
Fishbaugh et al, ISBI 2014
![Page 64: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/64.jpg)
Pediatric Brain Development
![Page 65: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/65.jpg)
1) Estimation with images only
2) Estimation jointly but only showing image
3) Estimation jointly and showing both image and white matter
4) Estimation with white matter surfaces only
Pediatric Brain Development
![Page 66: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/66.jpg)
Pediatric Brain Development
Fishbaugh et al, ISBI 2014
![Page 67: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/67.jpg)
Huntington’s Disease: Joint 4D Modeling of Shapes and Images
![Page 68: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/68.jpg)
Subject-specific 4D shape & image regression
Control 2yrs Interval Huntington’s D. 2yrs Interval
Fishbaugh et al., IPMI 2013
![Page 69: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/69.jpg)
Work in Progress: Patient-specific 4D
shape & image regression Control Extrapolated HD Extrapolated
interpolation extrapolation time
Fishbaugh et al., ISBI ‘13, IPMI ‘13
![Page 70: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/70.jpg)
Deformetrics with Sparsity: Tackling fundamental problem of high-dim
features & low-dim sample size (HDLSS)
Image evolution described by considerably fewer parameters, Concentrated in areas undergoing most dynamic changes
Durrleman, 2013 / Fishbaugh IPMI 2013, GSI 2013
Initialization Atlas
![Page 71: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/71.jpg)
Statistics
Common template for
8 Down’s syndrome patients + 8 Ctrls
Most discriminative axis
Classification (leave-2-out) with 105 control points:
specificity sensitivity
Max Likelihood 100% (64/64) 100% (64/64)
LDA 98% (63/64) 100% (64/64)
Importance of optimization in control points positions!
Classification (leave-2-out) with 8 control points:
specificity sensitivity
Max Likelihood 97% (62/64) 100% (64/64)
LDA 94% (60/64) 89% (57/64)
![Page 72: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/72.jpg)
Content
• Motivation Longitudinal Modeling
• Overview Correspondence
• Currents: Correspondence-free modeling
• Driving Applications: • Early Brain Development in Autism
• Huntington’s Disease (HD)
• Mandibular Growth
• Shape Analysis via Grid Deformations
• Growth Trajectories via Time Warp
![Page 73: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/73.jpg)
4D Shape Analysis: Spatiotemporal Registration
Comparing two evolutions: Align shape trajectories:
Spatiotemporal variability analysis of longitudinal shape data, S. Durrleman, X. Pennec, A. Trouvé, G. Gerig, N. Ayache, MICCAI’10, IJCV 2012
ii MtS )(
ii NtT )(
Formalism captures: • continuous regression: ϕ(t) • morphological change: χ • change of growth speed: ѱ(t)
)()( 0MtS t
function change time:
ndeformatio lgeometrica :
where))),((()(
tStT
![Page 74: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/74.jpg)
4D Shape Analysis: Spatiotemporal Registration
Spatiotemporal variability analysis of longitudinal shape data, S. Durrleman, X. Pennec, A. Trouvé, G. Gerig, N. Ayache
![Page 75: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/75.jpg)
4D Shape Analysis: Spatiotemporal Registration
Spatiotemporal Variability: • morphological changes • change of growth speed
Spatiotemporal variability analysis of longitudinal shape data, S. Durrleman, X. Pennec, A. Trouvé, G. Gerig, N. Ayache
(Homo habilis to sapiens vs. homo erectus)
![Page 76: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/76.jpg)
Preliminary Results: Amygdala Growth in Autism
![Page 77: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/77.jpg)
Results: Amygdala Growth in Autism
![Page 78: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/78.jpg)
Conclusions
• Spatio-temporal Image & Shape Analysis: Emerging field with
new fundamental problems (math, stats, imaging, modeling):
– Multidisciplinary by definition.
– Actively developing field driven by new imaging technologies and
novel biomedical driving problems.
– Challenging fundamental, algorithmic and statistical problems.
– Research progress enables new scientific discoveries.
• Main driving motivation: Trajectory of change vs. Cross-
sectional comparisons.
• Clinically highly relevant for quantitative analysis of
longitudinal image data.
![Page 79: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/79.jpg)
Acknowledgements • NIH-NINDS: 1 U01 NS082086-01: 4D Shape Analysis
• NIH-NIBIB: 2U54EB005149-06 , NA-MIC: National Alliance for MIC
• NIH (NICHD) 2 R01 HD055741-06: ACE-IBIS (Autism Center)
• NIH NIBIB 1R01EB014346-01: ITK-SNAP
• NIH NINDS R01 HD067731-01A1: Down’s Syndrome
• NIH P01 DA022446-011: Neurobiological Consequences of Cocaine Use
• USTAR: The Utah Science Technology and Research initiative at the Univ. of Utah
• UofU SCI Institute: Imaging Research Team
• Insight Toolkit ITK
![Page 80: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/80.jpg)
Acknowledgements Methodology Development:
• James Fishbaugh, Utah
• Stanley Durrleman, INRIA Paris
• Xavier Pennec, INRIA Sophia Antipolis
• Ross Whitaker, Utah
• Martin Styner, UNC
Clinical Longitudinal Imaging:
• Joseph Piven, UNC Psychiatry
• Jane Paulsen and Hans Johnson, U of Iowa
• Lucia Cevidanes, UMICH
![Page 81: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/81.jpg)
National Alliance for Medical Image Computing
http://na-mic.org
Freely available Software
ExoshapeAccel: C/C++ NAMIC toolkit SW
for estimating continuous evolution from a
discrete collection of shapes,
James Fishbaugh Public download
Stanley Durrleman
http://www.deformetrica.org/
![Page 82: Spatio-Temporal Shape Modeling and Analysis · • Concept: Given a set of discrete shapes, ... • Method: Continuous flow of diffeomorphisms via correspondence-free “currents”.](https://reader034.fdocuments.us/reader034/viewer/2022050313/5f759a857a45f764ce0fc5f5/html5/thumbnails/82.jpg)