Lec15: Medical Image Registration (Introduction)
-
Upload
ulas-bagci -
Category
Science
-
view
37 -
download
1
Transcript of Lec15: Medical Image Registration (Introduction)
![Page 1: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/1.jpg)
MEDICAL IMAGE COMPUTING (CAP 5937)
LECTURE 15: Medical Image Registration I (Introduction)
Dr. Ulas BagciHEC 221, Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), Orlando, FL [email protected] or [email protected]
1 SPRING 2017
![Page 2: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/2.jpg)
Outline
• Motivation– Image registration is an alignment problem
• Registration basics• Rigid registration• Non-rigid registration• Example Applications
2
![Page 3: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/3.jpg)
Image Registration Taxonomy• Dimensionality
– 2D-2D, 3D-3D, 2D-3D
• Nature of registration basis– Image based
• Extrinsic, Intrinsic– Non-image based
• Nature of the transformation– Rigid, Affine, Projective, Curved
• Interaction– Interactive, Semi-automatic, Automatic
• Modalities involved– Mono-modal, Multi-modal, Modality to model
3
• Subject: Intra-subject Inter-subject Atlas
• Domain of transformation
• Local, global
• Optimization procedure
• Gradient Descent, SGD, …
• Object
• Whole body, organ, …
![Page 4: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/4.jpg)
Open Source Implementation• ITK• ANTS (advanced normalization tools) (PICSL of Upenn)• CAVASS (MIPG of Upenn)• Nifty Reg (UCL)• Elastix (www.elastix.isi.uu.nl)• FAIR (Modersitzki 2009), mostly matlab.• 3D Slicer• FSL• …
4
![Page 5: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/5.jpg)
Modalities in Medical Imaging• Mono-modality:
5
![Page 6: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/6.jpg)
Modalities in Medical Imaging• Mono-modality:
ü A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
6
![Page 7: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/7.jpg)
Modalities in Medical Imaging• Mono-modality:
ü A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
ü Images may be acquired weeks or months apart; taken from different viewpoints.
7
![Page 8: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/8.jpg)
Modalities in Medical Imaging• Mono-modality:
ü A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
ü Images may be acquired weeks or months apart; taken from different viewpoints.
ü Aligning images in order to detect subtle changes in intensity or shape
8
![Page 9: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/9.jpg)
Modalities in Medical Imaging• Mono-modality:
ü A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
ü Images may be acquired weeks or months apart; taken from different viewpoints.
ü Aligning images in order to detect subtle changes in intensity or shape
• Multi-modality:
9
![Page 10: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/10.jpg)
Modalities in Medical Imaging• Mono-modality:
ü A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
ü Images may be acquired weeks or months apart; taken from different viewpoints.
ü Aligning images in order to detect subtle changes in intensity or shape
• Multi-modality:ü Complementary anatomic and functional information from multiple
modalities can be obtained for the precise diagnosis and treatment.
10
![Page 11: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/11.jpg)
Modalities in Medical Imaging• Mono-modality:
ü A series of same modality images (CT/CT, MR/MR, Mammogram pairs,…).
ü Images may be acquired weeks or months apart; taken from different viewpoints.
ü Aligning images in order to detect subtle changes in intensity or shape
• Multi-modality:ü Complementary anatomic and functional information from multiple
modalities can be obtained for the precise diagnosis and treatment.
ü Examples: PET and SPECT (low resolution, functional information) need MR or CT (high resolution, anatomical information) to get structure information.
11
BEFORE
AFTER
PET/CT EXAMPLE
![Page 12: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/12.jpg)
In other words,…• Combining modalities (inter
modality) gives extra information.
• Repeated imaging over time same modality, e.g. MRI, (intra modality) equally important.
• Have to spatially register the images.
12
![Page 13: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/13.jpg)
13 Before Registration
![Page 14: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/14.jpg)
14 After Registration
![Page 15: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/15.jpg)
15
![Page 16: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/16.jpg)
16
![Page 17: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/17.jpg)
17
![Page 18: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/18.jpg)
18
![Page 19: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/19.jpg)
19
![Page 20: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/20.jpg)
20
![Page 21: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/21.jpg)
21
![Page 22: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/22.jpg)
22
![Page 23: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/23.jpg)
Summary of Mostly Used Applications• Diagnosis
– Combining information from multiple imaging modalities• Studying disease progression
– Monitoring changes in size, shape, position or image intensity over time
• Image guided surgery or radiotherapy– Relating pre-operative images and surgical plans to the physical reality
of the patient• Patient comparison or atlas construction
– Relating one individual’s anatomy to a standardized atlas
23
![Page 24: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/24.jpg)
Then, What is Image Registration (formally)?
24
![Page 25: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/25.jpg)
Image Registration is a• Spatial transform that maps points from one image to
corresponding points in another image
25
matching two images so that corresponding coordinate points in the two images correspond to the same physical region of the scene being imaged also referred to as image fusion, superimposition, matching or merge
MR SPECT registered
![Page 26: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/26.jpg)
Image Registration is a• Spatial transform that maps points from one image to
corresponding points in another image– Rigid
• Rotations and translations– Affine
• Also, skew and scaling– Deformable
• Free-form mapping
26
![Page 27: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/27.jpg)
Registration Framework
27
Deformation Model
Optimization Metho
d
Matching Criteria (Objective Function)
![Page 28: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/28.jpg)
Recap: Image Registration Taxonomy• Dimensionality
– 2D-2D, 3D-3D, 2D-3D
• Nature of registration basis– Image based
• Extrinsic, Intrinsic– Non-image based
• Nature of the transformation– Rigid, Affine, Projective, Curved
• Interaction– Interactive, Semi-automatic, Automatic
• Modalities involved– Mono-modal, Multi-modal, Modality to model
28
• Subject: Intra-subject Inter-subject Atlas
• Domain of transformation
• Local, global
• Optimization procedure
• Gradient Descent, SGD, …
• Object
• Whole body, organ, …
![Page 29: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/29.jpg)
Deformation ModelsMethod used to find the transformation • Rigid & affine
– Landmark based– Edge based– Voxel intensity based– Information theory based
• Non-rigid– Registration using basis functions– Registration using splines– Physics based
• Elastic, Fluid, Optical flow, etc.
29
![Page 30: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/30.jpg)
Deformation Model (Transformation)• Rigid
– Rotation, translation• Affine
– Rigid + scaling• Deformable
– Affine + vector field• ….
30
![Page 31: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/31.jpg)
31
Deformation Model (linear vs. non-linear)
![Page 32: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/32.jpg)
Linear Registration -> Separable
32
RIGID TRANSFORMATION
rotation
![Page 33: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/33.jpg)
Linear Registration -> Separable
33
RIGID TRANSFORMATION
rotation
![Page 34: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/34.jpg)
Rigid Registration - Rotation
34
![Page 35: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/35.jpg)
Example Rigid Transformation Formulation
35
![Page 36: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/36.jpg)
Example Rigid Transformation Formulation
36
Old location new location
![Page 37: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/37.jpg)
Linear Registration -> Separable
37
AFFINE TRANSFORMATION = Rigid + Scaling (+ skew)
9 parameters, Affine = 6 parameters (rotation + translation) + 3 parameters (scaling) 12 parameter, Affine = …+ 3 parameters (skew)
![Page 38: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/38.jpg)
Shear in 3D
38
![Page 39: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/39.jpg)
Affine Transformation
39
p’ = M p + t
![Page 40: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/40.jpg)
Homogenous Coordinates for Transformations
40
![Page 41: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/41.jpg)
Homogenous Coordinates for Transformations
41
![Page 42: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/42.jpg)
Translation in Homogenous Coordinate System
42
![Page 43: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/43.jpg)
Registration is an alignment problem
p = (825,856) q = (912,632)
q = T(p;a)
Pixel location in first image Homologous pixel location in second image
Pixel location mapping function
![Page 44: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/44.jpg)
Registration is an alignment problem
p = (825,856) q = (912,632)
q = T(p;a)
Pixel scaling and translation
![Page 45: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/45.jpg)
Similarity Criteria
45
![Page 46: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/46.jpg)
Intensity Based• Method
– Calculating the registration transformation by optimizing some measure calculated directly from the voxel values in the images
• Algorithms used– Registration by minimizing
intensity difference – Correlation techniques – Ratio image uniformity– Partitioned Intensity
Uniformity
46
![Page 47: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/47.jpg)
Intensity Based• Intensity-based methods compare intensity
patterns in images via some similarity metrics
– Sum of Squared Differences– Normalized Cross-Correlation– Mutual Information
47
![Page 48: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/48.jpg)
Feature Based• Feature-based methods find correspondence between image
features such as points, lines, and contours.
• Distance between corresponding points• Similarity metric between feature values
– e.g. curvature-based registration
48
![Page 49: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/49.jpg)
Information Theory Based• Image registration is considered as
to maximize the amount of shared information in two images– reducing the amount of information in
the combined image • Algorithms used
– Joint entropy• Joint entropy measures the amount
of information in the two images combined
– Mutual information• A measure of how well one image
explains the other, and is maximized at the optimal alignment
– Normalized Mutual Information
49
![Page 50: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/50.jpg)
50
![Page 51: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/51.jpg)
51
![Page 52: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/52.jpg)
Simple Code for Joint Entropy Computation
rows=size(x,1); cols=size(y,2); N=256; h=zeros(N,N);
for i=1:rows;for j=1:cols;
h(x(i,j)+1,y(i,j)+1)= h(x(i,j)+1,y(i,j)+1)+1; end
end imshow(h) end
52
![Page 53: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/53.jpg)
53 Because the images are identical, all gray value correspondences lie on the diagonal.
![Page 54: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/54.jpg)
54
![Page 55: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/55.jpg)
• I(A,B) = H(A) + H(B) – H(A,B)– Maximizing mutual information is related to minimizing joint entropy
55
Less sensitive to changes in overlap!
![Page 56: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/56.jpg)
Registration Algorithm• Fixed and Moving Images (target and source, …)• Preprocessing• Define Similarity Measure (NMI, CC, MSE, …)• Define Spatial Transformation (Rigid, Affine, Deformable)• Implementation
1. Initialize2. Transform (and Interpolate) moving image3. Measure similarity4. Optimize (decide parameters of the transform)5. If converged
STOP6. Else7. Go to Transform Step 2. Repeat
56
![Page 57: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/57.jpg)
Summary• Introduction to the medical image registration• Transformation types
– Rigid, affine, non-rigid• Mono-modal, multi-modal image registration• Similarity metric• Mutual Information
• Next Lecture(s): further details on the topic.
57
![Page 58: Lec15: Medical Image Registration (Introduction)](https://reader034.fdocuments.us/reader034/viewer/2022042707/58ed20e51a28ab43498b4671/html5/thumbnails/58.jpg)
Slide Credits and References• Credits to: Jayaram K. Udupa of Univ. of Penn., MIPG• Sir M. Brady’s Lecture Notes (Oxford University)• Darko Zikic’s MICCAI 2010 Tutorial• Bagci’s CV Course 2015 Fall.• K.D. Toennies, Guide to Medical Image Analysis,• Handbook of Medical Imaging, Vol. 2. SPIE Press.• Handbook of Biomedical Imaging, Paragios, Duncan, Ayache.• Seutens,P., Medical Imaging, Cambridge Press.• Aiming Liu, Tutorial Presentation.• Jen Mercer, Tutorial Presentation.
58