Tissue Engineering introduction for physicists - Lecture three
-
Upload
ali-bakhshinejad -
Category
Engineering
-
view
61 -
download
1
Transcript of Tissue Engineering introduction for physicists - Lecture three
Tissue Engineering
Medical Imaging / Volume visualization
September 2014
Part Two
Medical Imaging / VV
“Visualization is a method of computing. It transforms the symbolic into
the geometric, enabling researchers to observe their simulations and
computations. Visualization offers a method for seeing the unseen. It
enriches the process of scientific discovery and fosters profound and
unexpected insights.”
McCormick, B. H. (1988). Visualization in scientific
computing. ACM SIGBIO Newsletter, 10(1), 15–21.
doi:10.1145/43965.43966
Medical Imaging / VV
Medical Imaging / VV
Sources of Error● Data acquisition
○ Sampling: are we (spatially) sampling data with enough precision to get what we need
out of it?
○ Quantization: are we converting “real” data to a representation with enough precision
to discriminate the relevant features?
● Filtering
○ Are we retaining/removing the “important/non-relevant” structures of the data?
○ Frequency/spatial domain filtering
■ Noise, clipping and cropping
● Selecting the “right” variable
○ Does this variable reflect the interesting features?
○ Does this variable allow for a “critical point” analysis?
Medical Imaging / VV
Sources of Error● Functional model for resampling
○ What kind of information do we introduce by interpolation and approximation?
● Mapping
○ Are we choosing the graphical primitives appropriately in order to depict the kind of
information we want to get out of the data?
○ Think of some real world analogue (metapher)
● Rendering
○ Need for interactive rendering often determines the chosen abstraction level
○ Consider limitations of the underlying display technology
■ Data color quantization
○ Carefully add “realism”
■ The most realistic image is not necessarily the most informative one
Medical Imaging / VV
Segmentation of anatomical structures of interest
Aligning multiple dataset (registration)
Three-dimensional visual representation
Medical Imaging / VV → Segmentation
Segmentation generally means dividing an image into connected regions.
Segmentation in the domain of medical imaging has some characteristic that
make the segmentation task easier and difficult at the same time.
● The imaging is narrowly focused on an anatomic region.
● The imaging context is well-defined.
● The pose variation are limited, and there is usually prior knowledge of
the number of tissues and the Region of Interest (ROI).
❖ The images produced in this field are one of the most challenging due
to the poor quality of imaging.
Medical Imaging / VV → Segmentation
Depending on the level of feature extraction as an input to the
segmentation, we can methodically classify segmentation as:
● Pixel-oriented
● Edge-oriented
● Texture-oriented
● Region-oriented
Or hybrid approaches, which result from combination of single
procedures.
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation
Pixel-based procedures of segmentation only consider the grayscale or
color value of current pixels disregarding its surroundings.
Pixel-based approaches are NOT segmentation procedures in the strict
sense of our definition. Since each pixel is considered only isolated from its
neighborhood, it cannot be ensured that actually only connected segments
are obtained. For this reason, post-processing is required.
Most pixel-based procedures use thresholds in the histogram of an image.
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Static Thresholding
If the assignment of Pixel intensities is well known and constant for a certain
type of tissue, static thresholds are applicable.
A static threshold is independent of the individual instance in a set of similar
images.
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Static Thresholding / Example
Pixel-based segmentation in CT relies on Hounsfield Units (HU), which
allow the definition of windows for different type of tissue:
Bone [200 … 3000] Water [-200 … 200]
Fat [-500 … -200] Air [-1000 … -500]
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Adaptive Thresholding
Global adaptive thresholds result from analyzing each individual image
entirely.
The well-known method of Otsu is based on a simple object vs. background
model.
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Adaptive Thresholding
Otsu’s method is used to automatically perform clustering-based image
thresholding, or, the reduction of a gray level image to binary image.
The algorithm assumes that the image contains two classes of pixels
(foreground pixels and background pixels)
It then calculates the optimum threshold separating the two classes so that
their combined spread (intra-class variance) is minimal.
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Adaptive Thresholding
In Otsu’s method we search for the threshold that minimizes the intra-class variance (the variance
within the class), defined as a weighted sum of variances of the two classes:
Weights are the probabilities of the classes separated by a threshold t
and variances of these classes.
Otsu shows that minimizing the intra-class variance is the same as
maximizing inter-class variance
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Adaptive Thresholding
Otsu shows that minimizing the intra-class variance is the same as
maximizing inter-class variance
which is expressed in terms of class probabilities and class means
Medical Imaging / VV → Segmentation
Pixel- Based Segmentation → Adaptive Thresholding
The class probability is computed from the histogram as t:
While the class mean is
where x(i) is the value at the center of ith histogram bin.
Medical Imaging / VV → Segmentation
Medical Imaging / VV → Segmentation
Using locally adaptive thresholds, the threshold is computed not only for each image individually,
but also for each region within an image.
This is particularly necessary if the simple object to background assumption is globally invalid
because of continuous brightness gradient.
Medical Imaging / VV → Segmentation
Example: (Threshold)
Segments obtained from pixel-based analysis usually are incoherent and highly noisy. Therefore,
pot-processing is required.
Blood tracer detection with using pixel-based segmentation (Thresholding)
Medical Imaging / VV → Segmentation
● Spatial information: Since intensity histograms do not preserve spatial contiguity of pixels, one
variation is to add spatial position (x,y) or (x,y,z) to form a multi-dimensional feature vector
incorporating spatial layout.
● Temporal information: If the medical images are in a time sequence, then time can be added as
an additional feature in the representation space.
Medical Imaging / VV → Segmentation
Clustering
Pixel clustering is another way of pixel-based segmentation. This statistical method is particularly
suitable if more than one value is assigned to each pixel and regarded in the segmentation
process (e.g., color images).
Medical Imaging / VV → Segmentation
Clustering
There is a large body of work on clustering algorithms. For our purposes, we can categorize them
into three broad classes:
● Deterministic clustering
● Probabilistic clustering (model-based clustering)
● Graph-theoretic clustering
Medical Imaging / VV → Segmentation
Clustering → Deterministic algorithms
This algorithms known as the simplest methods such as
❏ K-means
❏ mean-shift
❏ agglomerative methods
Medical Imaging / VV → Segmentation
Clustering → Deterministic algorithms
K-means approach, provides good results when the data is convex or blob-like.
Def. : Given a set of observations (x , x ,..., x ), where each observation is a d-dimensional real
vector, k-means clustering aims to partition the n observations into k(< n) sets S={S , S ,...,S } so
as to minimize the within-cluster sum of squares (WCSS). In other words, its objective to find:
where is the mean of points in S
1 2 n
1 2 k
i
Medical Imaging / VV → Segmentation
Edge-Based Segmentation
This type of segmentation is based on the abstract level of edges and tries
to capture the objects due to their closed outline in the image.
Because of that, edge-based segmentation, only used where objects are
presented as clearly defined boundaries.
Medical Imaging / VV → Segmentation
Edge-Based Segmentation→ Livewire Segmentation
In practice, edge-based segmentation is often realized semi-automatically.
By the interactive livewire segmentation, the user clicks onto or near by the edge of the Object of
Interest (OOI), and the computer determines the exact edge location based on local gradients.
Then, the computer calculates a cost function, which again is based on local gradients. For all
paths (wire) to the current position of the cursor, the path with the lowest cost is displayed in real
time (live) as the cursor is moved manually.
Medical Imaging / VV → Segmentation
Edge-Based Segmentation→ Livewire Segmentation
Medical Imaging / VV → Segmentation
Region-Based Segmentation
As an advantage of region-based segmentation, only connected segments are produced, and
morphological post-processing is avoided.
There are agglomerative (bottom-up) and divisive (top-bottom) approaches.
All approaches are based on a certain distance or similarity measure to guide the assignment of
neighbored pixels or regions.
Medical Imaging / VV → Segmentation
Region-Based Segmentation → Agglomerative Algorithm
Region growing, in 3D also referred to as volume growing, is a well known example of an
agglomerative procedure.
Starting from seed points, which may be placed either automatically or manually, neighbored pixel
are iteratively associated to the growing areas if the distance measure is below a certain threshold.
This process is iterate until no more merges can be carried out.
Medical Imaging / VV → Segmentation
Region-Based Segmentation → Divisive Algorithm
The divisive approach somehow inverts the agglomerative strategy. By splitting, the regions are
iteratively subdivided until they are considered sufficiently homogeneous in terms of the chosen
similarity measure.
Advantage:
Seed points are not require anymore, because the first split is performed throughout
the whole image.
Disadvantage:
The dividing lines are usually drawn horizontally or vertically, and this arbitrary
separation may separate the image objects.
Medical Imaging / VV → Segmentation
Region-Based Segmentation → Divisive Algorithm
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Active contour models apply edge-based segmentation considering region-based aspects and an
object-based model of a priori knowledge.
In the medical application domain, so called snake and balloon approaches are applied for
segmentation of 2D and 3D image data and the tracing of contours in 2D image and 3D image
sequences.
The contour of the objects, which is usually closely modeled, is presented by individual nodes,
which are - in the simplest case - piecewise connected with straight lines forming a closed
polygon.
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Snake approach:
In 1988, the classical snake approach was introduced by Kass et al. (Snakes: Active Contour
Models)
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Snake approach:
It models an internal and an external quality criterion, both as undirected energy. The internal
energy results from a predefined elasticity and stiffness of the contour, which is high in places of
strong bends or on buckling.
The external energy is calculated from an edge-filtered image. The external energy is small, if the
contour runs along edges.
The idea behind this approach is an edge-based segmentation combined with the a priori
knowledge that biological objects rarely have sharp-bending boundaries.
With an optimal weighting of energy terms, the contour course is primarily determined by the
information of edges in the image.
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Snake approach:
However, if the object’s contour is partially covered or incompletely captured, the internal energy
ensures an appropriate interpolation of the region’s shape.
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Snake approach:
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Snake approach:
Medical Imaging / VV → Segmentation
Model-Based Segmentation → Active Contour Model
Balloon approach:
Balloons are based on forces rather than energies. Besides the internal and external forces, an
inner pressure or suction is modeled, which lets the contour continuously expand or shrink.
Medical Imaging / VV → Segmentation
Example:
Considering brain MR imaging, as an example, we know that there are three main tissues of
interest:
● White matter
● Gray matter
● Cerebrospinal fluid
➔ We also know that in a pathological situation there may be one additional class such as
lesions or tumors.
The core operation is the division of the image into a finite set of regions, which are smooth and
homogeneous in their content and their representation.
When posed in this way, segmentation can be regarded as a problem of finding clusters in a
selected feature space.
Medical Imaging / VV → Segmentation
Example: (Threshold)
Such a simple segmentation approach, is often
insufficient for medical images where the imaging
protocol can lead to variations in regional contrast
making the task of segmentation difficult.
Medical Imaging / VV → Segmentation
Several variations on classical histogram
thresholding have been proposed for medical
image segmentation.
● Multi-modal or multi-sequence data
Multi-dimensional are histograms formed from the
intensity values produced by each of the imaging
protocols. It is often the case that several
acquisitions are available for the same image.
T
T1
2
Medical Imaging / VV → Segmentation
References for segmentation:
● M. Sonka, J. M. Fitzpatrick, Eds., Handbook of Medical Imaging, Volume 2.
Medical Image Processing and Analysis, SPIE, 2000
● Elnakib, A., Gimel’farb, G., Suri, J. S., & El-Baz, A. (2011). Medical Image
Segmentation: A Brief Survey. In A. S. El-Baz, R. Acharya U, A. F. Laine, &
J. S. Suri (Eds.), Multi Modality State-of-the-Art Medical Image
Segmentation and Registration Methodologies. New York, NY: Springer
New York. doi:10.1007/978-1-4419-8204-9
Medical Imaging / VV → Image Registration
Image Registration
Image Registration is an important problem in image analysis with many
applications:
● Several images of the same object are taken using different imaging
modality
● Several images of the same object are taken at different time instants
● It is necessary to compare two objects
● It is desired to match an image to a model
Medical Imaging / VV → Image Registration
● The problem of image registration is to determine an unknown geometric
transformation that maps one image into another (to a certain degree
accuracy)
○ In other word, after registration problem is solved, for each pixel in the
first image we know the corresponding pixel in the second image.
● This assumes that the images are similar in the sense that both images
contain the same (or similar) object, which may be rotated, translated, or
elastically deformed
Medical Imaging / VV → Image Registration
● In medical applications, image registration is usually done for two-
dimensional and three-dimensional images
● In general, registration problem can be solved in any number of spatial or
temporal dimensions.
Medical Imaging / VV → Image Registration
● When two images are registered it is possible to:
○ Analyze (detect) differences between the images (e.g. images taken
at two different time instants or difference between the template and a
tested product in visual inspection)
○ Combine information contained in multiple images into a single image
(image fusion) with the goal of easier interpretation by humans (e.g. in
radiology it is possible to do multimodality image registration - MR to
CT, etc. )
Medical Imaging / VV → Image Registration
● Medical image registration is required for:
○ Use of different imaging modalities (e.g. MR/CT)
○ Progressive disease tracking (imaging in regular time intervals and
detection of changes, e.g. for tumor treatment evaluation)
○ In computer assisted surgery (e.g. in neurosurgery preoperative MR
images may be registered with intraoperative MR images for surgical
navigation)
○ Matching of patient images to a model
Medical Imaging / VV → Image Registration
Example: Hand Registration
● X-ray image (anatomical information)
● Nuclear medicine image (functional information)
● After registration, hand image obtained by
nuclear medicine imaging is pseudocolored and
superimposed on the gray scale X-ray hand
image
● Red color corresponds to the largest isotope
concentration.
Medical Imaging / VV → Image Registration
Example: Brain Registration
● MR image showing anatomy (left), PET* FDG image showing function
superimposed on MR image (right)
*Positron emission tomography (PET) is a nuclear medicine, functional imaging technique that produces a three-dimensional image of functional processes in the body.
Medical Imaging / VV → Image Registration
Classification of Methods
● Registration methods can be classified with respect to several different
criteria such as:
○ Dimensionality of images that are registered (2D, 3D or 4D methods)
○ Image features being matched (extrinsic and intrinsic methods)
○ Mechanism of interaction with the user
○ Type of geometric transformation used for registration
Medical Imaging / VV → Image Registration
Classification of Methods→ Dimensionality
● We can register 2D, 3D or 4D images
● 3D registration can be done for two 3D images or for a temporal sequence
of 2D images (video)
● If we want to register a sequence of 3D images this represents 4D
registration problem
● There is also a problem of 2D image (perspective projection of 3D space) to
3D image registration
○ In this case it is necessary to determine the view parameters so that the
obtained perspective transformation of 3D image matches the 2D image
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Features
● This classification is motivated by the type of image features used for image
registration:
○ Extrinsic methods (external objects or markers are used as reference
points for registration)
○ intrinsic methods (registration is based on pixel values - no external
objects are used)
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Features >> Extrinsic Methods
● Extrinsic methods use artificial external objects (markers) attached to the
object to be registered
● Markers are detected in both images and used for registration
○ Example: For brain image registration skin markers or stereotactic
frames may be used
● Disadvantage: Registration is based on external markers so accuracy
depends on the accuracy of marker detection
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Features >> Intrinsic Methods
● Intrinsic registration methods do not use artificial external objects
● Intrinsic methods use:
○ Anatomical landmarks (points, contours, or surfaces), or
○ Pixel values (intensity-based methods)
● Anatomical landmarks must be detected and this represents a disadvantage
(possibility of error)
● Intensity-based methods have advantage of relying only on pixel values
without the need for detection of special landmarks
Medical Imaging / VV → Image Registration
Classification of Methods→ User Interaction
● With respect to user interaction, registration methods can be divided into:
○ Interactive (require user interaction to define the geometric
transformation for registration)
○ Semi-automatic (user interaction is only required for initialization,
guidance, or stopping the registration procedure)
○ Automatic (do not require any user interaction)
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations
● This classification is based on the type of transformation used for
registration:
○ Rigid registration: distance between any two object points is preserved
(rotation, translation)
○ Affine transformation: A line is mapped into a line, parallelism between
lines is preserved
○ Projection transformation (e.g. perspective projection) is like affine, but
it does not preserve parallelism of lines
○ Elastic transformation: line is mapped into a curve
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations
● Rigid transformations are a subset of affine transformations
● Affine transformations are a subset of projective transformations
● Projective transformations are a subset of elastic transformations
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations
● In the next several slides we present an overview of the basic geometric
transformations:
○ Rigid transformations
○ Scaling transformations
○ Affine transformations
○ Projective transformations
○ Perspective transformations
○ Elastic transformations
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Rigid Transformations
● A rigid transformation of vector consists translation and rotation:
Where is translation vector, R is a 3x3 orthogonal rotation matrix ( ,
d and are rigid body rotation angles around z,y and x axes)
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Scaling Transformations
The simplest affine transformations are those that only include scaling, while the
rest of the transformation is rigid:
x’ = RSx + t
Where S= is scaling matrix in x,y and z direction, R is rotation
matrix, and t is translation vector
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Affine Transformations
Affine transformations preserve lines and parallel lines and are defined by
expression
x’ = Ax +t
where A is affine transformation matrix that can have any value, and t is
translation vector
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Affine Transformations
● For easier manipulation of matrix expressions, a representation using
homogeneous coordinates is often used
● Homogeneous coordinate vector of a 3D point is 4D
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Projective Transformations
● Projective transformations are similar to affine (lines are preserved), only
there is no preserved), only there is no preservation of parallel relation
● The analytical form is given by:
x’ = (Ax +t) / (p.x + )
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Projective Transformations
In homogeneous coordinates we have:
where
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Projective Transformations
● Perspective transformations map 3D space into a 2D image plane
● Examples: xamers imaging, X-ray imaging, microscopy
● Perspective transformations are a subset of projective transformations
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Elastic Transformations
● Elastic transformations do not preserve lines (i.e. a line can be mapped into
a curve)
● An elastic transformation can be defined by any non-linear mapping of
spatial coordinates
● Polynomial are often used in practice for simplicity
● For 3D case
Medical Imaging / VV → Image Registration
Classification of Methods→ Geometric Transformations >> Elastic Transformations
● In practice, polynomial order is limited because of oscillations present in
high-order polynomials
● For this reason, polynomial order usually chosen so that I,J,K ≤ 2
● For the same reason it is often taken that I+J+K ≤ 5
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms
● In the following material we present three groups of algorithms for 3D
medical image registration:
○ Algorithms using corresponding points identified in images
○ Algorithms using corresponding surfaces
○ Algorithms using voxel intensity values
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
● Point-based registration requires:
○ Identification of corresponding 3D points in images to be aligned
○ Registration of the points (to identify unknown geometric transformation
that maps one set of points into another)
○ Use determined transformation to map all other image points (i.e. to
establish point correspondences)
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
● In medical image registration, 3D points used for registration are often called
fiducial markers or fiducial points
● 3D points can be either external markers attached to human body or
anatomical landmarks identified in the images
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
● The usual approach to point-based registration problem is to find the least-
square rigid-body or affine transformation that aligns the points
● The obtained transformation can then be used to transform any point from
one image to another
● This problem is often referred to as the orthogonal Procrustes problem
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
● The usual approach to point-based registration problem is to find the least-
square rigid-body or affine transformation that aligns the points
● The obtained transformation can then be used to transform any point from
one image to another
● This problem is often referred to as the orthogonal Procrustes problem
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
Orthogonal Procrustes Problem
● Problem definition: Given two configurations of N points in D dimensions
P = {p } i Q = {q }, it is necessary to find transformation T that minimizes
error:
G(T) = | T(P) - Q|
● P i Q are NxD matrices whose rows are coordinates of points p i q , and
T(P) is the corresponding matrix of transformed points p
● The standard case is when T is a rigid-body transformation
i i
2
i i
i
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
Orthogonal Procrustes Problem
● If T is affine transformation, we obtain the standard least-squares problem
● In the following slides we show the solution for case when T is a rigid-body
transformation defined by rotation matrix R and translation vector t
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
Orthogonal Procrustes Problem >> Solution
● First replace vectors in P and Q by their demeaned versions (mean value
equal to zero):
● This reduces the problem to the orthogonal Procrustes problem in which we
need to determine orthogonal rotation matrix R
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
Orthogonal Procrustes Problem >> Solution
● Central to the problem is the DxD correlation matrix , which shows
how much the points in Q are predicted by points in P
● The singular value decomposition of matrix K is given by:
where U and V are orthogonal matrices containing left and right singular vectors
and D is a diagonal matrix containing singular values
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Point-based
Orthogonal Procrustes Problem >> Solution
Orthogonal matrix R is determined by expression:
Translation vector t may be determined by expression
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
● The second approach to 3D registration is by using surfaces in medical
images, which are often more distinct than point landmarks
● Segmentation algorithms are used to locate surfaces:
○ For example tissue to air boundaries often have high contrast, which
makes surface detection easier
● If two corresponding surfaces can be detected in images to be matched,
then rigid-body registration can be achieved by fitting the surfaces
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
● Some of the best known surface registration algorithms are:
● The head and hat algorithm
● Distance transform-based algorithms
● Iterative closest point (ICP) algorithm
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Head and Hat Algorithms
● Developed by Pelizzari i ostali, 1989, for 3D registration of CT, MR and PET
head images
● The first surface (head) is obtained from higher resolution modality and is
represented as a stack of image slices
● The second surface (hat) is represented as a list of unconnected 3D points
● Registration is performed by iterative transformation of the hat surface to
find the best fit onto the head surface
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Head and Hat Algorithms
● Registration accuracy is measured by the square of the distance between
the point on the hat and the nearest point on the head in the direction of the
head centroid
● Iterative optimization using Powell steepest descent algorithm, which
performs a series of 1D optimizations in each of the six dimensions:
○ For 3D rigid-body registration we have six degrees of freedom (three
rotational and three translations)
● This method is useful only for spherical surfaces
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Distance Transforms
● Head and hat algorithm can be improved using a distance transform to
preprocess head images
● A distance transform maps a binary image into a distance image
○ In the distance image each pixel has the value of the distance of that
pixel to the nearest surface in the binary image
● Distance transform is computed for one of the images, which makes it easy
to calculate distance from one surface to another
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Iterative Closest Point Algorithm
● Iterative closest point (ICP) algorithm was developed by Besl and McKay,
1992, for 3D registration
● It was developed for general use, but is now the most widely used surface
matching algorithm for biomedical applications
● Let us assume that we have two surfaces:
○ The first surface (call it M) is the model surface
○ The second is given as a point set {p }i
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Iterative Closest Point Algorithm
● Repeat:
○ For each point p identify the closest point q on the model surface M
○ Use Procrustes method to register point sets p and q
○ Apply identified geometric transformation to point set p to obtain the
new set of points, call them p’
○ Let p = p’ , for each i
● Until the change in registration mean square error falls below a defined
threshold
i i
i
i
i
i i
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Intensity-based Methods
● In all previous methods registration was based on data extracted from
images (landmark points or surfaces)
○ Advantage: it is not important how images are taken (different imaging
modalities can be used) - registration is based on registration of points
or surfaces
○ Disadvantage: A necessary segmentation step was required to extract
landmark point or surfaces from images (which adds a possibility of
error)
Medical Imaging / VV → Image Registration
Classification of Methods→ Image Registration Algorithms >> Surface-base
Intensity-based Methods
● Intensity-based methods use pixel or voxel values
○ Advantage: It is not necessary to have a separate segmentation step to
extract points or surface of interest
○ A disadvantage is that images that are registered cannot be different
(e.g. recorded using different imaging modalities)
● This approach is most natural when images of the same kind (same
modality) are registered (e.g. CT to CT images)
● Intensity-based methods are various voxel similarity measures
Medical Imaging / VV → Image Registration
References for image registration:
● M. Sonka, J. M. Fitzpatrick, Eds., Handbook of Medical Imaging, Volume 2.
Medical Image Processing and Analysis, SPIE, 2000
● J. V. Hajnal, D. L. G. Hill, D. J. Hawkes, Eds., Medical Image Registration,
CRC Press, 2001
Medical Imaging / VV
● Data is discretized in space and / or time.
● Finite number of samples
○ The continuous signal is usually known only at a few points (data
points)
○ In general, data is needed in between these points
● By interpolation we obtain a representation that matches the function at the
data points
○ Evaluation at any other point possible
● Reconstruction of signal at points that are not samples
● Assumptions needed for reconstruction
○ Often smooth functions
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Given irregularly distributed positions without connectivity information
● Problem: obtain connectivity to find a “good” triangulation
● For a set of points there are many possible triangulations
○ A measure for the quality of a triangulation is the aspect ratio of the so-
defined triangles
○ Avoid long, thin ones
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Scattered data triangulation
○ A triangulation of data points consists of
■ Vertices (0D) = S
■ Edges (1D) connecting two vertices
■ Faces (2D) connecting three vertices
● A triangulation must satisfy the following criteria
○ faces = conv(S), i.e. the union of all faces including the boundary is the
convex hull of all vertices
○ The intersection of two triangles is either empty, or common vertex, or a
common edge, or a common face (tetrahedra)
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Triangulation with
Are NOT valid
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● How to get connectivity/triangulation from scattered data?
○ Voronoi diagram
○ Delaunay triangulation
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Voronoi Diagram
○ For each sample every point within a Voronoi region is closer to it than
to every other sample
○ Given: a set of points for and a distance function
dist(x,y)
○ The Voronoi diagram Vor(x) contains for each point a cell with
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Voronoi cells
○ The half space is separated by the perpendicular bisector
between and
○ contains
○ Voronoi cell:
Voronoi cells are convex
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Delaunay graph Del(x)
○ The geometric dual (topologically equal) of
the Voronoi diagram Vor(x)
○ Points in X are nodes
○ Two nodes and are connected iff the
Voronoi cells V( ) and V( ) share same
edge
● Delaunay cells are convex
● Delaunay triangulation = triangulation of the
Delaunay graph
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Delaunay triangulation in 2D
○ Three points , , in X belong to a face from Del(X) iff no further
point lies inside the circle around
● Two points form an edge iff there is a circle around
that does not contain a third point from X
● For each triangle the circumcircle does not contain any
other sample
● Maximizes the smallest angle
● Maximizes the ratio of (radius of incircle)/(radius of
circumcircle)
● It is unique (independent of the order of sample) for all but
some very specific cases
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Local Delaunay property
Medical Imaging / VV
Voronoi Diagrams and Delaunay Triangulation
● Algorithms for Delaunay triangulations
○ Edge flip algorithm
○ Plane-sweep algorithm for finding an initial triangulation
○ Bowyer-Watson algorithm
○ Other techniques:
■ Radial sweep
■ Intersecting halfspaces
■ divide and conquer (merge-based or split-based)
Medical Imaging / VV
Univariate Interpolation
● Univariate Interpolation: Interpolation for one variable
○ Nearest neighbor (0 order)
○ Linear (first order)
○ Smooth (higher order)
Medical Imaging / VV
Univariate Interpolation
● Taylor interpolation
● Basic functions: monom basis (polynomials)
● is m+1 dimensional vector space of all polynomials with
maximum degree m
● Coefficients with
● Representation problem
● Interpolation problem
f = samples
c = coefficients (to be solved)
Medical Imaging / VV
Univariate Interpolation
● Properties of Taylor interpolation
○ Unique solution
○ Numerical problems / inaccuracies
○ Complete system has to be solved again if a single value is changed
Medical Imaging / VV
Medical Imaging / VV Other basis functions result in other interpolations scheme:
● Lagrange interpolation
● Newton interpolation
● Bernstein basis: Bezier curves (approximation)
● Hermite basis
Medical Imaging / VV ● Piecewise linear interpolation
○ Simplest approach (except for nearest-neighbor sampling)
○ Fast to compute
○ Often used in visualization applications
○ continuity at segment boundaries
● Data points:
● For any point x with
described be local coordinate
that is
evaluate
Medical Imaging / VV ● First approach
○ Replace differential by “finite differences”
○ Note that approximating the derivative by
causes subtractive cancellation and large rounding errors for small h
● Second approach
○ Approximate/interpolate (locally) by differentiable function and differentiate this function
Medical Imaging / VV ● Finite differences on uniform grids with grid size h (1D case)
Medical Imaging / VV ● Finite differences on uniform grids with grid size h (1D case)
Forward differences
Backward differences
Central differences
● Error estimation
○ Forward/backward differences are first order
○ Central differences are second order
Medical Imaging / VV ● 2D or 3D uniform or rectangular grids
○ Partial derivatives
● Same as in univariate case along each coordinate axis
● Example: gradient in a 3D uniform grid
Medical Imaging / VV
Medical Imaging / VV ● Volume rendering techniques
○ Techniques for 2-D scalar fields
○ Indirect volume rendering techniques (e.g. surface fitting)
■ Convert/reduce volume data to an intermediate representation (surface
representation), which can be rendered with traditional techniques
○ Direct volume rendering
■ Consider the data as a semi-transparent gel with physical properties and directly get a
3D representation of it
Medical Imaging / VV ● Slicing:
○ Display the volume data, mapped to colors, on a slice plane
● Isosurfacing:
○ Generate opaque/semi-opaque surfaces
● Transparency effects:
○ Volume material attenuates reflected or emitted light
Medical Imaging / VV
Medical Imaging / VV ● Direct volume rendering techniques
○ Direct volume rendering allows for the “global” representation integrating physical
characteristics
○ But prohibits interactive display due to its numerical complexity, in general
● Indirect volume rendering techniques
○ Often result in complex representations
○ Pre-processing the surface representation might help
○ Use graphics hardware for interactive display
● Goal
○ Integrate different techniques in order to represent the data as”good” as possible
○ But, keep in mind that the most correct method in terms of physical realism must not be the
most optimal one in terms of understanding the data
Medical Imaging / VV ● Different grid structures:
○ Structured: uniform, rectilinear, curvilinear
○ Unstructured
○ Scattered data
Medical Imaging / VV ● Pixel (picture element)
● Voxel ( volume element)
○ Values are constant within a region around a grid point
● Cell
○ Values between grid points are resampled by interpolation
Medical Imaging / VV Classification
● Color table for volume visualization
● Maps raw voxel value into presentable entities:
○ color, intensity, opacity, etc.
● Transfer function
● Goals and issues:
○ Empowers user to select “structures”
○ Extract important features of the data set
○ Histogram can be a useful hint
○ Often interactive manipulation of transfer functions needed
Medical Imaging / VV Examples of different transfer functions
Medical Imaging / VV Most widely used approach for transfer functions:
● Assign to each scalar value a different color value
● Assignment via transfer function T
○ T : scalar value → color value
● Common choice for color representation : RGBA
● Alpha value is very important, describes opacity
● Code color values into a color lookup table
● On-the-fly update of color LUT
Medical Imaging / VV
Medical Imaging / VV Shading
Simulate reflection of light
Simulate effect on color
We want to make use of the human visual system’s
ability to efficiently deal with shaded objects
Medical Imaging / VV Shading
Medical Imaging / VV Shading
Medical Imaging / VV Shading
Medical Imaging / VV
Medical Imaging / VV
References:
● McCormick, B. H. (1988). Visualization in scientific computing. ACM SIGBIO Newsletter, 10(1),
15–21. doi:10.1145/43965.43966
● Bartz, D., & Preim, B. (2011). Visualization and Exploration of Segmented Anatomic Structures. In
T. M. Deserno (Ed.), Biomedical Image Processing. Berlin, Heidelberg: Springer Berlin Heidelberg.
doi:10.1007/978-3-642-15816-2
● Visualization and Interactive Systems course from University of Stuttgart