Post on 09-Apr-2018
2/24/2012
1
Aaron Ponti
Anatomy SkillsImage processing fundamentals
Image processing and analysis circle
Image processing fundamentals-- Definitions
Definitions• Digital image
• Image resolution• Grayscale resolution• Spatial resolution
• Image data types• Binary (1 bit)• Intensity (8 bit, gray)• Indexed (8 bit, color)• RGB (24 bit, color)
• Color spaces
2/24/2012
2
Definitions• Digital image
• Image resolution• Grayscale resolution• Spatial resolution
• Image data types• Binary (1 bit)• Intensity (8 bit, gray)• Indexed (8 bit, color)• RGB (24 bit, color)
• Color spaces
Digital image• A digital image is a discrete function defined over a rectangular grid
(lattice) representing the characteristics of the objects being imaged.
Sampling (digitization): example in 1D.
sample
continuous (analog) signal
sampling interval
Digital image Digital image
• In 2D images, each grid element, or pixel(picture element), is defined as a location and a value representing the characteristic of the object in that location.
• In 3D images, the pixel is called voxel(volume element).
• A 3D image is just a stack of 2D images.
2/24/2012
3
Definitions• Digital image
• Image resolution• Grayscale resolution• Spatial resolution
• Image data types• Binary (1 bit)• Intensity (8 bit, gray)• Indexed (8 bit, color)• RGB (24 bit, color)
• Color spaces
Image resolution The resolution of an image is a measure of the fidelity of the representation
of the original scene.
Resolution is related firstly to the characteristics of the imaging system and secondly to the number of pixels (i.e. spatial resolution) and the range of brightness values (i.e. grayscale resolution) that are used for digitization.
The grayscale resolution of an image is expressed as its bit depth.
The maximum number of brightness (i.e. gray) levels in an n-bit image is 2n.
Image resolution ::Grayscale resolution
Image resolution ::Grayscale resolution
2/24/2012
4
Image resolution ::Spatial resolution Image resolution summary
Definitions• Digital image
• Image resolution• Grayscale resolution• Spatial resolution
• Image data types• Binary (1 bit)• Intensity (8 bit, gray)• Indexed (8 bit, color)• RGB (24 bit, color)
• Color spaces
Image data types ::Binary
2 “gray values”: black (0) and white (1)
Binary values (0 or 1)
1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 1 1 1 11 1 1 1 0 0 0 1 1 1 11 1 1 1 1 1 0 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1
2/24/2012
5
178 181 182 182 183 183 186 181 185 185 186 185 186 …178 182 182 183 184 183 183 182 182 182 188 184 186 …181 182 184 185 184 181 183 185 187 184 183 189 186 …186 186 184 187 185 183 189 185 184 186 190 187 186 …185 182 183 189 185 187 187 186 187 187 188 190 185 …184 183 185 185 187 185 189 186 189 189 189 193 193 …183 184 187 182 184 187 189 185 190 188 193 199 198 …183 182 181 186 190 189 195 191 187 162 156 116 30 …185 184 191 192 209 195 198 168 102 49 22 15 11 …187 187 187 194 197 173 77 25 19 19 26 17 11 …190 187 190 191 158 37 15 14 20 20 15 12 11 …187 182 180 127 32 16 13 16 14 12 13 11 14 …184 186 172 100 20 13 15 18 13 18 14 15 10 …186 190 187 127 18 14 15 14 12 10 11 11 10 …189 192 192 148 16 15 11 10 10 9 9 9 9 …192 195 181 37 13 10 10 10 10 10 9 9 9 …189 194 54 14 11 10 10 10 9 8 9 8 9 …194 140 19 16 11 11 10 10 9 9 9 9 9 …192 88 12 11 11 10 10 10 9 9 9 9 9 …194 37 10 10 9 9 9 9 9 9 9 8 8 …… … … … … … … … … … … … … …
Image data types :: Intensity
256 gray values
Intensity (brightness)
Image data types :: Indexed
256 colors
colormap = 1.0000 1.0000 1.0000 11.0000 1.0000 0 21.0000 0 1.0000 31.0000 0 0 4
0 1.0000 1.0000 50 1.0000 0 60 0 1.0000 70 0 0 8
0.5176 0 0 90.4510 0 0 100.3882 0 0 110.5490 0.0627 0.0627 120.4824 0.0627 0.0627 130.4196 0.0627 0.0627 140.5490 0.0941 0.0941 15... ... ... ...
145 140 121 89 78 42 117 87 117 116 ...140 140 140 86 86 157 114 137 135 117 ...105 140 145 150 137 135 135 135 126 137 ...86 149 140 126 135 114 150 157 135 114 ...137 114 117 137 137 135 114 114 114 117 ...140 121 117 137 137 117 142 142 157 142 ...129 137 161 135 135 117 161 149 157 142 ...140 121 89 156 142 87 14 86 137 114 ...114 86 78 138 138 138 156 117 137 114 ...150 157 157 156 156 142 156 117 135 157 ...... ... ... ... ... ... ... ... ... ... ...
Indices
Image data types ::RGB (24 bit)
3 x 256 “gray values”: 3 x 8 bit = 24 bit image
8 bit gray (→ red)
8 bit gray (→ blue)8 bit gray (→ green)
3 channels RGB, 32 bit color
Definitions• Digital image
• Image resolution• Grayscale resolution• Spatial resolution
• Image data types• Binary (1 bit)• Intensity (8 bit, gray)• Indexed (8 bit, color)• RGB (24 bit, color)
• Color spaces
2/24/2012
6
The Lab color space isdesigned to approximatehuman vision. It aspiresto perceptual uniformity,and its L componentclosely matches humanperception of lightness.
The RGB color model isan additive color model inwhich red, green, and bluelight are added together invarious ways to reproducea broad array of colors.
HSL and HSV are the twomost common cylindrical-coordinate representations ofpoints in an RGB colormodel, which rearrange thegeometry of RGB in anattempt to be moreperceptually relevant thanthe Cartesian representation.
Color spaces
Image processing fundamentals-- Grayscale image processing basics
Grayscale image processing basics• Color lookup tables (LUT)
• Image contrast
• Image histogram
• Pixel statistics• Profile statistics• Region-of-interest statistics
• Fundamental spatial image processing tools• Neighbors and connections• (Matrix) transformations: scaling, rotation, other transforms• Interpolation
Grayscale image processing basics• Color lookup tables (LUT)
• Image contrast
• Image histogram
• Pixel statistics• Profile statistics• Region-of-interest statistics
• Fundamental spatial image processing tools• Neighbors and connections• (Matrix) transformations: scaling, rotation, other transforms• Interpolation
2/24/2012
7
Color lookup tables (LUT)• In biomedical imaging it is common to apply false colors while retaining
the original 8 bit scale.
Colored SEM image of soybean cyst nematode and egg. The color makes the image easier for non-specialists to view and understand the structures and surfaces revealed in micrographs.
• Color can also be used to convey a visual meaning to the values or measurements in the image.
Color lookup tables (LUT)
Heat map generated from DNA microarray data reflecting gene expression values in several conditions
Differences in blood and oxygen levels in the brain shown by fMRI maps.
Grayscale image processing basics• Color lookup tables (LUT)
• Image contrast
• Image histogram
• Pixel statistics• Profile statistics• Region-of-interest statistics
• Fundamental spatial image processing tools• Neighbors and connections• (Matrix) transformations: scaling, rotation, other transforms• Interpolation
Image contrast• Contrast is a measure of brightness difference, both globally (for the whole
image) and locally (in neighboring regions). One possible way to express it is:
C = ( IS – IB ) / IB
2/24/2012
8
Grayscale image processing basics• Color lookup tables (LUT)
• Image contrast
• Image histogram
• Pixel statistics• Profile statistics• Region-of-interest statistics
• Fundamental spatial image processing tools• Neighbors and connections• (Matrix) transformations: scaling, rotation, other transforms• Interpolation
Image histogram• The grayscale image histogram is a way of illustrating the distribution of
gray levels in an image: it shows how many pixels have particular gray values (or gray value ranges: “histogram bins”).
• The histogram is a very important tool for performing image enhancement.
Dynamic range ( 0 – 255 for 8 bit)
Image histogram• Color (RGB) images have three independent channels, displayed in Red,
Green and Blue, respectively.• Each channel has its own histogram.
Grayscale image processing basics• Color lookup tables (LUT)
• Image contrast
• Image histogram
• Pixel statistics• Profile statistics• Region-of-interest statistics
• Fundamental spatial image processing tools• Neighbors and connections• (Matrix) transformations: scaling, rotation, other transforms• Interpolation
2/24/2012
9
Pixel statistics ::Profile statistics (plots)
Profile
Pixel statistics :: ROI statistics
Region of interest (ROI)
SquareCircularFree-formManually drawn(Semi-)automatically extracted…
Grayscale image processing basics• Color lookup tables (LUT)
• Image contrast
• Image histogram
• Pixel statistics• Profile statistics• Region-of-interest statistics
• Fundamental spatial image processing tools• Neighbors and connections• (Matrix) transformations: scaling, rotation, other transforms• Interpolation
Common pixel neighborhoods
4-connected(2D)
6-connected(3D)
8-connected(2D)
18-connected(3D)
6-connected(2D)
26-connected(3D)
2/24/2012
10
Matrix transformations
scaling
rotationfree transform
Interpolation Interpolation is a method of constructing new data points within the
range of a discrete set of known data points.
Used e.g. when scaling or generally transforming an image. There are several interpolation methods: linear, polynomial, spline,…
92 72 54
92 72 5484 63Simplest linear interpolation.
Image processing fundamentals-- The five classes of image processing
The five classes of image processing• Image enhancement
• Image histogram operations
• Spatial domain filtering
• Frequency domain filtering
• Image restoration• Deconvolution
• Correction of geometrical distortions
• Correction of gray-level inhomogeneities
• Image analysis• Segmentation
• Classification
• Image compression
• Image synthesis• Registration
• Visualization
• 3D rendering
2/24/2012
11
The five classes of image processing-- Image enhancement
Image enhancement• Used to improve some aspects of the quality of an image:
• Increase contrast or and/or brightness
• Sharpen details
• Remove noise
• Image enhancement operations
• Contrast enhancement
• Spatial filtering
• Frequency filtering
• Often performed interactively
• Often performed as preprocessing step in an automated image analysis operation
• Result is often subjective, but quantitative measures of image quality do exist (i.e. contrast to noise ratio)
Image histogram operations :: Contrast enhancement (gray)
Low contrast image
Histogram
Datatype (8 bit) dynamic range: 0 – 255
Image histogram operations ::Contrast enhancement (gray)
Actual dynamic rangeLow contrast image
Histogram
Datatype (8 bit) dynamic range: 0 – 255
2/24/2012
12
Image histogram operations ::Contrast enhancement (gray)
Actual dynamic rangeLow contrast image
Histogram
Datatype (8 bit) dynamic range: 0 – 255
Histogram stretching (linear mapping)
Image histogram operations ::Contrast enhancement (gray)
Actual dynamic rangeLow contrast image
Histogram
Datatype (8 bit) dynamic range: 0 – 255
Histogram stretching (linear mapping)
Adaptive histogram equalization
(local)
Histogram stretching
(global)
Image histogram operations :: Contrast enhancement (gray)
Histogram equalization
(global)
Original
Image histogram operations ::Contrast enhancement (color)• Contrast enhancement of color images is typically done by transforming an
image to a color space that has image intensity as one of its components (e.g. Lab) and then work on the luminosity layer ‘L’ of the image.
• Manipulating luminosity affects the intensity of the pixels, while preserving the original colors.
2/24/2012
13
Image histogram operations ::Contrast enhancement (color)
Original Histogram stretching
Histogram equalization Adaptivehistogram equalization
(global)
(global) (local)
Spatial filtering operations ::Rank filters (nonlinear filters)• Spatial filters are operations applied to a pixel using information taken from
the neighborhood of that pixel.
• In rank (or ordered) filtering, the gray values of pixels within a defined neighborhood around the pixel of interest are arranged in a list in ascending order. The new value of for the pixel of interest is the value at the required rank position in the list.
• Common rank filters:• maximum ( rank = n )• minimum ( rank = 1 )• median ( rank = n/2 )• range ( maximum – minimum )
• The outcome of the filtering depends on the size and shape of the neighborhood.
Rank filters ::Median filter
Original image Added salt-and-pepper noise Median filtered image
Rank filters ::Median filter
Original image Added salt-and-pepper noise Median filtered image
The rank filter causes some (limited) blurriness in the image.
2/24/2012
14
Spatial filtering operations ::Convolution filters (linear filters)• Like rank filtering, convolution filter also replaces a pixel value with a new
value obtained from the pixel neighborhood. But there are a few more calculations involved.
• A square kernel is defined to represent the neighborhood around the pixel. Each location in the kernel is associated a numerical value, called weight.
a b c
d e f
g h i
• The kernel is moved across the image, and the pixel value under the center of the kernel is replaced by the weighted sum of the surrounding pixels.
Kernel size can vary a lot,depending on the application.This is a 3x3 kernel.
Spatial filtering operations
Convolution filters
a b c
d e f
g h i
A B C
F G H
K L M
D E
I J
N O
P Q R
U V W
S T
X Y
A B C
F cG bH
K fL eM
D EaI JdN O
P iQ hR
U V W
gS T
X Y
A B C
F aG bH
K dL eM
D EcI JfN O
P gQ hR
U V W
iS T
X Y
kernel image
convolution
correlation
Mirrored in case of convolution!
Used to find similarities between two images.
Spatial filtering operations• The convolution kernel is mirrored.
• For most applications, the kernel is symmetric, and mirroring does not change it. Many (lazy) convolution implementations indeed use correlation.
Convolution: M’ = iG + hH + gI + fL + eM + dN + cQ + bR + aS
Correlation: M’ = aG + bH + cI + dL + eM + fN + gQ + hR + iS
Spatial filtering operations
919191919191919191
h
3x3 average filter
0.07510.12380.07510.12380.20420.12380.07510.12380.0751
h
3x3 Gaussian filter
111181111
h
High-pass filter
Examples of convolution filters
2/24/2012
15
Spatial filtering operations
121/1...121/1121/1121/1...............121/1...121/1121/1121/1121/1...121/1121/1121/1121/1...121/1121/1121/1
h
11x11 average filter
101-101-101-
h
PrewittVertical edge
111000111
h
PrewittHorizontal edge
Examples of convolution filters
• Given the same kernel size, convolution filters introduce more blurring than rank filters.
Spatial filtering operations
Low-pass filterGaussian kernel
Band-pass filterDifference of Gaussians kernel
High-pass filter1 - Gaussian kernel
• Hybrid filters may include both rank and convolution steps, or involve other image processing operations.
• Adaptive filters perform a different operation depending on the image content in the region in which they are being applied.• For instance, an adaptive filter could be designed to blur only those regions of an
image that do not contain an edge (e.g. anisotropic filtering).
Spatial filtering operations :: Hybrid filters and adaptive filters
Frequency filtering has an effect equivalent to convolution filtering, but the approach is different.
Shapes in images are made up of changes in gray levels across the image, from dark to light and back to dark. The rate of this change is called spatial frequency.
A real image will be much more complex and contain many more frequencies. However, we can say that: high spatial frequencies correspond to fine detail, such as noise and edges
low spatial frequencies correspond to larger objects with fairly uniform gray values.
Frequency domain filtering ::Spatial and frequency domains
Increasing spatial frequency
2/24/2012
16
• A high-pass filter preserves high frequencies.
• A low-pass filter preserves low frequencies.
• A band-pass (or notch) filter preserves a particular band, or range, or frequencies.
Frequency domain filtering ::Types of frequency filters
Frequency domain filtering ::Approach to frequency filtering• Generate a representation to show which spatial frequencies are present in
an image:• i.e. generate a frequency spectrum using Fourier transformation.
• Remove selected frequencies from this representation.
• Reverse the process to get back to the image (spatial representation):• i.e. apply the inverse Fourier transformation.
Frequency domain filtering ::Advantages of frequency domain filtering• It is possible to perform operations that are difficult in the spatial domain,
such as removing or enhancing only specific frequencies in the image.
• Periodic patterns can be selectively removed or enhanced.
• For operations that require larger kernel sizes in the spatial domain (especially in 3D), frequency domain filtering is computationally faster.
Frequency domain filtering
Increasing frequency
Increasingfrequency
2/24/2012
17
Frequency domain filtering
Fourier transform
Frequency domain filtering ::Low-pass filtering
InverseFourier transform
Set high frequencies to 0.
RemarkIn practice, one does not create sharp cut-offs in frequency domain, since this creates ringing artifacts that appear as spurious signals near sharp transitions in a signal, i.e. they appear as "rings" near edges.
Frequency domain filtering ::High-pass filtering
InverseFourier transform
Set low frequencies to 0.
Frequency domain filtering :: Band-pass filtering
InverseFourier transform
Preserve frequencies only in this band.
2/24/2012
18
Frequency domain filtering
Fourier transform ?
Frequency domain filtering
Frequency domain filteringFrequency domain filtering ::Convolution theorem
Convolution in direct space=
multiplication in frequency (Fourier) space
Inverse filtering
Fourier transform
Inverse Fourier transform
2/24/2012
19
The five classes of image processing-- Image analysis
Image analysis• The goal of image analysis is to extract and assign identities to objects in
images, and then extract some form of (numerical) measurement from them.
• Operations:
• Image segmentation
• Object classification
• Image arithmetic
• Binary operations
• Examples of measurements:
• Length, area, volume, density,…
• Usually statistical analyses are then performed on the obtained measurements
Image analysis ::Segmentation• Image segmentation refers to the process of partitioning a digital image
into multiple sets of pixels with the goal of simplifying and/or changing the representation of an image into something that is more meaningful and easier to analyze.
• Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images.
• More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics.
Image analysis ::Classification• Classification assigns objects obtained from segmentation into categories
based on quantitative information on one or more characteristics inherent in the items (referred to as traits, variables, characters, etc).
• There are many classification algorithms divided in several categories:• Linear classifiers (Fisher's linear discriminant, Logistic regression, Naive Bayes classifier, Perceptron,
Support vector machines)
• Quadratic classifiers• k-nearest neighbor• Boosting• Decision trees (Random forests)
• Neural networks• Bayesian networks• Hidden Markov models
• The separation between segmentation and classification is quite blurred in all but the simplest segmentation approaches!
2/24/2012
20
Segmentation• There are three approaches to segmentation:
• Manual
• Semiautomatic
• Automatic
Manual segmentation• Advantages:
• Simple, provided there are suitable tools
• Disadvantages:
• Very time-consuming
• Subject to human error
• Subjective
• Poor (intra-observer) reproducibility
Manual segmentation• Manual segmentation involves an expert observer outlining the object of
interest in the image.
• The computer simply provides tools to help with drawing.
Manual segmentation ::Simple thresholding• The user chooses an intensity threshold level in the histogram.
• The software shows a preview to facilitate the selection of the threshold.
2/24/2012
21
Manual segmentation :: Simple thresholding• After the threshold is set, all values below the threshold are set to 0.
• The values above the threshold are either left unchanged, or set to white (e.g. 255).
Semi-automatic segmentation• In semi-automatic segmentation, the observer helps the segmentation by
providing the software with a rough selection to refine.
• Region growing
• Active contours, snakes, and deformable surfaces
Semi-automatic segmentation ::Region growing• The operator defines a range of gray levels that represent the region, and
chooses a pixel (the seed point) that is known to be part of the region.
• All the pixels that are in the intensity range and are connected to the seed point are selected as being part of the region.
• Alternatively, the software returns the boundary of the region as an editable line.
Semi-automatic segmentation ::Region growing
Range = 10 gray values
2/24/2012
22
Semi-automatic segmentation ::Active contours, deformable surfaces• Active contour model, also called snakes ( = deformable splines), is a
framework for delineating an object outline from a possibly noisy 2D image.
• This framework attempts to minimize an energy associated to the current contour as a sum of an internal and external energy:
• The external energy is supposed to be minimal when the snake is at the object boundary position (e.g. low energy values for high gradient values);
• The internal energy is supposed to be minimal when the snake has a shape which is supposed to be relevant considering the shape of the sought object (e.g. high energy to elongated contours (elastic force) and to bended/high curvature contours (rigid force), considering the shape should be as regular and smooth as possible).
• The framework also allows for training.
Semi-automatic segmentation ::Active contours / snakes
Range = 10 gray values
Semi-automatic segmentation ::Deformable surfaces
Range = 10 gray values
Automatic segmentation • Automatic (computational) segmentation methods are areas of
active research.
• We will only look into a very simple automatic segmentation algorithm: Otsu’s method.
2/24/2012
23
Automatic segmentation ::Otsu’s method• Otsu's method is used to automatically perform histogram shape-based
image thresholding.
• The algorithm assumes that the image to be thresholded contains two classes of pixels (e.g. foreground and background) then calculates the optimum threshold separating those two classes so that their combined spread (intra-class variance) is minimal.
Range = 10 gray values
threshold
Ideal case
Automatic segmentation ::Otsu’s method
Range = 10 gray values
This image probably contains more than two classes of pixels.
There are extensions of Otsu’s algorithm that find multiple thresholds.
Image arithmetic Image arithmetic is the implementation of standard arithmetic operations,
such as addition, subtraction, multiplication, and division, on images.
Image arithmetic has many uses in image processing both as a preliminary step in more complex operations and by itself. Examples: Image subtraction can be used to detect differences between two or more images of the
same scene or object.
Image subtraction or division are used for shading corrections. Image multiplication is used for masking. Image addition is used for visualizing composite images (where each channel has a
color which is not exclusively either Red, Green, or Blue).
Binary and morphological operations Binary and morphological operations can be used to perform common
image processing tasks, such as contrast enhancement, noise removal, thinning, skeletonization, filling, and segmentation.
Morphology is a broad set of image processing operations that process images based on shapes.
Morphological operations apply a structuring element to an input image, creating an output image of the same size.
In a morphological operation, the value of each pixel in the output image is based on a comparison of the corresponding pixel in the input image with its neighbors.
By choosing the size and shape of the neighborhood, you can construct a morphological operation that is sensitive to specific shapes in the input image.
2/24/2012
24
Binary and morphological operations ::Example: Image dilation Dilation is one of the two basic operators in the area of morphology, the
other being erosion.
It is typically applied to binary images, but there is a version for grayscale images.
Dilation gradually enlarges the boundary of regions of foreground pixels (i.e. white pixels): thus area of foreground pixels grow in size, while holes within these regions become smaller.
The dilation operator takes two pieces of data as inputs. The first is the image which is to be dilated. The second is a (usually small) set of coordinate points known as a structuring element, for example:
1 1 1
1 1 1
1 1 1
0 1 0
1 1 1
0 1 0
Binary and morphological operations ::Example: Image dilation Mathematically, dilation is defined as:
This basically means, place the structuring element B on some position on the image A. If the pixel value of A under the center of B is 1, replace the n x n pixels in A with the structuring element B.
Example:
Binary and morphological operations
Problem: detecting (i.e. segmenting) a cell
Binary and morphological operations
A first rough segmentation is given by a threshold of gradient of the image, to give the strongest edges.
Binary gradient mask
2/24/2012
25
Binary and morphological operations
Image dilation: the binary gradient mask is dilated.
Dilated gradient mask
Binary and morphological operations
Image flood-fill: the dilated gradient mask shows the outline of the cell quite nicely, but there are still holes in the interior of the cell that we flood-fill.
Dilated gradient mask with filled holes
Binary and morphological operations
Here we got rid of the cell at the border. Admittedly, this is not strictly a morphological operation…
Cleared border image
Binary and morphological operations
Image erosion: finally, in order to make the segmented object look natural, we smoothen the object by eroding the image twice with a diamond structuring element.
Segmented image
2/24/2012
26
Binary and morphological operations
An alternate method for displaying the segmented object would be to place an outline around the segmented cell.
Segmented image with outline
The five classes of image processing-- Image restoration
Image restoration• Required if the acquisition method (e.g. the microscope) causes:
• geometric distortion
• blurring
• gray-level inhomogeneities
Geometric distortionAlthough distortion can be irregular or follow many patterns, the most commonly encountered distortions are radially symmetric, arising from the symmetry of a lens. The radial distortion can usually be classified as one of two main types: Barrel distortion and Pincushion distortion.
In barrel distortion, image magnification decreases with distance from the optical axis. The apparent effect is that of an image which has been mapped around a sphere (or barrel).
In pincushion distortion, image magnification increases with the distance from the optical axis. The visible effect is that lines that do not go through the centre of the image are bowed inwards, like a pincushion.
2/24/2012
27
Blurring• Any imaging system has a blurring effect.
• If a point object is imaged, the resulting image is not a perfect point, but a spread-out version of that point.
• Mathematically, this blurring effect is described using the convolution operation:
• Every imaging system has a characteristic blurring function called point spread function (PSF): h(x)
Point object, whose dimensions are below the resolution limit.
Point spread function, lateral view.
Axial view.
Removing blur Deconvolution• Knowledge of the PSF of a system is useful because of the potential for removing
blurring by applying the inverse of convolution, deconvolution.
• We saw that convolution can be performed in frequency space. Can we deconvolve by inverse filtering, then?
Point spread function, lateral view.
Axial view.
f
h
F
H
G g
f
F -1(G/H)?
F -1(G)
F (f)
F (h)
Deconvolution :: Cookie cutter
XZ
Missing frequencies
“Sub-resolution train“Noise-free convolution and deconvolution
Deconvolution :: Deconvolving trains
2/24/2012
28
“Sub-resolution train“Noise-free convolution and deconvolution
Confocal
Widefield
Deconvolution :: Deconvolving trains
Confocal
Widefield
Restored confocal
Restored widefield
“Sub-resolution train“Noise-free convolution and deconvolution
Deconvolution :: Deconvolving trains
Deconvolution :: MLE
Extreme noise amplification!
Inverse filtering will never allow us to recover the true object function f.
Artifacts!
H = 0 at many places!
• Future promising algorithms are based on wavelets.
Deconvolution is performed in practice with iterative algorithms like the maximum likelihood estimation (MLE) algorithm.
Correction of gray-level inhomogeneities(shading correction)
Background estimation
• “Rolling ball”• Morphological Opening
Subtraction
2/24/2012
29
Correction of gray-level inhomogeneities(shading correction)
A correction is important, for instance, for stitching applications to prevent border effects.
Correction of gray-level inhomogeneities(shading correction)
A correction is important, for instance, for stitching applications to prevent border effects.
The five classes of image processing-- Image compression
Image compression• Image data files are often very large in terms of number of bytes
• Large disk space usage
• Slow transfer over networks
• Compression algorithms exists
• Lossless
• Lossy
2/24/2012
30
Image compression :: Lossy compression
100%, 64kb
Image compression ::Lossy compression
Jpeg, 80%, 12kb
Image compression ::Lossy compression
Jpeg, 50%, 7kb
Image compression ::Lossy compression
Jpeg, 10%, 3kb
2/24/2012
31
Image compression ::Lossy compression
Jpeg, 1%, 1kb
Image compression ::Lossless compression
100%, 64kb
Image compression ::Lossless compression
PNG, best compression, 48kb
The five classes of image processing-- Image synthesis
2/24/2012
32
Image synthesis• Image synthesis is the general term for bringing together information from
more than one image.
• Synthesis can be separated into two parts:
• Image registration, covering processes required to bring images into spatial alignment
• Visualization, which allows information (from the aligned datasets) to be viewed
• Reasons for performing image synthesis include:
• Assessment of disease progression or growth using series of images
• Combination of structural and functional information from different imaging modalities
• Comparison of corresponding regions in different individuals by matching both to a standard coordinate system
• Generation and analysis of atlases or templates representing the typical appearance in health or disease
• Arithmetical or statistical operations which require registered images
Image registration• How to map one onto the other?
Image 1
Image 2
Smaller, rotated version of image 1.
Image registration• The four steps of image registration:
• Feature extraction
• One needs to identify features that appear in both images. This can be done manually, by placing markers on the images, or automatically. All pixels intensities could also be used.
• Pairing (also called: “identifying correspondences”)
• Once features have been identified in both images, they need to be analyzed to determine which feature in one image matches which feature in the other image.
• The features can be fiducial landmarks or pixel intensities (or related values).
• The term similarity metric is the general way of describing a measure that is used in the process of pairing points with similar properties. Examples: correlation coefficient, mutual information, entropy
• Calculation of transformation
• The transformation is the mathematical operation that will give the best alignment (mapping) of all pairs of features. There arefour different types of transformations:
• Rigid: in a rigid transformation all parts of the object are assumed to move as a whole, e.g. translation, rotation
• Affine: in affine transformations, straight lines remain straight and parallel lines remain parallel: e.g. scaling, shearing
• Projective: projective transformation include further deformation. Straight lines remain straight, but parallel lines do not remain parallel.
• Curved or elastic (non-rigid): this is the most general case of transformation. Straight lines need not remain straight. It should be used with care.
• Application of transformation
• This step is often called matrix transformation or matrix operation.
• Applying the transformation will very often require interpolation.
Image registrationExample: rigid transformation
1. The user chooses some fiduciary marks by clicking on them2. The positions are refined by cross-correlation3. The (rigid) transformation matrix is obtained by Least-Square fitting4. Image 1 is transformed (mapped) onto image 2 by matrix transformation and interpolation.
11
22
33
4455
12
3
4
5
111222
443
334
555
2/24/2012
33
Image registration• End result of registration
Image registration example ::Atlas generation
Mouse hippocampus
A user draws the outline of the hippocampus overmany aligned slices to reconstruct the 3-D shape.
The same operation is performed on 3 mice, and the resulting 3D objects are registered on top of each other to generate an atlas (i.e. a model of the hippocampus).
1)2)
3)
Image registration example ::Correlative microscopy
Biocytin labelled cell, Z projection of 80 confocal planes. (200nm X-Y pixel size)
Same cell imaged in SBFSEM (98x98x30 microns)
Visualization• Visualization is the display of image data.
• Visualization in biomedical imaging needs specialized approaches:
• Visualized objects can have a huge range of scales, from single molecules and cells to body parts
• Different attributes of these objects can be visualized, including biophysical, biomechanical and physiological properties.
• The result of visualization can be used for diagnosis, treatment planning, rehearsal, assessment, and intra-operative guidance.
• There are several approaches to visualization:
• Slice mode
• Projection mode
• Volume mode
• Surface mode
2/24/2012
34
Visualization ::Slice mode
Fluorescence image of the mouse retina. Red: collagen IV, Green: GFAP (source Imaris demos)
Shown: one slice (focal plane) in the middle of the 3D acquisition.
Visualization ::Slice mode
Shown: all slices in one view.
Visualization ::Maximum intensity projection
Visualization ::Volume rendering
2/24/2012
35
Visualization ::Surface rendering
Visualization ::Surface rendering (fancy)
Last remarks
Last remarks• Every image processing action taken must make sense:
• Do not apply any filter just because like this the image looks better.
• Remember that applying the wrong filter can destroy real features in the image and also introduce spurious ones that actually DO NOT EXIST.
• Keep a complete protocol of the actions taken:
• The result (i.e. the figure in a paper) must be reproducible by you and by others.
• Make sure you know the characteristics of your acquisition system (e.g. a microscope), because this will define:
• The parameters for acquiring the images.
• The parameters for processing the images.
• Image processing was born to process different kinds of pictures than biomedical ones (which are diffraction-limited, with extremely low SNR, 3D or more, …)
• There is a lot of work going on extending standard image processing techniques for these new fields.
• Image processing of microscopy data requires a lot of statistical modeling, learning algorithms, and the like (which we haven’t touched in this introductory course).