1

Methods in Image Analysis – Lecture 2Local Operators and Global Transforms

CMU Robotics Institute 16-725

U. Pitt Bioengineering 2630

Spring Term, 2004

George Stetten, M.D., Ph.D.

2

Preface

• Some things work in n dimensions, some don’t.• It is often easier to present a concept in 2D.• I will use the word “pixel” for n dimensions.

3

Point Operators

• f is usually monotonic, and shift invariant.• Inverse may not exist.• Brightness/contrast, “windowing”.• Thresholding.• Color Maps.• f may vary with pixel location, eg., correcting for

inhomogeneity of RF field strength in MRI.

( ) ( )[ ]yxIfyxI ,,' =

4

Histogram Equalization

• A pixel-wise intensity mapping is found that produces a uniform density of pixel intensity across the dynamic range.

5

Adaptive Thresholding from Histogram

• Assumes bimodal distribution.• Trough represents boundary points between

homogenous areas.

6

Algebraic Operators

• Assumes registration.• Averaging multiple acquisitions for noise reduction.• Subtracting sequential images for motion detection, or

other changes (eg. Digital Subtractive Angiography).• Masking.

( ) ( ) ( )[ ]yxIyxIgyxI ,,,,' 21=

7

Re-Sampling on a New Lattice

• Can result in denser or sparser pixels.• Two general approaches:

– Forward Mapping (Splatting)

– Backward Mapping (Interpolation)

• Nearest Neighbor

• Bilinear

• Cubic

• 2D and 3D texture mapping hardware acceleration.

8

Neighborhood Operators

• Kernels• Cliques• Markov Random Fields• Must limit the relationships to be practical.

9

Convolution and Correlation

• Template matching uses correlation, the primordial form of image analysis.

• Kernels are mostly used for “convolution” although with symmetrical kernels equivalent to correlation.

• Convolution flips the kernel and does not normalize.• Correlation subtracts the mean and generally does

normalize.

10

Neighborhood PDE Operators

• For discrete images, always requires a specific scale.

• “Inner scale” is the original pixel grid.• Size of the kernel determines scale.• Concept of Scale Space, Course-to-Fine.

11

• Vector• Direction of maximum change of scalar intensity I.• Normal to the boundary.• Nicely n-dimensional.

Intensity Gradient

zyxzyx ˆˆˆˆˆˆ zyx IIIdz

dI

dy

dI

dx

dII ++=++=∇

12

• Scalar• Maximum at the boundary• Orientation-invariant.

Intensity Gradient Magnitude

€

∇I = Ix2 + Iy

2 + Iz2

( ) 1

2 =∇I ⋅ ˆ n

13

IxI

yI I∇

14

Classic Edge Detection Kernel (Sobel)

xI⇒⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−

101202101

yI⇒⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−− 121000121

15

Isosurface, Marching Cubes (Lorensen)

• 100% opaque watertight surface• Fast, 28 = 256 combinations, pre-computed

16

• Marching cubes works well with raw CT data.

• Hounsfield units (attenuation).• Threshold calcium density.

17

Direct shading from gradient (Levoy, Drebin)

• Voxels are blended (translucent).• Opacity proportional to gradient magnitude.• Rendering uses gradient direction as “surface” normal.

18

• Ixy = Iyx = curvature• Orientation-invariant.• What about in 3D?

Jacobian of the Intensity Gradient

⎥⎦

⎤⎢⎣

⎡=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

yyyx

xyxx

II

II

dy

Id

dxdy

Iddydx

Id

dx

Id

2

22

2

2

2

19

Viewing the intensity as “height”• Differential geometry: the surface and tangent plane • Example: cylindrical surface, curvature = 0.• Move in the y direction, no roll: Ixy = 0• Move in the x direction, no pitch: Iyx = 0

= Ix

= Iy

20

Laplacian of the Intensity

• Divergence of the Gradient.• Zero at the inflection point of the

intensity curve.

222

2

2

22

2

22

2

22

zzyyxx IIIdz

Id

dy

Id

dx

IdI ++=⎟⎟

⎠

⎞⎜⎜⎝

⎛+⎟⎟

⎠

⎞⎜⎜⎝

⎛+⎟⎟

⎠

⎞⎜⎜⎝

⎛=∇

I

Ix

Ixx

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−−−−−

111

181

111

21

Difference of Gaussian Operators (DOG)

• Conventionally, 2 concentric Gaussians of different scale.• Acts like a Laplacian,

22

Binomial Kernel

• Repeated averaging of neighbors => Gaussian by Central Limit Theorem.

1 1 1 1 2 1 1 3 3 1 1 4 6 4 1

1 1 2 4 2 3 9 9 3 4 16 24 16 4

1 2 1 3 9 9 3 6 24 3624 6

1 3 3 1 4 16 24 16 4

1 4 6 4 1

23

Binomial Difference of Offset Gaussian (DooG)

• Not the conventional concentric DOG• Subtracting pixels displaced along the x axis after repeated

blurring with binomial kernel yields Ix

-1 0 1 -1 -2 0 2 1 -1 -4 -6 -4 0 4 6 4 1

-2 -4 0 4 2 -4 -16 -24 -16 0 16 24 16 4

-1 -2 0 2 1 -6 -24 -36 -24 0 24 36 24 6

-4 -16 -24 -16 0 16 24 16 4

-1 -4 -6 -4 0 4 6 4 1

24

Boundary Profiles (Tamburo)

• Splatting in an ellipsoid along the gradient.

25

Boundary Profiles

• Fitting the cumulative Gaussian

distance along gradient

μ

σ σ

d d

p1

p2

sampled region ofprofile

μ

1 2intensity

I1

I2

26

Boundary profiles reduce error in location due to sampling.

27

Other basis sets for boundary kernel.

• Partial derivatives of the Gaussian• Wavelets• Statistical texture analysis.

28

Texture Boundaries

• Two regions with the same intensity but differentiated by texture are easily discriminated by the human visual system.

29

Ridges

• Attempt to tie the image structure together in a locally continual manner.

• Along some manifold in less than n dimensions.• Local maximum along the normal to the ridge.• Canny edge detector is a boundariness ridge.• A “core” is a medialness ridge.• Medialness within an object is the property of

being equidistant from two boundaries.

30

Global Transforms in n dimensions

• Geometric (rigid body)

– n translations and rotations.

• Similarity– Add 1 scale (isometric).

• Affine– Add n scales (combined with rotation => skew).

– Parallel lines remain parallel.

• Projection

⎟⎟⎠

⎞⎜⎜⎝

⎛2

n

31

Orthographic Transform Matrix

• Capable of geometric, similarity, or affine.• Homogeneous coordinates.• Multiply in reverse order to combine• SGI “graphics engine” 1982, now standard.

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡′′

1

10013,22,21,2

3,12,11,1

y

x

aaa

aaa

y

x

32

Translation by (tx , ty)

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡′′

1

100

10

01

1

y

x

t

t

y

x

y

x

Scale x by sx and y by sy

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡′′

1

100

00

00

1

y

x

s

s

y

x

y

x

33

• 2 x 2 rotation portion is orthogonal (orthonormal vectors).• Therefore only 1 degree of freedom, .

Rotation in 2D

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ −=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡′′

1

100

0cossin

0sincos

1

y

x

y

x

θθ

θθ

θ

34

• 3 x 3 rotation portion is orthogonal (orthonormal vectors).• 3 degree of freedom (dotted circled), , as expected.

Rotation in 3D

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

′′′

1

1000

0

0

0

13,32,31,3

3,22,21,2

2,12,11,1

z

y

x

aaa

aaa

aaa

z

y

x

⎟⎟⎠

⎞⎜⎜⎝

⎛2

n

35

• For X-ray or direct vision, projects onto the (x,y) plane.• Rescales x and y for “perspective” by changing the “1” in

the homogeneous coordinates, as a function of z.

Non-Orthographic Projection in 3D

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

′′′

1

100

0100

0010

0001

1

z

y

x

k

z

y

x

36

Anisotropic Scaling of Vectors (ITK Software Guide 4.25, 4.26)

dx1

dI

dx2

dI

itkCovarientVector objects are used to represent properties such as the gradient, since stretching the image in the x dimension lowers the x gradient component (slope).

€

dx1 > dx2 ⇒dI

dx1

<dI

dx2

itkVector

itkCovarientVector

dx1 dx2

dy

itkVector objects are used to represent distances between locations, velocities, etc.

dx1 dx2

dy