Mauricio CerdaICBM, CIMT, Faculty of Medicine

Universidad de Chilemauriciocerda@med.uchile.cl

www.scian.cl

Image Processing 1images, segmentation, shape

Outline

Castañeda V, Cerda M, 2014Computational methods for analysis of dynamic events in cell migration

Outline

1. Introduction

• Images• Segmentation

2. Descriptors

• Shape

• A digital image can be defined as a function over a discrete space

],[],[]1dim,0[]1dim,0[),(

),(

jiji

yx

yxfyxIyx

yxfI

=

-´-Î=

Digital Images

– A typical 2D image model isthe raster image: array(matrix) of pixeles in cartesian coordinates (x, y)

– A numeric value forbrightness (intensity) orcolor is associated to eachpixel

Aubert & Kornprobst (2006)Mathematical Problems In Image Processing

• Greyscale image

– A brightness (intensity) level is defined for each pixel

Digital Images

I(290,267) = 220

8 bit greyscale image

A n bit greyscale imageencodes up to 2nintensity values

Binaryvalue

Decimal value

0000 0000 0 (black)

0000 0001 1

0000 0010 2

0000 0011 3

0000 0100 4

0000 0101 5

0000 0110 6

0000 0111 7

0000 1000 8

… …

1111 1011 251

1111 1100 252

1111 1101 253

1111 1110 254

1111 1111 255 (blanco)

Digital Images

• RGB image

– Three channels for respective primary colors:Red, Green, Blue

• Other color spaces are HSV, LAB

Digital Images

• It is possible to define color tables (or lookup tables, LUTs) for visualization purposes. A grayscale image can bedisplayed using a green scale.

r g b 000::::0::0

012::::200::255

000::::0::0

Digital Images

• Segmentation

– The partitioning of a given image into regions of interest (ROIs) according to given criteria (e.g. color).

– After segmentation, further characterizations can be performed upon the resulting ROIs.

Shapiro LG and Stockman GC (2001):“Computer Vision”, pp 279-325New Jersey, Prentice-Hall

Image Analysis

Sky / FlatSol 3, Mars Pathfinder Mission

Dust / Horizon Final segmentation

…etc…

Segmentation

Images from NASA

Segmentation

Max Z-Project from confocal microscopy of Fundulus N. [data by German Reig 2015]

Segmentation

Problems

• Lack of absolute criteria or standards(Ground Truth, Gold Standard [1,2])

• Missing or erroneus information(e.g. non-specific markers in samples)

• What to do? A “good” (i.e. carefully performedand controlled) acquisition ease this process

[1] Jason D. Hipp et all. Tryggo: Old norse for truth: The real truth about ground truth. New insights into the challenges of generating ground truth maps for WSI CAD algorithm evaluation. Pathol. Inform 2012, 3:8

[2] Luc Bidaut, Pierre Jannin. Biomedical multimodality imaging for clinical and research applications: principles, techniques and validation. In Molecular Imaging:Computer Reconstruction and Practice (NATO Science for Peace and Security Series B: Physics and Biophysics), Springer, 2008, ISBN-13: 978-1402087516.

• Segmentation is the first step toward further quantifications

Parameter estimation…

Endoplasmic reticulum in a COS-7 cellO Ramírez, L Alcayaga (2012)

•Size: area, perimeter•Boundary: inflections, shape•Topology: connectivity, endpoints

Lipid monolayersJ Jara (2006), Fanani et al (2010)

Segmentation

Segmentation

1. Classic approaches (filters)– Thresholding– Matrix convolution filters + threshold– Mathematical morphology

2. Advanced approaches– Shape priors (pattern matching)– Deformable models (active contours)

• parametric• implicit

• Threshold filtersegmentation: ROIs (white) / background (black)

Intensity histograms

Segmentation – basics

• Convolution– Lots of filters based on this principle

http://en.wikipedia.org/wiki/Convolution

• Matrix convolution, in our case, is an operation between two matrices, namely…– the input image, I– a kernel, K

Adapted from James Matthews, 2002http://www.generation5.org/content/2002/convolution.asp

Segmentation – basics

17

• Intensity gradients (discrete approximation)

»¶¶xI

»¶¶yI

Segmentation – basics

flat + -

18

I = I(x, y)

x

y

19

IxIy

x

y

20

“Edgemap”|▼I|=|Ix|+|Iy|

x

y

• Intensity gradients (discrete approximation)

pixel1

),(),(

=D

Ä=D

-D+»

¶¶

x

IKxx

yxIyxxIxI

ïþ

ïý

ü

ïî

ïí

ì-=

000110000

Kx

ïþ

ïý

ü

ïî

ïí

ì-=

000010010

Kypixel1

),(),(

=D

Ä=D

-D+»

¶¶

y

IKyy

yxIyyxIyI

),( yxII =

Segmentation – basics

flat + -

ïþ

ïý

ü

ïî

ïí

ì-=

010141010

LK

Laplace

IKx

yyxfyxxfyxfyyxfyxxfI

yyyxfyxfyyxf

xyxxfyxfyxxfI

yI

xII

L Ä=D

D-+D-+-D++D+Ȅ

DD-+-D+

+D

D-+-D+Ȅ

¶¶

+¶¶

=Ñ

22

222

2

2

2

22

)(),(),(),(4),(),(

)(),(),(2),(

)(),(),(2),(

),( yxII =

• Kernels…

pixel1=D=D yx

Segmentation – basics

• Morphology based filters– Example: size selection

Input greyscale image After thresholding…

Size selectionHow to define a size-select algorithm?

Segmentation – basics

• Mathematical morphology

Segmentation – basics

Dilate

Erode

What is B?

• Mathematical morphology

Segmentation – basics

To open:

To close:

• “Some” times more information is needed in order to achieve a good segmentation (see additional material)

Segmentation – basics

Outline

27

1. Introduction

• Images• Segmentation

2. Descriptors

• Shape

Shape descriptors

Motivation

2D image and segmentations, Lisette Leyton’s Lab (BNI) Original image and 3D segmentation , Karina Palma (LEO, BNI).

Ex. 1Shape and structure analysis of fibroblast actin fibers (astrocytes)

Ex. 2Zebrafish parapineal organ neuron

28

1. Geometrical descriptors: location, perimeter, area, volume, curvature

2. Moments of morphology (order 0-2)3. Topology in computer science (skeletons)

Shape & topology descriptors…

29

Location, perimeter, area and volume partially describe an object, but not its shape.

Moments of morphology

Original image (2D),Prof. Lisette Leyton

30

How to quantify the amount of roundness of an object?

Variance

Moments of morphology

31

Axes U and V maximize the variance in U

Covariance

Positive covariance Zero covariance

We look for two parameters to characterize the binary ROI…

Moments of morphology

32

X

Y

α

L1Semi-major axis

L2

• If the variance/covariance matrix is diagonal for a certain rotation α,

• Semi-major axis length l is a function of second-ordermoments

Moments of morphology

33

L21 =

1

2

⇣�2xx + �2

yy +q

(�2xx � �2

yy)2 + 4(�2

xy)2⌘

<latexit sha1_base64="GV6QEwpSL22PFjxJjfT2zhS/afI=">AAACXnicbZHLSgMxFIYz473eRt0IboJFqIgyUwTdCKIbFy4UrAqdWjLxTBuauZicEYcwb+gLuNM3Ma1dtOpZ/ef8X0jOnyiXQqPvfzjuzOzc/MLiUm15ZXVt3dvYvNdZoTi0eCYz9RgxDVKk0EKBEh5zBSyJJDxEg8uh//AKSossvcMyh07CeqmIBWdoR13PXD81u8FZGCvGTVCZZkVDCTHSBg216CXM2ubtraIHE31ZVgehflFoGlPQ4RSy/9S0p44nkdHQ3qBEr4/7Xa/uH/mjon9FMBZ1Mq6brvcePme8SCBFLpnW7cDPsWOYQsElVLWw0JAzPmA9aFuZsgR0x4xCquhenCmKfaCjfpI1LNG6TCLLJAz7+rc3HP7ntQuMTztGpHmBkHKLWC8uJMWMDrOmz0IBR1lawbgS9pWU95lNGu2P1Oz6we9l/4r75lFg9e1x/fxiHMQi2SG7pEECckLOyRW5IS3CyZez4HjOhvPpzrur7voP6jrjM1tkqtztb3dos/Y=</latexit><latexit sha1_base64="GV6QEwpSL22PFjxJjfT2zhS/afI=">AAACXnicbZHLSgMxFIYz473eRt0IboJFqIgyUwTdCKIbFy4UrAqdWjLxTBuauZicEYcwb+gLuNM3Ma1dtOpZ/ef8X0jOnyiXQqPvfzjuzOzc/MLiUm15ZXVt3dvYvNdZoTi0eCYz9RgxDVKk0EKBEh5zBSyJJDxEg8uh//AKSossvcMyh07CeqmIBWdoR13PXD81u8FZGCvGTVCZZkVDCTHSBg216CXM2ubtraIHE31ZVgehflFoGlPQ4RSy/9S0p44nkdHQ3qBEr4/7Xa/uH/mjon9FMBZ1Mq6brvcePme8SCBFLpnW7cDPsWOYQsElVLWw0JAzPmA9aFuZsgR0x4xCquhenCmKfaCjfpI1LNG6TCLLJAz7+rc3HP7ntQuMTztGpHmBkHKLWC8uJMWMDrOmz0IBR1lawbgS9pWU95lNGu2P1Oz6we9l/4r75lFg9e1x/fxiHMQi2SG7pEECckLOyRW5IS3CyZez4HjOhvPpzrur7voP6jrjM1tkqtztb3dos/Y=</latexit><latexit sha1_base64="GV6QEwpSL22PFjxJjfT2zhS/afI=">AAACXnicbZHLSgMxFIYz473eRt0IboJFqIgyUwTdCKIbFy4UrAqdWjLxTBuauZicEYcwb+gLuNM3Ma1dtOpZ/ef8X0jOnyiXQqPvfzjuzOzc/MLiUm15ZXVt3dvYvNdZoTi0eCYz9RgxDVKk0EKBEh5zBSyJJDxEg8uh//AKSossvcMyh07CeqmIBWdoR13PXD81u8FZGCvGTVCZZkVDCTHSBg216CXM2ubtraIHE31ZVgehflFoGlPQ4RSy/9S0p44nkdHQ3qBEr4/7Xa/uH/mjon9FMBZ1Mq6brvcePme8SCBFLpnW7cDPsWOYQsElVLWw0JAzPmA9aFuZsgR0x4xCquhenCmKfaCjfpI1LNG6TCLLJAz7+rc3HP7ntQuMTztGpHmBkHKLWC8uJMWMDrOmz0IBR1lawbgS9pWU95lNGu2P1Oz6we9l/4r75lFg9e1x/fxiHMQi2SG7pEECckLOyRW5IS3CyZez4HjOhvPpzrur7voP6jrjM1tkqtztb3dos/Y=</latexit><latexit sha1_base64="GV6QEwpSL22PFjxJjfT2zhS/afI=">AAACXnicbZHLSgMxFIYz473eRt0IboJFqIgyUwTdCKIbFy4UrAqdWjLxTBuauZicEYcwb+gLuNM3Ma1dtOpZ/ef8X0jOnyiXQqPvfzjuzOzc/MLiUm15ZXVt3dvYvNdZoTi0eCYz9RgxDVKk0EKBEh5zBSyJJDxEg8uh//AKSossvcMyh07CeqmIBWdoR13PXD81u8FZGCvGTVCZZkVDCTHSBg216CXM2ubtraIHE31ZVgehflFoGlPQ4RSy/9S0p44nkdHQ3qBEr4/7Xa/uH/mjon9FMBZ1Mq6brvcePme8SCBFLpnW7cDPsWOYQsElVLWw0JAzPmA9aFuZsgR0x4xCquhenCmKfaCjfpI1LNG6TCLLJAz7+rc3HP7ntQuMTztGpHmBkHKLWC8uJMWMDrOmz0IBR1lawbgS9pWU95lNGu2P1Oz6we9l/4r75lFg9e1x/fxiHMQi2SG7pEECckLOyRW5IS3CyZez4HjOhvPpzrur7voP6jrjM1tkqtztb3dos/Y=</latexit>

L22 =

1

2

⇣�2xx + �2

yy �q

(�2xx � �2

yy)2 + 4(�2

xy)2⌘

<latexit sha1_base64="Jac54VWXhIwsSdeO8kQVrtvyQMw=">AAACXnicbZHNTuswEIWdwOWnXCBcNkhsLCqkIgRKIiTYXAnBhgULkCggNaVyzKS16vxgTxCRlTfkBdjBm+CWLlpgVmfmfJY9x3EhhUbff3Pcufk/C4tLy42Vv6tr697Gv1udl4pDm+cyV/cx0yBFBm0UKOG+UMDSWMJdPDwf+XfPoLTIsxusCuimrJ+JRHCGdtTzzOVD2Av/R4li3AS1CWsaSUiQtmikRT9l1jYvLzXdn+qrqj6I9JNC05qBDmaQvYfQnjqaRsZDe4MS/QHu9bymf+iPi/4UwUQ0yaSuet5r9JjzMoUMuWRadwK/wK5hCgWXUDeiUkPB+JD1oWNlxlLQXTMOqaa7Sa4oDoCO+2nWsFTrKo0tkzIc6O/eaPib1ykxOekakRUlQsYtYr2klBRzOsqaPgoFHGVlBeNK2FdSPmA2abQ/0rDrB9+X/Sluw8PA6uuj5unZJIglsk12SIsE5JickgtyRdqEkw9n0fGcDefdXXBX3fUv1HUmZzbJTLlbn3zes/k=</latexit><latexit sha1_base64="Jac54VWXhIwsSdeO8kQVrtvyQMw=">AAACXnicbZHNTuswEIWdwOWnXCBcNkhsLCqkIgRKIiTYXAnBhgULkCggNaVyzKS16vxgTxCRlTfkBdjBm+CWLlpgVmfmfJY9x3EhhUbff3Pcufk/C4tLy42Vv6tr697Gv1udl4pDm+cyV/cx0yBFBm0UKOG+UMDSWMJdPDwf+XfPoLTIsxusCuimrJ+JRHCGdtTzzOVD2Av/R4li3AS1CWsaSUiQtmikRT9l1jYvLzXdn+qrqj6I9JNC05qBDmaQvYfQnjqaRsZDe4MS/QHu9bymf+iPi/4UwUQ0yaSuet5r9JjzMoUMuWRadwK/wK5hCgWXUDeiUkPB+JD1oWNlxlLQXTMOqaa7Sa4oDoCO+2nWsFTrKo0tkzIc6O/eaPib1ykxOekakRUlQsYtYr2klBRzOsqaPgoFHGVlBeNK2FdSPmA2abQ/0rDrB9+X/Sluw8PA6uuj5unZJIglsk12SIsE5JickgtyRdqEkw9n0fGcDefdXXBX3fUv1HUmZzbJTLlbn3zes/k=</latexit><latexit sha1_base64="Jac54VWXhIwsSdeO8kQVrtvyQMw=">AAACXnicbZHNTuswEIWdwOWnXCBcNkhsLCqkIgRKIiTYXAnBhgULkCggNaVyzKS16vxgTxCRlTfkBdjBm+CWLlpgVmfmfJY9x3EhhUbff3Pcufk/C4tLy42Vv6tr697Gv1udl4pDm+cyV/cx0yBFBm0UKOG+UMDSWMJdPDwf+XfPoLTIsxusCuimrJ+JRHCGdtTzzOVD2Av/R4li3AS1CWsaSUiQtmikRT9l1jYvLzXdn+qrqj6I9JNC05qBDmaQvYfQnjqaRsZDe4MS/QHu9bymf+iPi/4UwUQ0yaSuet5r9JjzMoUMuWRadwK/wK5hCgWXUDeiUkPB+JD1oWNlxlLQXTMOqaa7Sa4oDoCO+2nWsFTrKo0tkzIc6O/eaPib1ykxOekakRUlQsYtYr2klBRzOsqaPgoFHGVlBeNK2FdSPmA2abQ/0rDrB9+X/Sluw8PA6uuj5unZJIglsk12SIsE5JickgtyRdqEkw9n0fGcDefdXXBX3fUv1HUmZzbJTLlbn3zes/k=</latexit><latexit sha1_base64="Jac54VWXhIwsSdeO8kQVrtvyQMw=">AAACXnicbZHNTuswEIWdwOWnXCBcNkhsLCqkIgRKIiTYXAnBhgULkCggNaVyzKS16vxgTxCRlTfkBdjBm+CWLlpgVmfmfJY9x3EhhUbff3Pcufk/C4tLy42Vv6tr697Gv1udl4pDm+cyV/cx0yBFBm0UKOG+UMDSWMJdPDwf+XfPoLTIsxusCuimrJ+JRHCGdtTzzOVD2Av/R4li3AS1CWsaSUiQtmikRT9l1jYvLzXdn+qrqj6I9JNC05qBDmaQvYfQnjqaRsZDe4MS/QHu9bymf+iPi/4UwUQ0yaSuet5r9JjzMoUMuWRadwK/wK5hCgWXUDeiUkPB+JD1oWNlxlLQXTMOqaa7Sa4oDoCO+2nWsFTrKo0tkzIc6O/eaPib1ykxOekakRUlQsYtYr2klBRzOsqaPgoFHGVlBeNK2FdSPmA2abQ/0rDrB9+X/Sluw8PA6uuj5unZJIglsk12SIsE5JickgtyRdqEkw9n0fGcDefdXXBX3fUv1HUmZzbJTLlbn3zes/k=</latexit>

Second order moments of morphology describe a ROI main axis (principal components)

Moments of morphology

• The major axis corresponds to thedirection with the biggest variance.

• The secondary axis (minor in 2D) isorthogonal to the major axis, giving in 3D the direction of the second biggestvariance.

• The third axis (3D) is orthogonal to themajor and secondary axes.

34

Moments of morphology

Principal axes are useful objectdescriptors because…

• they directly define the objectlength, height, and width

• using principal axes, similaritiesbetween objects can be found

35

Parámetros del Programa

By combining the principal axes, we can define Elongation, Relative Elongation, and Flatness.

evevElong°°

-=121

2°ev

1°ev 3°ev

1°ev >> 2°evElong. ~ 1

2°ev

1°ev 3°ev

evevElongR°°

-=131.

1°ev >> 3°evR. Elong. ~ 1

evevFlatness°°

-=231

2°ev

1°ev 3°ev

2°ev >> 3°evFlatn. ~ 1

Moments of morphology

36

Resumen

• The variance allows to computer principal axes (eigenvectors) and to quantify dispersion for each principal axis direction (eigenvalues)

• Composed parameters between eigenvalues deliver morphological parameters like elongation or flatness

• Higher order moments describe more detailed information like asymmetry or kurtosis

Moments of morphology - remarks

37

• EXERCISE: Compute elongation [python notebook]