Image Understanding 2
description
Transcript of Image Understanding 2
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
1
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
2
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
3
Blocks-World Scene Analysis(Adolfo Guzman, MIT AI Lab, 1967)• Input: a line drawing representing a 3D
scene consisting of polyhedra• Output: lists of faces grouped into
objects• Method: (a) classify vertices, (b) link
faces according to rules, (c) extract connected components of doubly linked regions.
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
4
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
5
Guzman Vertex Categories
ell, arrow, fork, tee, kay
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
6
Guzman Linking Rules
ell, arrow, fork, tee, kay
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
7
Guzman Face Grouping Criterion:Two adjacent faces must be doubly linked to be considered part of the same object.
E
F
DG
CB
A
Obj1: (A B C)
Obj2: (D E F G)
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
8
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
9
Impossible Figures
Locally consistent, globally inconsistent
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
10
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
11
Edge Detection
Why? Find boundaries of objects in order to compute their features: shape, area, moments, corner locations, etc.
Methods:Compute measures of change in the neighborhood of each pixel.Trace boundariesTransform each neighborhood into a vector space whose basis includes "edge" vectors.
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
12
Horizontal differences: a – b
Both horiz. & vertical diffs.
Roberts' cross operator (a-d)2 +(b-c)2
Local Edge Detectors
a b
a b
c d
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
13
Express each 3x3 neighborhood as a vector.
N = <a, b, c, d, e, f, g, h>
Frei-Chen Edge Detection
e
h
c d
f g
aa b
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
14
Frei-Chen Edge Det. (cont)Transform it linearly into the 9D vector space spanned by the following basis vectors (each shown in 3x3 format)
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
15
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
16
Instead of using 1st derivatives, let's consider 2nd derivatives.
The Laplacian effectively finds subtle changes in intensity.
2 f(x,y) = 2 f(x,y)/x2 + 2f(x,y)/y2
discrete approximation:
Edge Finding with the Laplacian
-1
0
-1 4
0 -1
00 -1
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
17
Remedy: First, smooth the signal with a Gaussian filter.
g(x,y) = 1/(22) exp(-(x2+y2)/22 )
Then take the Laplacian.
Or, directly convolve directly with a Laplacian of Gaussian (LOG).
Laplacian: Very Sensitive to Noise
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
18
The edges tend to be where the second derivative crosses 0.
Zero Crossings
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
19
Zero Crossings: Example
(a) orig. (b) LoG with = 1, (c) zero crossings. R Fischer et al. http://homepages.inf.ed.ac.uk/rbf/HIPR2/zeros.htm
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
20
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
21
If two neighboring pixels share a side, they are 4-adjacent.
d's 4-adjacent neighbors are: b, c, e, g
If two neighboring pixels share a corner, they are 8-adjacent
d's 8-adjacent neighbors are:
a,b,c,e,f,g,h
4- and 8-adjacency
e
h
c d
f g
aa b
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
22
A set of pixels R is said to be 4-connected, provided for any pi, pj in R, there is a chain [(pi, pi' ), (pi' , pi'' ), ..., (pk, pj)] involving only pixels in R, and each pair in the chain is 4-adjacent. The chain may be of length 0.
R is 8-connected, iff there exists such a chain where each pair is 8-adjacent.
4- and 8-connectedness
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
23
4- and 8-connected components
4-connected components of black: 48-connected components of black: 24-connected components of white: 28-connected components of white: 2
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
24
"Dual approach" to edge detection.
Let P be a digital image: P = {p0, p1, ..., pn-1}
A segmentation S = {R0, R1, ..., Rm-1} is a partition of P, such that
For each i, Ri is 4-connected, and Ri is "uniform",
and for all i,j if Ri is adjacent to Rj,
then (Ri U Rj) is not uniform.
Segmentation into Regions
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
25
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
26
Can compute a segmentation if U(R) means all pixels in R have the same pixel value (e.g., intensity, color, etc.)
Algorithm: Initialize a UNION-FIND ADT instance, with each pixel in its own subset.Scan the image. At each pixel p, check its neighbor r to the right. p and r have the same pixel value, determine whether FIND(p) = FIND(r), and if not, perform UNION(FIND(p), FIND(r)). Then check the pixel q below doing the same tests and possible UNION.
When the scan is complete, each subset represents one 4-connected component of the image.
Image Connected Components
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
27
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
28
Gestalt Grouping
Identification of regions using a uniformity predicate based on wider-neighborhood features (e.g., texture).
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
29
Gestalt Grouping
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
30
Gestalt Grouping
Texture element = “texel”
Texel directionality
Texel granularity
Alignments of endpoints
Spacing of texels
Groups cue for surfaces, objects.
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
31
Image Understanding 2Outline:
Guzman Scene AnalysisLocal and Global ConsistencyEdge DetectionLaplacians and Zero CrossingsSegmentation into RegionsImage Connected ComponentsGestalt GroupingMorphology
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
32
"Mathematical Morphology"
Developed at the Ecoles des Mines, France by J. Serra, from mathematical ideas of Minkowski.
Idea is to take two sets of points in space and obtain a new set.
Set A: the main set.Set B: the "structuring element"Set C: the result.
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
33
Math. Morph. Operation 1: "Erosion"
Consider each point b of B to be a vector and use it to translate a copy of A getting Ab
Take the intersection of all the Ab
The result is called the erosion of A by B.
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
34
Erosion: example
A: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 1 0 0
B: 1 1
C: 0 0 1 0 0
0 1 1 0 0
0 0 1 1 0
0 0 0 0 0
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
35
Dilation: Take Union instead of Intersection.
A: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 1 0 0
B: 1 1
C: 0 1 1 1 0
1 1 1 1 0
0 1 1 1 1
0 0 1 1 0
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
36
Opening: Erode by B, then Dilate by –B (B rotated by )
A: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 1 0 0
B: 1 1
C: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 0 0 0
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
37
Closing: Dilate by B, then Erode by –B (B rotated by )
A: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 1 0 0
B: 1 1
C: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 1 0 0
(In this example, erosion undoes the dilation.)
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
38
Closing – another example
A: 0 1 1 0 0
1 0 1 0 0
0 1 0 1 0
0 0 1 0 0
B: 1 1
C: 0 1 1 0 0
1 1 1 0 0
0 1 1 1 0
0 0 1 0 0
(In this example, closing fills in holes.)
CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II
39
Math. Morphol. UsesRepair regions corrupted by noise;
Eliminate small regions
Identify holes, bays, protrusions, isthmi, peninsulas, and specific shapes such as crosses.
Printed circuit board inspection (find cracks, spurious wires.)
Popular primitives for writing industrial machine vision apps.