Recognition of object by finding correspondences between features of a model and an image. Alignment...
-
date post
22-Dec-2015 -
Category
Documents
-
view
217 -
download
2
Transcript of Recognition of object by finding correspondences between features of a model and an image. Alignment...
Recognition of object by finding correspondences between features of a model and an image.
Alignment repeatedly hypothesize correspondences between minimal set of features of a model and an image and then tries to find model poses.
For computing poses a model of projection must be selected.
A minimal number of points needed to compute a model pose is three.
Alignment
General idea:
1. Given an input image and a candidate model, establish correspondence between them.
2. Determine transformation from the model to the image
3. Apply the recovered transformation to the model
4. Compare the transformed model with the viewed object
5. Based on this comparison choose the best model
Alignment cont.
General steps before alignment : 1. Selection of object of interest in the picture.2. Segmentation – delineation of a sub-part of the image
to which subsequent recognition process will be applied.
3. Image description – extraction of information which will be used for matching the viewed object with stored object models
4. Extracting an alignment key. Alignment key is an information used to bring the viewed object and models into alignment.
Before, During and After Alignment
Alignment1. Viewed object is brought into correspondence with a
large number of models stored in the memory.2. Individual alignmentsGeneral steps after alignment:
1. Indexing (classification)– use some criteria to “filter out” unlikely models.
Matching
Before, During and After Alignment
We consider a work “3D pose from 3 corresponding points under weak perspective projection” of T.D.Alter
The problem is to determine the pose of 3 points in space given 3 corresponding points in image.
It gives direct expressions for 3 matched model points in image coordinates and an expression of a position in the image of any additional, unmatched model point.
3D pose from 3 corresponding points
Hypothesize a correspondence between three model points and three image points.Compute the 3D pose of the model from three-point correspondence.Predict the image positions of the remaining model points and extended features using the 3D pose.Verify whether the hypothesis is correct by looking in the image near the predicted positions of the model features for corresponding image features.
Alignment Algorithm
Fig.1 Model points undergoing perspective projectionto produce image points
The perspective solution
0 1 2, ,m m m
0 1 2, ,i i i
Let image points be extended as follows: Then
The problem is: givenfind a,b, and c. From the law of cosines:
Given a, b, and c, we can compute the 3D locations of themodel points:
The perspective solution cont.
0 1 2, ,i i i
( , ) ( , , )x y x y f
01 0 1 02 0 2 12 1 2ˆ ˆ ˆ ˆ ˆ ˆcos , cos , cosi i i i i i
01 02 12 01 02 12, , , cos ,cos ,cosR R R
2 2 201 012 cosa b ab R
2 2 202 022 cosa c ac R
2 2 212 122 cosb c bc R
0 0 1 1 2 2ˆ ˆ ˆ, ,m ai m bi m ci
Approximate perspective projection closely in many cases.
Less complicated.
Conceptually simpler.
We do not need to know the camera focal length and the central point.
Fewer solutions (four for perspective an two for weak perspective).
Justification of the weak perspective approximation
Weak-Perspective Solution
Fig.2 Model points undergoing orthographic projection plusScale to produce image points
0 1 2, ,m m m
0 1 2, ,i i i
To recover the 3D pose of the model we should know the distances between the model points
and distances between the image points
The parameters of the geometry in Fig. 2 are (will be proved
later ):
See eq. (7)-(13).
Weak-Perspective Solution cont.
01 02 12( , , )R R R01 02 12( , , )d d d
2b b acs
a
2 2 2 21 2 01 01 02 02( , ) ( ( ) , ( ) )h h sR d sR d
1 2 1 2
1( , ) ( , )H H h h
s
Computing the Weak-Perspective Solution
From Fig. 3 we have three constraints:
2 2 21 01 01
2 2 22 02 02
2 2 21 2 12 12
( ) (1)
( ) (2)
( ) ( ) (3)
h d sR
h d sR
h h d sR
Computing the Weak-Perspective Solution cont.
Multiplying (3) by –1 and adding all three gives
Squaring (4) and using (1) and (2) to eliminate and
which leads to biquadratic in s :
2 2 2 2 2 2 21 2 01 02 12 01 02 122 ( ) ( ) (4)h h s R R R d d d
21h 2
2h
2 2 2 2 2 2 2 2 2 2 2 2 2 201 01 02 02 01 02 12 01 02 124( )( ) ( ( ) ( )) (5)s R d s R d s R R R d d d
4 22 0 (6)as bs c
where
The positive solutions of biquadratic are
Computing the Weak-Perspective Solution cont.
2 2 2 2 2 201 02 01 02 12
01 02 12 01 02 12 01 02 12 01 02 12
4 ( )
( )( )( )( )
a R R R R R
R R R R R R R R R R R R
2 2 2 2 2 2 2 2 2 201 02 02 01 01 02 12 01 02 12
2 2 2 2 2 2 2 2 2 2 2 201 01 02 12 02 01 02 12 12 01 02 12
2 2 ( )( )
( ) ( ) ( )
b R d R d R R R d d d
d R R R d R R R d R R R
2 2 2 2 2 201 02 01 02 12
01 02 12 01 02 12 01 02 12 01 02 12
4 ( )
( )( )( )( )
c d d d d d
d d d d d d d d d d d d
2
(10)b b ac
sa
From (1),(2) and (4)
Computing the Weak-Perspective Solution cont.
2 2 2 21 2 01 01 02 02( , ) ( ( ) , ( ) ) (11)h h sR d sR d
2 2 2 2 2 2 201 02 12 01 02 121 ( )
(12)1
if d d d s R R R
otherwise
1 2 1 2
1( , ) ( , ) (13)H H h h
s
The solution fails when the model triangle degenerates to a line, at which case a=0.
Computing the Weak-Perspective Solution cont.
Let the image points be
Given we can invert the projection to get the tree model points:
where unknown w can not be recovered.
Image location of a fourth model point.
0 0 0
1 1 1 1
2 2 2 2
1( , , )
1( , , ) (14)
1( , , )
m x y ws
m x y h ws
m x y h ws
0 0 0 1 1 1 2 2 2( , ), ( , ), and ( , )i x y i x y i x y
1 2, ,s h h
Denote the model points in arbitrary model coordinate
frame. Using solve the following vector
equation for the “extended affine coordinates”, of
Let
Using the three model points with
Image location of a fourth model point.
0 1 2, , andp p p
( , , )
ip
3p
3 1 0 2 0 1 0 2 0 0( ) ( ) ( ) ( ) (15)p p p p p p p p p p
01 1 0 01 1 0
02 2 0 02 2 0
,
(16)
x x x y y y
x x x y y y
0 0 1 1 2 2, , andp m p m p m
1 0 01 01 1
1( , , ) (17)p p x y h
s
2 0 02 02 2
1( , , ) (18)p p x y h
s
1 0 2 0 01 2 02 1 02 1 01 2 01 02 02 012
1( ) ( ) ( , , ) (19)p p p p y h y h x h x h x y x y
s
Substituting (17)-(19) into (15) we’ll get
Image location of a fourth model point.
3 01 01 1 02 02 2
01 2 02 1 01 2 02 1 01 02 02 01 0 02
01 2 02 101 02 0
01 2 02 101 02 0
01 021 2
1 1( , , ) ( , , )
1 1)
1
m x y h x y hs s
y h y h x h x h x y x y x y ws s
y h y hx x x
s sx h x h
y y ys
x yh h
02 01 ) (20)x y
ws
To project, first apply the scale factor s :
Then project orthographically to get the image location of the
fourth point:
Image location of a fourth model point.
01 2 02 13 01 02 0
01 2 02 101 02 0
01 02 02 011 2 ) (21)
y h y hsm x x x
sx h x h
y y ys
x y x yh h w
s
3 01 02 01 2 02 1 0
01 02 01 2 02 1 0
( ) ( )
( ) ) (22)
sm x x y H y H x
y y x H x H y
3D pose from 3 corresponding points under weak perspective projection” T.D.Alter., MIT A.I.Memo No. 1378, 1992
3D Pose from 3 Points Using Weak Perspective T.D.Alter, IEEE Transactions on Pattern Analysis and Machine Intelligence,v.16 No.8,1994
References