Course14 Dynamic Vision. Biological vision can cope with changing world. ----- Moving and changing...
-
Upload
agatha-doyle -
Category
Documents
-
view
221 -
download
1
Transcript of Course14 Dynamic Vision. Biological vision can cope with changing world. ----- Moving and changing...
Course14 Dynamic Vision
Course14 Dynamic Vision
Biological vision can cope with changing world.
----- Moving and changing objects----- Change illumination----- Change View-point----- Other changes
Computer vision does in the similar ways. But we have only focus on:
----- Stationary camera, stationary objects----- Stationary camera, moving objects----- Moving camera, stationary objects----- Moving camera, moving objects
1.Change Detection
----- Stationary camera, moving objects in scene.
----- Detect moving object from stationary scene.
----- Two or more frames of image.
----- Images are well aligned.
(1) Difference Pictures:DPjk(x, y)=1 , if |fj(x,y) – fk(x,y)| > TDPjk(x, y)=0 , otherwise
PDPjk(x, y)=1 , if fj(x,y) – fk(x,y) > TPDPjk(x, y)=0 , otherwise
NDPjk(x, y)=1 , if fj(x,y) – fk(x,y) < TNDPjk(x, y)=0 , otherwise
Size filter should used to remove small connected components that are due to noisy pixels.Small motion cannot be detected when use size filter.
(2) Accumulated Difference Pictures
----- a sequence of images taken by a stationary camera at different time.
----- to detect small/slow motion objects by enhancing the difference of pixel values.
ADP0(x,y) = 0
ADPk(x,y) = ADPk-1(x,y) + DP1k(x,y)
ADP0(x,y) = 0
ADPk(x,y) = ADPk-1(x,y) + DP1k(x,y)
Where
(3) Time-varying Edge Detection:
----- Detect the edges of moving objects.
----- Intensity changes in space and in time make up each other.
dt
tyxdf
ds
tyxdfzyxEt
),,(),,(),,(
otherwise1
1),(if),(),,( yxfyxf
ds
tyxdf
otherwise1
1),,(if),,(),,( tyxdftyxdf
dt
tyxdf
Since spatial and temporal changes of image
complement each other. Either Good intensity
change/Slow object motion or Poor intensity
change/Proper object motion can yield good result
of edge detection of moving objects.
(4) Segment moving object with moving camera
----- Moving objects in stationary scene.
----- Translational moving camera.
When camera is performing translational motion in a stationary scene. The image of the scene looks as if caming out from FOE of the image.
dz
dydz
dx
Y
XFOE
f
f
Do Ego-motion-polar transformation for every frame of images:
In the new images of EMP space, stationary points are displayed only along the polar axis between the frames of image sequence, moving points are not so. Thus, moving points and stationary points can be
separated.
),,(),,( tyxftrE
22 )()( ff YYXXr
)(tan 1
f
f
xX
yY
2. 3D structure from motion
3D structure from images means to find the depth of
scene of the corresponding feature points in image.
(1) 3D structure from motion of point correspondences:
(1)
TpRp3
since perspective projection
We have
(2)
Apply to both sides of equ (2)
So,
TPzRPz
pz
P 1 pz
P 1
P
0 TPPRPz
RPP
TPz
Once knowing point correspondences and motion
parameters, 3D depth of image point can be
computed.
Note: the 3D structure will be up to a scale
factor if the absolute value of translation is
uncertain, such as being computed from two
frames of images.
(2) 3D structure from motion of Line Correspondences.
and
are line correspondence over two frames of images.
C
B
A
N
C
B
A
N
Let the corresponding 3D line has the form:
parameter,: nQpL
NRN
NRNn
1
1
(
)(
f)by yoursel derive()(
)()sgn(
1
1
nN
nN
NRN
NRTznNQ
(3) 3D structure from ego-motion camera. ----- Stationary scene.
----- Translationally moving camera with known speed.
----- FOE of image has been computed.
----- Determine the depth of 3D scene.
Ego-motion Complex Log Mapping:
dz
dydz
dx
b
aFOE
jrebYjaX )()(
Where
Define
i.e., u = lnr v =
Now, we want to find the relation of u, v and camera movement in z-direction. Remember that:
22 )()( bYaXr
)(tan 1
aX
bY
),(),(lnln rjvrujrw
z
xX
z
yY (assume f = 1)
dz
dr
rr
dz
d
dz
du 1ln
dz
bzy
azx
d
bzy
azx
22
22
1
z
1
du
dzz So,
dz
azx
bzy
d
dz
d
dz
dv
1tan
dz
az
x
bz
y
d
az
x
bz
y
2
1
1
02
a
zx
z
zy
bazx
zy
bazx
We can conclude that:
----- in ego-motion complex log mapping (EMCLM) space,
is only related the depth changes of scene.
----- Since dz can be measured from the speed of camera movement and du can be computed from image, the depth of scene can be found by
----- The movement of camera in z-direction does not affect the EMCLM velocity. i.e.
dz
du
du
dzz
0dt
dv
(4) Match Correspondences
-----Find the corresponding image features (Points or Lines) from two images frames that correspond to the same features in 3D scene.
1) Point Matching (relaxation labeling)
Given: a set of feature points in image frame 1 and another set of feature point in image 2.
Find: a unique correspondences of points between the two sets of points.
(a) Defines object set O={o1, o2, …, om} from image points of
frame 1, each element is a node. Define Label set L={l1,
l2, …, ln} from points of frame 2.
(b) Establish relationship set among the nodes of object
nodes, such as neighboring points.
(c) Establish an initial match set
M(0)={(<o1,l1>,<o1,l2>,…<o1,ln>),
……
(<oi,l1>,<oi,l2>,…<oi,ln>)
(<om,l1>,<om,l2>,…<om,ln>)};
i.e. each object node aligned to all labels.
(d) Establish consistency measurement of each node and its related nodes with respect to aligned pairs.
e.g. <oi,li> consistent with <oj,lh>, <ow,lv>, …
Consistency measurement may be based on:
----- geometric relation among node in image.
----- gray level or gradient in the original image of the node.
(e) Compute similarity (or disparity) of each node with respect to matched pair.
e.g. where di, l----- disparity,
such as displacement vector.
Probability of match between oi and li :
lili d
w,
, 1
1
kki
lii w
wilP
,
,)|(
(f) Update match set M(k) iteratively:
If the similarity of <oi,li> is high, encourage the match of
its consistent nodes, otherwise, discourage the match of
its consistent nodes.
e.g.
and, normalize the updated probability; where and are
constant.
i
ii
o,lo
kj
kil jhpq
of nodes related of
yconsistencby near
)()( )|(
))(|()|( )()()1( kil
ki
ki qilpilp
(g) Remove the match pair of small similarity (small match
probability ) from match set M(k)
(h) Repeat step (f) and (g) until each node has no more
than one label in M(k).
)|()( ilp ki
(2) Match Line correspondences
Given: Two set of Lines in image A and image B resp
ectively.
Find: Unique correspondences of lines between imag
e A and B.
(a) Matching function:
----- Position disparity.
Relative position in an image:
where is edge direction.
Position disparity between two sub-sets of image
lines from images A and B.
oiojioiojij ppuppd )]sgn([
iu
1
1 121 ),,,;,,2,1(
n
i
n
ijkBkAijn ji
ddkkknd
----- Orientation disparity:
1
1 121 ),,,;,,2,1(
n
i
n
ijkBkAijn ji
kkkn
----- Other disparity:• Length of Line• Intensity of original image• Contrast• Steepness• Straightness (residues of Least squares)
(b) Kernel match:
Match a small sub-sets from image Lines of frames A and B, for robustness consideration of the kernel,----- Number of lines should be no less than 3.----- Lines should be long (stable).----- Lines should not be paralleled.----- Lines should be separated as much as possible.
Minimize the match function over selected subjects between two image frames.
----- attribute, such as position ,orientation.
----- weight
(c) Match expansion:
Once kernel matching is completed, the line correspondences obtained will serve a reference for the match of remaining lines.
)21),,,;,,2,1( 2121 nxxn ,..,k,k,n;k,,(kkkn
,1,0 x thx
Choose a longest line from unmatched line of image
A. Add it into the subset of matched kernel of image
A, calculate match functions for every unmatched
line in image B. The line of image B with minimum
match function is considered a matched line. Add this
matched pair of lines into matching kernel and repeat
the process until no further line needs to match.
(3) Tracking
Given: m objects moving in scene, a sequence of n image frames is taken from the scene.
Find: the trajectories of each object in the image sequence.
i.e.
Ti = < Pi1, Pi
2,…, Pin>; i=1,2, … m
If we use dik to be trajectory deviation of frame k.
dik=( Pi
k-1 Pik, Pi
k Pik+1)
We want: for each object.
1
2
minn
k
kii dD
(a) Path Coherence Function
Assume in consecutive image frames that:
----- change of object location is small. ----- change of scalar velocity is small. ----- change of moving direction is small.
Path coherence function can be defined as
where w1, w2 are weight.
)21()cos1(),,(21
21
21
11
dd
ddwwPPP k
i
k
i
k
i
----- turning angel from Pik-1 Pi
k to Pik Pi
k+1
d1, d2 ----- distances between Pik-1 , Pi
k and Pik, Pi
k+1
respectively.
(b) Occlusion problem
When occlusions occur in scene from viewpoint. Some target points may lose from some image frames. This causes problem in tracking.
To solve this problem, we introduce the concept of phantom point:
----- Virtual object point in frames. ----- No specified coordinates. ----- Has maximum distance dmax from the consecutive
frames. ----- Has a maximum value of path coherence function .
point phantom ispoint any if
features trueare points both if)distance()(Disp
max
11
d
, PP, PP
ki
kik
ik
i
otherwise
points trueare points threeif),(
point phantom is if0
),Dev(
max
11
1
11
k
ik
ik
i
ki
ki
ki
ki , PPP
P
, PPP
Thus:
Tracking algorithm a) Path Initialization
Make initial trajectories for each feature point by linking the nearest point in consecutive frames from frame 1 to frame n. If feature points are missing in some frames, introduce phantom points.
b) Exchange assignments:
From frame 2 to frame n–1 , for each frame:
i) Perfrom forward exchange:
For each feature point i of frame k, find all the combinations wi
th possible feature j of k+1 frame within a window of dmax.
Calculate:
Gijk = [ Dev ( Pi
k-1 , Pik, Pi
k+1) + Dev( Pjk-1 , Pj
k, Pjk+1)]
–[ Dev( Pik-1 , Pi
k, Pjk+1) + Dev ( Pj
k-1 , Pjk, Pi
k+1)]
Find the i-j pair with maximum value of Gmax=( Gijk).
If Gmax > 0 , exchange the trajectory assignment of i
and j in k+1 frame.
(ii) Perform backward exchange:
For each point i of frame k, find all the combinations with poss
ible point j of frame k-1 within a window of dmax.
Calculate:Gij
k = [ Dev( Pik-1 , Pi
k, Pik+1) + Dev ( Pj
k-1 , Pjk, Pj
k+1)]
–[ Dev( Pjk-1 , Pi
k, Pik+1) + Dev ( Pi
k-1 , Pjk, Pj
k+1)]
Find the ij pair with Gmax=( Gijk).
If Gmax > 0 ,exchange trajectory assignments of i and j in frame k–1.
Note: Final results of trajectories may involve phantom points.
The lecture is overThe lecture is over
And And
Thank you for your Thank you for your attendanceattendance
Hope you have leant Hope you have leant
some Basic Concept some Basic Concept
ofof
Computer VisionComputer Vision
in this lecture, which in this lecture, which
is helpful in you is helpful in you
research.research.