Real-time Object Tracking
-
Upload
otto-von-guericke-university-magdeburg -
Category
Technology
-
view
2.960 -
download
5
description
Transcript of Real-time Object Tracking
![Page 1: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/1.jpg)
Real-time Detection and Real-time Detection and Tracking of Multiple Objects with Tracking of Multiple Objects with Partial Decoding in H.264/AVC Partial Decoding in H.264/AVC
Bitstream DomainBitstream Domain
Wonsang YouUniversity of Augsburg, Germany
Electronic Imaging, 19 January 2009
![Page 2: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/2.jpg)
MOTIVATION
Real-time Object Detection and Tracking in H.264|AVC Bitstream
2
![Page 3: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/3.jpg)
Pixel Domain Approach
• Categories of Object Detection and Tracking Approaches.– Pixel domain approach– Compressed domain approach
• Pixel domain approach.– Using raw pixel data– High accuracy– High computational complexity– Require additional computation for compressed videos
• Compressed domain approach– Exploit encoded information (DCT, motion vectors, etc)– Poor performance
• Applicable for simple scenarios• Weak for occlusion
3
![Page 4: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/4.jpg)
Compressed Domain Approach
• Basic idea– Exploit encoded information (DCT, motion vectors, etc)
• Advantages– Remarkably fast processing time– Adaptive to compressed videos
• Disadvantages– Unreliability of encoded information– Sparse assignment of block-based data– Poor performance
• Applicable for simple scenarios• Weak for occlusion
4
![Page 5: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/5.jpg)
Related Works in Compressed Approach
• Basic Solution– Using a low-resolution image from DCT coefficients
– Unfortunately, impossible for AVC bitstreams
5
DC
![Page 6: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/6.jpg)
Our Solution for H.264/AVC Bitstreams
• Basic idea– We use partially-decoded pixel data instead of low-resolution images.
• Advantages– Reliable performance in more natural scenes
• Articulated objects such as humans• Objects changing in size• Objects which have monotonous color or a chaotic set of motion vectors
– Occlusion handling– Detecting and tracking multiple objects in stationary background– Real-time processing– Partial decoding in I-frames
• It has been considered to be impossible• Due to spatial prediction dependency on neighboring blocks
6
![Page 7: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/7.jpg)
Overview of the Proposed Algorithm
• Extraction Phase– Probabilistic Spatiotemporal Macroblock Filtering– Roughly extracting the block-level region of objects– Constructing the approximate object trajectories in each P-frame
• Refinement Phase– Accurately refining the obect trajectories– Background subtraction and partial decoding in I-frames– Motion interpolation in P-frames
7
![Page 8: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/8.jpg)
EXTRACTION PHASE
Real-time Object Detection and Tracking in H.264|AVC Bitstream
8
![Page 9: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/9.jpg)
Probabilistic Spatiotemporal Macroblock Filtering
• Probabilistic Spatiotemporal Macroblock Filtering– Block-based filtering of background parts (BGs)– By using spatial and temporal properties of macroblocks– Rapid processing of segmenting object regions and tracking each
object
9
![Page 10: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/10.jpg)
Block Clustering
• Block clustering– Removing skip
macroblocks
– Eliminating probable background parts
– Clustering the remaining MBs into several fragments
• Block group (BG)– Set of non-skip blocks
10
B4
F1
B1
B2
B3 B5
B6
B7
B8
BGs
![Page 11: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/11.jpg)
Spatial Filtering
• Filtering block groups which are likely to be background
– Removing BGs of • One-macroblock• All zero IT coefficients
• Active Block Group (ABG)– Remaining BGs after spatial
filtering
11
B4
F1
B1
B2
B3 B5
B6
B7
B8
ABGs: Remaining BGsafter Spatial Filtering
![Page 12: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/12.jpg)
Temporal Filtering
• Filtering ABGs which are likely to be background
– Removing ABGs of background
• Based on temporal consistency of each ABG over a given period
– Fragments with high occurrence probability: considered as a part of objects
12
B4
F1
B1
B2
B3 B5
B6
B7
B8
Remaining ABGsafter Temporal Filtering
![Page 13: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/13.jpg)
Temporal Filtering
• Observing occurrence of ABGs during a finite period
– ABGs with high occurrence for finite period are judged as "Real Object".
– Occurrence Probability is measured.
13
frame
1T
2T
3T
4T
5T
6T
AG 16
26G
iG6
6G
Real object
Real object
Real object Observation
period
36G
lllll GGGLPP ,...,,R 21
ABGs
![Page 14: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/14.jpg)
Temporal Filtering
14
frame
1T
2T
3T
4T
5T
6T
AG 16
26G
iG6
6G
Real object
Real object
Real object Observation
period
36G
lllll GGGLPP ,...,,R 21
ABGs
Criteria for survival of ABGas an object
![Page 15: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/15.jpg)
REFINEMENT PHASE
Real-time Object Detection and Tracking in H.264|AVC Bitstream
15
![Page 16: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/16.jpg)
Background Subtraction in I-frames
16
(a) (b)
(c) (d)
lD lS
A
B C D
Reference Blocks (A-D) aresubstituted into background image
Partial Decodingin I-frames
ROI Refinement in I-frames
![Page 17: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/17.jpg)
Motion Interpolation in P-frames
17
I P P P I
t2R
t3R
t4R
Assumption: The object moves slowly nearly with uniform motion in one GOP
(a) (b)
(c) (d)
ROI Refinement in P-frames
In the ROI prediction stage, ROI significantly vary over P-frames.So, ROI refinement is needed for P-frames.
![Page 18: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/18.jpg)
Occlusion Handling
18
Comparing Hue color histogram of two objects
![Page 19: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/19.jpg)
Experimental Results (1/3)
19
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 400 500 600 700 800 900
0
0.5
1
1.5
2
2.5
3
3.5
0 100 200 300 400 500 600 700 800 900
(a)
(b)
merged active trains
frames
frames
Spatial filtering rates
Act
ive
grou
p tr
ains
real
obj
ects
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
1 11 21 31 41 51 61 71 81 91
Partial DecodingFull Decoding
(a)
sec
frames
I-frames
I-frames
Indoor Sequence: 49.5 frames/secOurdoor Sequence: 37.12 frames/sec
![Page 20: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/20.jpg)
Experimental Results (2/3)
20
(a) (d)
(b) (e)
(c) (f)
(g) (j)
(h) (k)
(i) (l)
![Page 21: Real-time Object Tracking](https://reader033.fdocuments.us/reader033/viewer/2022061115/5463a000af795904328b56e3/html5/thumbnails/21.jpg)
Experimental Results (3/3)
21
(a) (d)
(b) (e)
(c) (f)
(g) (j)
(h) (k)
(i) (l)