Network-Aware H.264/AVC Region-of-Interest Coding...
Transcript of Network-Aware H.264/AVC Region-of-Interest Coding...
![Page 1: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/1.jpg)
Network-Aware H.264/AVC Region-of-Interest Coding for a Multi-Camera
Wireless Surveillance Network
NetworkNetwork--AwareAware H.264/AVC H.264/AVC RegionRegion--ofof--InterestInterest CodingCoding forfor a a MultiMulti--CameraCamera
WirelessWireless SurveillanceSurveillance NetworkNetwork
PierpaoloPierpaolo BaccichetBaccichet, Xiaoqing Zhu and Bernd , Xiaoqing Zhu and Bernd GirodGirod
Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University
![Page 2: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/2.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 2
OutlineOutline
Overview of network-aware wireless surveillance
H.264/AVC compliant Region-of-Interest (ROI) coding
Cross-Layer design
– Reference Picture Selection
– Dynamic Rate Control
![Page 3: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/3.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 3
Wireless Surveillance Camera NetworkWireless Surveillance Camera Network
Important to capture all (and possibly only) significant changes– ROI coding was proposed for JPEG 2000 [Hata et al., MMSP 2005]
Dynamic, distributed resource allocation– Network driven rate control [Zhu et al., MMSP 2005]
Data Collection
![Page 4: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/4.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 4
Simulation ScenarioSimulation Scenario
1 0
6
7 3
2
4
5
Residential Area
Central Location 1
6
73
2
4
5
0Collection Node
(Stanford)
![Page 5: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/5.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 5
OverviewOverview of the of the ProposedProposed SchemeSchemeInput video
Noise reduction
filter
Pre-processor
-
Framestore
ROI / SKIPdetermination
H.264/AVCencoder
Video Packets
Feedback
ROI signalling Network
A simple pre-processor identifies important regions/pictures to code– The frame is skipped if no changes are detected– Only the region-of-interest is encoded for “active” frames
Network feedback exploited to dynamically adapt encoding parameters– Reference Picture Selection– Dynamic rate control
![Page 6: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/6.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 6
H.264/AVC Compliant ROI CodingH.264/AVC Compliant ROI Coding
-
0 1 0 0 0 0 0 0 0 0 00 0 1 0 1 4 1 0 0 0 00 0 0 0 5 8 3 0 0 0 00 0 0 0 14 13 6 0 0 0 00 0 0 0 4 11 3 0 0 0 00 0 0 0 0 4 3 0 0 0 00 0 0 0 0 0 1 1 0 0 00 0 0 0 0 0 0 1 1 0 00 0 0 0 0 0 1 0 1 1 0
0 1 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 1 1 0 0 00 0 0 0 0 0 0 1 1 0 00 0 0 0 0 0 1 0 1 1 0
SliceGroup 0
SliceGroup 1
SliceGroup 2
Slice Group 3
Stored image
Difference signalSignificance Map
(MAE for each Macroblock)Current image
ROI to be encoded
Picture Parameter Setwith the FMO2 mapping
ROI
![Page 7: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/7.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 7
BitBit--Rate SavingsRate Savings
0 50 100 150 200 250 300 35032
34
36
38
40
42
Bit-rate [Kbps]
Lum
a P
SN
R (R
OI)
[dB
]
Camera 5
ConventionalFrame-skippingFrame-skipping + ROI
One intra MB line inserted every 4 frames to recover from errors
![Page 8: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/8.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 8
Coded Frame SizeCoded Frame Size
5 10 15
0
500
1000
1500
2000
2500
Time [s]
Enc
oded
Fra
me
Siz
e [b
ytes
]
Camera 5
ConventionalFrame-SkippingFrame-Skipping + ROI
![Page 9: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/9.jpg)
Original uncompressed H.264/AVC (~ 1.16Mbit/sec)
Frame skipping(~ 630 Kbit/sec)
Frame skipping + ROI(~ 180 Kbit/sec)
![Page 10: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/10.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 10
Impact of Packet LossesImpact of Packet Losses
300 400 500 600 700 80026
27
28
29
30
31
Bit-rate [Kbps]
Lum
a P
SN
R (R
OI)
[dB
]
Camera 5
ConventionalFrame-skippingFrame-skipping + ROI
Simulation performed using ns2, maximum delay 0.5 secAutomatic retransmission every 0.2 sec
![Page 11: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/11.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 11
ReferenceReference Picture Picture SelectionSelection
Error control by means of Reference Picture Selection– ACK based approach
1 2 3 4 5 6 7 8 9
ACK(3) ACK(4) ACK(6)ACK(1)
![Page 12: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/12.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 12
ReferenceReference Picture Picture SelectionSelection
200 400 600 800 200 400 600 80026
27
28
29
30
31
32
33
34
35
36
37
Bit-rate [Kbps]
Lum
a P
SN
R (R
OI)
[dB
]Camera 5
noRPS - Frame-skippingnoRPS - Frame-skipping + ROIRPS - Frame-skippingRPS - Frame-skipping + ROI
![Page 13: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/13.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 13
252729313335373941
Lum
a PS
NR
(RO
I) [d
B]
1 2 3 4 5 6 7 AVGCamera number
Fixed QP Adaptive QP
NetworkNetwork--AwareAware Rate ControlRate Control
QP manipulated according to:– number of unACKed packets– end-to-end delay
![Page 14: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/14.jpg)
Video Video SequenceSequence ComparisonComparison
Conventional H.264/AVC Network Aware ROI coding
![Page 15: Network-Aware H.264/AVC Region-of-Interest Coding …ivms.stanford.edu/~zhuxq/papers/pcs2006pres.pdf34 36 38 40 42 Bit-rate [Kbps] Luma PSNR (ROI) [dB] Camera 5 Conventional Frame-skipping](https://reader031.fdocuments.us/reader031/viewer/2022022504/5ab47d8c7f8b9ab47e8bfa04/html5/thumbnails/15.jpg)
P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 15
ConclusionsConclusions
ROI coding and automatic frame rate adaptation leads to more efficient representation of surveillance video
Reference Picture Selection and dynamic rate control enablebetter usage of available network resources
Highest sustainable quality increased by 4 to 10 dB