Distributed Video Compression for Smart Visual Sensors
Transcript of Distributed Video Compression for Smart Visual Sensors
Distributed Video Compression for Smart Visual Sensors
Nikos Deligiannis
Pag.
Wireless Multimedia Sensor Networks
6/06/2013 2
• Wireless Lightweight Multimedia Applications:– Wireless visual surveillance sensors (Little Sister Project)
The mouse sensor©Xetal
Pag.
Wireless Multimedia Sensor Networks
6/06/2013 3
• Wireless Lightweight Multimedia Applications:– Wireless visual surveillance sensors– Wearable/On-body visual sensors– In-body Sensors (Wireless capsule endoscopy)
Video Compression Requirements: Low complexity encoding High compression performance Scalability Communication error resilience
©Given Imaging
Pag.
Traditional Predictive Coding
6/06/2013 4
Traditional Predictive Video Coding (H.26x, MPEG)• Redundancies exploited at the encoder
– Motion Compensated Prediction• Characteristics
– High compression performance– Complex encoder − Simple decoder– Downlink model (Broadcast Scenario)
Pag.
Distributed Video Coding
6/06/2013 5
Distributed Video Coding• Redundancies exploited at the decoder
– Motion/Disparity Estimation at the decoder• Characteristics
– Simple encoder − Complex decoder– Error Resilience– Good compression performance– Uplink model (Multi-terminal Communication Scenario)– **No need for communication between the sensors**
Pag.
Intra Coding
Mono-View Distributed Video Coding**High-level Architecture**
6/06/2013 6
DCT Q SW Enc. SW Dec. IDCTQ-1
Correlation Channel
Side Information
Pag.
Stereo / Multi-View DVC**High-level Principle**
6/06/2013 7
Keyframe
WZ frame
WZframe
Key frame
Pag.
Efficient Intra-Frame Coding**Low-Resolution Image Sensors**
• Current DVC Systems H.263+ or H.264/AVC Intra
• Proposed Designated System:
6/06/2013 8
Pag.
Efficient Intra-Frame Coding**Low-Resolution Image Sensors**
6/06/2013 9
0 20 40 60 80 10015
20
25
30
35
40
45
Bitrate (Kbps)
PSN
R(d
B)
B2_Huffman_noIntraPredB2_Huffman_IntraPredB2_CAVLC_noIntraPredB2_CAVLC_IntraPredB4_Huffman_noIntraPredB4_Huffman_IntraPredB4_CAVLC_noIntraPredB4_CAVLC_IntraPredB8_Huffman_noIntraPredB8_Huffman_IntraPredB8_CAVLC_noIntraPredB8_CAVLC_IntraPred
Pag.
Efficient Intra-Frame Coding**Low-Resolution Image Sensors**
6/06/2013 10
Compared to H.264/AVC Intra:Vast Reduction in Encoding Execution Time
0 10 20 30 40 50 60 7020
25
30
35
40
45
Bitrate (Kbps)
PSN
R(d
B)
H.264/AVC Intra (JM18.4)B4_CAVLC_IntraPred
Pag.
Intra Coding
Distributed Video Coding**High-level Architecture**
6/06/2013 11
DCT Q SW Enc. SW Dec. IDCTQ-1
Correlation Channel
Side Information
Pag.
SI Generation at the Decoder** Motion-Compensated Interpolation **
6/06/2013 12
Pag.
Intra Coding
Endoscopic Distributed Video Coding
6/06/2013 13
DCT Q SW Enc. SW Dec. IDCTQ-1
Correlation Channel
Side Information
Hash CodingHash CodingThe EDVC system: The hash can act as SI predictor Modified OBMEC with HPS
Pag.
SI Generation in EDVC** OBMEC with HPS **
6/06/2013 14
Lanczos-3 Interpolation Filter Reliability Screening MSE-optimal Motion Compensation
Pag.
Compression Results** Endoscopic Sequences **
6/6/2013 15
43.3% BjøntegaardRate Savings
31
33
35
37
39
41
43
400 800 1200 1600 2000 2400
PSN
R Y
(dB
)
Total Rate Y (kbps)
Proposed DVC with HPS
TDWZ
“Endoscopy Test Video 2”, 480×320, GOP2, 30Hz
Pag.
Compression Results** Endoscopic Sequences **
H.264/AVC Intra (790kbps, 37.9dB)
6/6/2013 16
Proposed EDVC (800kbps, 39.2dB)
EDVC advantages: High compression performance Low encoding complexity (~56% less than H.264/AVC Intra) Scalable coding
Pag.
Compression Performance of HDVC
6/6/2013 17
31.4% BjøntegaardRate Savings
26
28
30
32
34
36
38
40
50 150 250 350 450 550
PSN
R Y
(dB
)
Total Rate Y (kbps)
DISCOVER
Proposed HDVC
Foreman Sequence, QCIF, 15Hz, GOP8
Pag.
System Architecture**Codeword Formation and Side Info Refinement**
6/06/2013 18
DCT Q SW Enc. SW Dec. IDCTQ-1
Successively Refined Correlation Estimation
Side Information(OBMEC/SAD)
Buffer
DCT
Partially Decoded Frame
Pag.
Proposed Distributed Video Coding**Results on Low-Resolution Sensor Data**
6/06/2013 19
Uncompressed Data Rate ~ 290 kbps
20 40 60 80 100 120 140 160 18028
30
32
34
36
38
40
42
44
Bitrate (Kbps)
PSN
R(d
B)
Proposed DVC GOP2Proposed DVC GOP4Proposed DVC GOP8H.264/AVC Intra
Pag.
Proposed Distributed Video Coding**Results on Low-Resolution Sensor Data**
6/06/2013 20
Uncompressed Data Rate ~ 316 kbps
20 40 60 80 100 120 140 160 18025
30
35
40
45
Bitrate (Kbps)
PSN
R(d
B)
Proposed DVC GOP2Proposed DVC GOP4Proposed DVC GOP8H.264/AVC Intra
Pag.
Architecture with Encoder Rate Control**Feedback Channel Removal**
6/06/2013 21
DCT Q SW Enc. SW Dec. IDCTQ-1
Successively Refined Correlation Estimation
Side Information
Buffer
DCT
Coarse Side Info
Rate Allocation
Mode Signal.
Entropy Dec-1
Mode Decision(Intra/Skip/WZ)
Arithmetic Entropy Enc.
Pag.
Architecture with Encoder Rate Control**Performance Evaluation**
Vs. feedback-based benchmark (DISCOVER) Better Rate-Distortion Performance No Encoder Syndrome Buffering No FB channel Latency
6/6/2013 22
26
28
30
32
34
36
38
40
0 100 200 300 400 500 600
PS
NR
Y (
dB
)
Total Rate (Kbps)
DISCOVERH.264/AVC IntraProposed feedback-free DVC
26
28
30
32
34
36
38
40
0 100 200 300 400 500 600
PS
NR
Y (
dB
)
Total Rate (Kbps)
DISCOVER
H.264/AVC Intra
Proposed feedback-free DVC
Foreman QCIF GOP4 Soccer QCIF GOP4
Pag.
Conclusions
Novel DVC architectures• Visual Sensor Applications• Significant gains over prior art
Novel Intra-Frame Codec Novel SI generation methods Novel Encoder-Rate Control Mechanism
6/6/2013 23
Pag.
Selected PublicationsJournal Papers
1. N. Deligiannis, F. Verbist, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Progressively refined Wyner-Ziv video coding for visual sensors," ACM Transactions on Sensor Networks, Special Issue on Advancements in Distributed Smart Camera Networks, to appear, May 2014. (IF 2011: 1.808).
2. S. M. Satti, N. Deligiannis, A. Munteanu, P. Schelkens, and J. Cornelis, "Symmetric scalable multiple description scalar quantization," IEEE Transactions on Signal Processing, vol. 60, no. 7, pp. 3628-3643, July 2012 (IF 2011: 2.628).
3. N. Deligiannis, F. Verbist, A. Iossifides, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Wyner-Ziv video coding for wireless lightweight multimedia applications," EURASIP Journal on Wireless Communications and Networking, no. 106, 2012. Special Issue on Recent Advances in Mobile Lightweight Wireless Systems. (IF 2011: 0.873).
4. N. Deligiannis, J. Barbarien, M. Jacobs, A. Munteanu, A. Skodras, and P. Schelkens, " Side-information-dependent correlation channel estimation in hash-based distributed video coding," IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1934-1949, April 2012 (IF 2011: 3.042).
5. J. Slowack, J. Škorupa, N. Deligiannis, P. Lambert, A. Munteanu, and R. Van de Walle, "Distributed video coding with feedback channel constraints," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 7, pp. 1014-1026, July 2012 (IF 2011: 1.649).
6. J. Škorupa, J. Slowack, S. Mys, N. Deligiannis, J. De Cock, P. Lambert, C. Grecos, A. Munteanu, and R. Van de Walle, "Efficient low-delay distributed video coding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 4, pp. 530-544, April 2012 (IF 2011: 1.649).
7. N. Deligiannis, A. Munteanu, T. Clerckx, J. Cornelis and P. Schelkens, “Overlapped block motion estimation and probabilistic compensation with application in distributed video coding,” IEEE Signal Processing Letters, vol. 16, no. 9, pp. 743-746, September 2009 (IF 2009: 1.173).
8. F. Verbist, N. Deligiannis, M. Jacobs, J. Barbarien, P. Schelkens, and A. Munteanu, "Maximum likelihood motion compensation for distributed video coding," Integrated Computer Aided Engineering, vol. 19, no. 3, pp. 215-227, 2012 (IF 2011: 3.451).
6/06/2013 24
Pag.
Selected PublicationsConference Papers
1. S. Wang, X. Jiang, L. Cui, W. Dai, N. Deligiannis, P. Li, H. Xiong, S. Cheng, L. Ohno-Machado, “Genome sequence compression with distributed source coding,” in IEEE Data Compression Conference, DCC’13, Snowbird, Utah, USA, March 2013.
2. F. Verbist, N. Deligiannis, S. M. Satti, A. Munteanu, and P. Schelkens, "Iterative Wyner-Ziv decoding and successive side information refinement in feedback channel-free hash-based distributed video coding," in SPIE Optics and Photonics, Optical Engineering and Applications, Applications of Digital Image Processing XXXV, San Diego, California, USA, August 2012. Special Session on Distributed Video Coding.
3. N. Deligiannis, F. Verbist, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Joint successive correlation estimation and side information refinement in distributed video coding," in European Signal Processing Conference, EUSIPCO’12, Bucarest, Romania, August 2012. Special Session on Distributed Video Coding.
4. J. Slowack, N. Deligiannis, P. Lambert, A. Munteanu, and R. Van de Walle, " Feedback-constrained Wyner-Ziv video coding," in Picture Coding Symposium, PCS’12, Krakow, Poland, May 2012.
5. N. Deligiannis, F. Verbist, J. Barbarien, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Distributed coding of endoscopic video," in IEEE International Conference on Image Processing, ICIP’11, Brussels, Belgium, September 2011. Special Session on Distributed Compression.
6. J. Slowack, J. Skorupa, P. Lambert, R. Van de Walle, N. Deligiannis, and A. Munteanu, "Intra-WZ quantization mismatch in distributed video coding," in IEEE International Conference on Image Processing, ICIP’11, Brussels, Belgium, September 2011.
7. F. Verbist, N. Deligiannis, M. Jacobs, J. Barbarien, P. Schelkens, and A. Munteanu, "A statistical approach to create side information in distributed video coding," in ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC’11, Ghent, Belgium, August 2011. (This paper received the ACM/IEEE ICDSC’11 Best Paper Award).
6/06/2013 25