Overview on Scalable Video Coding - II Chuan-Yu Cho.
-
date post
20-Dec-2015 -
Category
Documents
-
view
216 -
download
2
Transcript of Overview on Scalable Video Coding - II Chuan-Yu Cho.
![Page 1: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/1.jpg)
Overview on Scalable Video Coding - II
Chuan-Yu Cho
![Page 2: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/2.jpg)
Outline
HHI and MS SVC Codecs Scalable Video Model 3.0
![Page 3: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/3.jpg)
SVC: 68th meeting
14 full proposals in two categories Category 1: MCTF/2D Wavelet Category 2: AVC based (incl. AVC/MCTF)
Core Experiments (CE) Reference model (software) = the best performing
scheme in each category Category 1: (MCTF/2DWT) Microsoft Research Asia Category 2: (AVC/MCTF) HHI
Result of Core Experiments will be used for decision to adopt one complete codec architecture into the WD.
From: NCTU 69th MPEG meeting report
![Page 4: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/4.jpg)
HHI and MS SVC Codecs
Chuan-Yu Cho
![Page 5: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/5.jpg)
Motion Compensation Temporal Filtering (MCTF)
L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0
H2L2 H2
L2 H2L2
H3L3 H3
L1 H1L1 H1
L1 H1L1 H1
L1 H1L1 H1
L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0
H2H2L2L2 H2H2
L2L2 H2H2L2L2
H3H3L3L3 H3H3
L1L1 H1H1L1L1 H1H1
L1L1 H1H1L1L1 H1H1
L1L1 H1H1L1L1 H1H1
![Page 6: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/6.jpg)
Motion Compensation Temporal Filtering (MCTF)
H
LH
LLL LLH
video sequence
1st temporal level
2nd temporal level
3rd temporal level
![Page 7: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/7.jpg)
Wavelet tree with reduction of spatial size throughout the temporal levels
H
LH
LLL LLH
video sequence
1st temporal level
2nd temporal level
3rd temporal level
![Page 8: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/8.jpg)
Motion Adaptive Filtering
![Page 9: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/9.jpg)
Motion Compensation Temporal Filtering (MCTF)
![Page 10: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/10.jpg)
Adaptive vs Non-adaptive MCTF
![Page 11: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/11.jpg)
3D Sub-band Video Coding using Barbell lifting Status
Proposal Source
MSRA Asia IM group, MS CMPT group Authors
Jizheng Xu, Ruiqin Xiong, Bo Feng, Gary Sullivan, Ming-Chieh Lee, Feng Wu, Shipeng Li
![Page 12: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/12.jpg)
3-D Wavelet/Subband (Interframe Wavelet) Method
H
LH
LLL LLH
video sequence
1st temporal level
2nd temporal level
3rd temporal level
Temporal Sca
lability
Spatial Scalibility
![Page 13: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/13.jpg)
Barbell lifting
Barbellfunctions
t
S0 S1 S2
)(ˆ 000 Sfs
1s
a a
)(ˆ 222 Sfs
0s2s
t
S0 S1 S2
)(ˆ 000 Sfs
1s
a a
)(ˆ 222 Sfs
0s2s
210 ˆˆ sassat
![Page 14: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/14.jpg)
Lifting schemeH0
X0 X1 X2 X3 X4
)(ˆ 000 Xfx )(ˆ 444 Xfx
1x
H1
)(ˆ 222 Xfx
)(''ˆ 222 Xfx
-a -a -a -a
L0
x0
H0
x2
H1 X4
)(ˆ000 Hgh
)(''ˆ 111 Hgh
L2
)(''ˆ 000 Hgh
)(ˆ111 Hgh
2b b
X0 X2
L1
x4
b 2b
Prediction stage
Update stage
![Page 15: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/15.jpg)
L4: 256Kbps QCIF 15
L5: 512Kbps QCIF 15
![Page 16: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/16.jpg)
L1: 48Kbps QCIF 7.5
L2: 64Kbps QCIF 15
L3: 128Kbps QCIF 15
![Page 17: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/17.jpg)
Scalable Extension of H.264/AVC
Status Proposal => SVM 3.0
Source Fraunhofer Institute for Telecommunications -
Herinrich Hertz Institute (HHI) Authors
Heiko Schwarz, Detlev Marpe, and Thomas Wiegand
![Page 18: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/18.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 19: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/19.jpg)
HHI Temporal Scalability
L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0
H2L2 H2
L2 H2L2
H3
L3
H3
L1 H1L1 H1
L1 H1L1 H1
L1 H1L1 H1 {MP}1
{MP}2
{MP}3Temporal Enhancement Layer (Layer 1)
Temporal Base Layer (Layer 0)
Temporal Enhancement Layer (Layer 2)
Temporal Enhancement Layer (Layer 3)
L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0
H2H2L2L2 H2H2
L2L2 H2H2L2L2
H3H3
L3L3
H3H3
L1L1 H1H1L1L1 H1H1
L1L1 H1H1L1L1 H1H1
L1L1 H1H1L1L1 H1H1 {MP}1{MP}1
{MP}2{MP}2
{MP}3{MP}3Temporal Enhancement Layer (Layer 1)
Temporal Base Layer (Layer 0)
Temporal Enhancement Layer (Layer 2)
Temporal Enhancement Layer (Layer 3)
![Page 20: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/20.jpg)
L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0*
L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1
L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0
Spatial Base Layer (Layer 0)
Spatial Enhancement Layer (Layer 1)
reconstructedsequence
reconstructedand upsam pledsequence
H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1
reconstructedsequence
temporalsubbandpictures
Spatial upsampling
Base Layer Prediction
Reconstruction
L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0*L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0*
L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1
L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0
Spatial Base Layer (Layer 0)
Spatial Enhancement Layer (Layer 1)
reconstructedsequence
reconstructedand upsam pledsequence
H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1
reconstructedsequence
temporalsubbandpictures
Spatial upsampling
Base Layer Prediction
Reconstruction
HHI SNR Scalability
![Page 21: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/21.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 22: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/22.jpg)
I P
PI B B
Layer 0 & 11
0
32kbps
48kbps
![Page 23: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/23.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 24: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/24.jpg)
P B (Up sample)
2
1
![Page 25: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/25.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 26: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/26.jpg)
Intrapredictionfor intra MB
2
1
Intrapredictionfor intra MB
InterPrediction
H L
96Kbps
![Page 27: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/27.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 28: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/28.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 29: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/29.jpg)
4
3
SNR Scalability (QP)Temporal Scalability(Motion Compensation)
H00
L21
L22
![Page 30: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/30.jpg)
HHI Joint Scalability
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P B
H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2
{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0H22 H0
0 H12 H0
0 L22 H0
0 H12 H0
0 H22 H0
0 H12 H0
0
I B P B P BI B P B P B
H20 H1
0 L20 H1
0 H20 H1
0H20 H1
0 L20 H1
0 H20 H1
0
Spatial upsampling
H21 H1
1 L21 H1
1 H21 H1
1H21 H1
1 L21 H1
1 H21 H1
1
H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1H23 H0
1 H13 H0
1 L23 H0
1 H13 H0
1 H23 H0
1 H13 H0
1
{MP}1,2{MP}1,2
{MP}0{MP}0
Layer 0: QCIF, 7.5 Hz, 64 kbit/s
Layer 1: QCIF, 15 Hz, 128 kbit/s
Layer 2: CIF, 15 Hz, 256 kbit/s
Layer 3: CIF, 15 Hz, 512 kbit/s
Layer 4: CIF, 30 Hz, 1024 kbit/s
Layer 5: CIF, 30 Hz, 2048 kbit/s
![Page 31: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/31.jpg)
4
3
SNR Scalability (QP)Temporal Scalability(Motion Compensation)
H00
H21
H22
![Page 32: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/32.jpg)
Scalable Video Model 3.0
Tomas Wiegand
Fraunhofer Institute for Telecommunications – Heinrich Hertz Institute (HHI)
To be published in VCIP ‘05
![Page 33: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/33.jpg)
Outline
![Page 34: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/34.jpg)
Introduction
![Page 35: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/35.jpg)
MCTF using the Lifting Representation
![Page 36: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/36.jpg)
Motion model of H.264/MPEG-4 AVC
![Page 37: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/37.jpg)
Prediction and Update operators
![Page 38: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/38.jpg)
Derivation of Update Operators
![Page 39: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/39.jpg)
Temporal Coding Structure
![Page 40: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/40.jpg)
Temporal Decomposition Structure
![Page 41: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/41.jpg)
Impacts on H.264/MPEG-4 AVC
![Page 42: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/42.jpg)
Simulation Results for the MCTF Extension
![Page 43: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/43.jpg)
Simulation Results for the MCTF Extension
![Page 44: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/44.jpg)
SNR-Scalable Extension
![Page 45: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/45.jpg)
Adjustment of the Quantization Parameter
![Page 46: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/46.jpg)
Impacts on H.264/MPEG-4 AVC
![Page 47: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/47.jpg)
Results for Layered SNR-Scalability
![Page 48: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/48.jpg)
Results for Layered SNR-Scalability
![Page 49: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/49.jpg)
Spatial Scalability
![Page 50: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/50.jpg)
Spatial Prediction of Data
![Page 51: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/51.jpg)
Results for Layered Combined Scalability
![Page 52: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/52.jpg)
Results for Layered Combined Scalability
![Page 53: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/53.jpg)
Fine Granular Scalability
![Page 54: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/54.jpg)
Fine Granular Scalability
![Page 55: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/55.jpg)
Fine Granular Scalability
![Page 56: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/56.jpg)
Simulation Results: City
![Page 57: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/57.jpg)
Simulation Results: Crew
![Page 58: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/58.jpg)
Summary
![Page 59: Overview on Scalable Video Coding - II Chuan-Yu Cho.](https://reader038.fdocuments.us/reader038/viewer/2022103123/56649d445503460f94a20f68/html5/thumbnails/59.jpg)
References
“MCTF and Scalability Extension of H.264/AVC and its Application to Video Transmission, Storage, and Surveillance” R. Schäfer, H. Schwarz, D. Marpe, T. Schierl,
and T. Wiegand, Fraunhofer Institute for Telecommunications – Heinrich Hertz Institute (HHI)
To be published at VCIP ‘05