Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005
description
Transcript of Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005
![Page 1: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/1.jpg)
A hybrid error concealment A hybrid error concealment scheme for MPEG-2 video scheme for MPEG-2 video transmission based on best transmission based on best neighborhood matching algorithmneighborhood matching algorithm
Li-Wei Kang and Jin-Jang Leou
Journal of Visual Communication & Image Representation 2005
![Page 2: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/2.jpg)
OutlineOutlineIntroductionProposed hybrid error
concealment scheme for MPEG-2 intra-coded I frames
Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames
Simulation results
![Page 3: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/3.jpg)
Introduction (1/2)Introduction (1/2)Transmission errors and error
propagation
Network
Encoded stream
Encoded stream
Original pictures
Decoded pictures
![Page 4: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/4.jpg)
Introduction (2/2)Introduction (2/2)Error control
◦Channel coding
◦Error resilient coding
◦Detection and correction
◦Detection and concealment
Encoded stream
Encoded stream
Encoded stream + = Encoded
stream
currentprevious future current
Encoder control
Decoder control
![Page 5: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/5.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2MPEG-2I frame
◦Spatial search for similar blocks◦Selection between H.26L test model
and FBNM P frame
◦Temporal search for similar blocks◦Determination of block behavior
Small motions, middle motions, and large motions
◦Search for motion vectors
![Page 6: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/6.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (1/7)(1/7)H.26L test model [25]
Best neighborhood matching (BNM) [15]
a1
a2
a3
a4
4321 2222),( a
Nja
Nia
BjNa
NiNjif
N+mN+m
H
L
Best match
![Page 7: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/7.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (2/7)(2/7)Proposed fast BNM
◦Step 1: Initial candidate block by MAD
◦Step 2: Hexagon-based fast search by MAD
![Page 8: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/8.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (3/7)(3/7)Proposed hybrid error concealment
scheme◦Fitness function
H.26L or fast BNM? Check the smoothness of the boundaries
between the concealed block and its neighbors Check the similarity between the concealed
block and its neighbors Components
AIDB AMD AVD
![Page 9: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/9.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (4/7)(4/7)Average intersample difference
across the block boundaries between a concealed block and its four neighbors (AIDB)◦Smoothness of the boundaries
1
0
1
0
1
0
1
0
ofneighbor right theis if ,)0,()1,(1
ofneighbor left theis if ,)1,()0,(1
ofneighbor bottom theis if ,),0(),1(1
ofneighbor top theis if ,),1(),0(1
),(
N
i
N
i
N
j
N
j
BXiXNiBN
BXNiXiBN
BXjXjNBN
BXjNXjBN
XBABD
Bi BNB
iB
BBABDBN
BAIDB ),(1)(
![Page 10: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/10.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (5/7)(5/7)Average mean difference
between a concealed block and its eight neighbors (AMD)
Bi BNB
iB
BmeanBmeanBN
BAMD )()(||
1)(
mean
meanmean
mean
meanmeanmean
mean
mean
![Page 11: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/11.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (6/7)(6/7)Average variance difference
between a concealed block and its eight neighbors (AVD)
Bi BNB
iB
BBBN
BAVD )var()var(||
1)(
![Page 12: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/12.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (7/7)(7/7)Fitness function
Choose of concealed block
◦Purpose of TC: BH.26L is usually smaller than BFBNM, but it’s visual quality is not always better than BFBNM
)()()()( BAVDBAMDBAIDBBCF
o.w.
)()( if 26.26. CFBNMLH
FBNM
LHC
TBCFBCFBB
B
![Page 13: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/13.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 inter-coded (P or B) MPEG-2 inter-coded (P or B) frames (1/2)frames (1/2)MV magnitude
Determination of types of the corrupted blocks
Block matching
bN
ibiaveNMVMV
1
/
Small motion blocksCandidates:MV0
MVave
Middle motion blocksCandidates:MVi for i = 1 to 8MVave
MVmedium
Large motion blocksCandidates:Fast search algorithm
Tl Th
![Page 14: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/14.jpg)
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 inter-coded (P or B) MPEG-2 inter-coded (P or B) frames (2/2)frames (2/2)Fast search algorithm for large
motion blocks
Search range for the corrupted block
2D+1
2D+1
![Page 15: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/15.jpg)
Simulation results (1/8)Simulation results (1/8)MPEG-2
◦GOP size = 12◦IBBP structure◦1.5 Mbps◦30 fps◦One packet contains one slice
CIF◦Coastguard◦Football◦Salesman◦Table tennis
![Page 16: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/16.jpg)
Simulation results (2/8)Simulation results (2/8)parameters
◦I frame N = 16; m = 4; L = H = 120
◦P or B frame Tl = 8; Th = 12; TC = 200; D = 2
N+mN+m
H
L
o.w.
)()( if 26.26. CFBNMLH
FBNM
LHC
TBCFBCFBB
B
Tl Th
![Page 17: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/17.jpg)
Simulation results (3/8)Simulation results (3/8)Discussions
◦H and L for I frame Smaller: good candidates may be excluded Larger: bad candidates may be included
◦Tl and Th for P frame Smaller: better concealment results but high
computation cost Larger: worst concealment results but low
computation cost◦D for P frame
Smaller: worst concealment results but low computation cost
Larger: better concealment results but high computation cost
![Page 18: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/18.jpg)
Simulation results (4/8)Simulation results (4/8)1. replace all corrupted blocks by
zeros2. I frame: H.26L; P/B frame:
overlapped region matching [21]3. I frame: split matching; P/B frame:
forward-backward block matching [22]
4. I frame: H.26L; P/B frame: neighbor matching [23]
5. Proposed
MV interpolation
![Page 19: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/19.jpg)
Simulation results (5/8)Simulation results (5/8)Football
Salesman
Processing time (10% PLR)
12 34
![Page 20: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/20.jpg)
Simulation results (6/8)Simulation results (6/8)Visual quality (15% PLR)
![Page 21: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/21.jpg)
Simulation results (7/8)Simulation results (7/8)Visual quality (15% PLR)
![Page 22: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/22.jpg)
Simulation results (8/8)Simulation results (8/8)FootballPLR = 15%
![Page 23: Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005](https://reader035.fdocuments.us/reader035/viewer/2022062222/56815a6e550346895dc7cdf5/html5/thumbnails/23.jpg)
ReferencesReferences[15] Z. Wang, Y. Yu, D. Zhang, Best neighborhood matching: an
information loss restoration technique for block-based image coding systems, IEEE Trans. Image Process. 7 (7) (1998) 1056–1061.
[21] M.C. Hong, L. Kondi, H. Scwab, A.K. Katsaggelos, Video error concealment techniques, Signal Process.: Image Commun. 14 (1999) 437–492.
[22] S. Tsekeridou, I. Pitas, MPEG-2 error concealment based on block-matching principles, IEEE Trans. Circuits Syst. Video Technol. 10 (4) (2000) 646–658.
[23] S. Valente, C. Dufour, F. Groliere, D. Snook, An efficient error concealment implementation for MPEG-4 video streams, IEEE Trans. Consumer Electron. 47 (3) (2001) 568–578.
[25] Y.K. Wang, M.M. Hannuksela, V. Varsa, A. Hourunranta, M. Gabbouj, The error concealment feature in the H.26L test model, in: Proceedings IEEE International Conference on Image Processing, Rochester, NY, USA, 2002, pp. 729–732.