Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An
description
Transcript of Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An
![Page 1: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/1.jpg)
Selective Disparity Estimation And Variable Size Motion Estimation Based on
Motion Homogeneity for Multi-View Coding
Liquan Shen, Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An
IEEE Transactions on BroadcastingDec. 2009
![Page 2: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/2.jpg)
2
Outline
Introduction Observations and analysis Proposed algorithm Experimental results
![Page 3: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/3.jpg)
3
Introduction (1/2)
Coding structure proposed by HHI:
View-0View-1Time = t
Time = t-
1
disparity estimation
motion estimation
![Page 4: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/4.jpg)
4
Introduction (2/2)Global Disparity Vector (GDV)
View
Time
GDVahead GDVbehindGDVcur
……
…………
……
Non-anchor frame Anchor frame Anchor frame
Ref. view (view 0)
![Page 5: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/5.jpg)
5
Observations (1/2)
T: Temporal prediction Static BG, homogeneous region Small block size mode for complex motion
V: View prediction Complex motion
![Page 6: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/6.jpg)
6
Observations (2/2)
Block size distribution:
Only the MBs in the region with complex motion need DE and small mode size ME.
![Page 7: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/7.jpg)
7
Goal
Try to decide in advance: the optimal prediction direction (ME/DE) for
MBs the prediction size is 16×16 or not
![Page 8: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/8.jpg)
8
Motion homogeneity determined (1/4)
A uniform motion vector field at 4×4 block level is generated.
MBm,n: a MB located at the mth row, nth column. : the
MVs of its convered 4×4 blocks., , ,{ , }, [4 ,4 3], [4 ,4 3]i j i j i jmv mvx mvy i m m j n n
![Page 9: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/9.jpg)
9
Motion homogeneity determined (2/4)
Neighbor MBs used in calculating the motion homogeneity:
Current MB
4
4 4
4
![Page 10: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/10.jpg)
10
Motion homogeneity determined (3/4)
The motion homogeneities of MBm,n in horizontal and vertical directions are defined as:
The motion homogeneities of MBm,n is defined as:
, ,( , ) ( , )
1 1( , )64 64i j u v
i j Z u v Z
MDy m n mvy mvy
, ,( , ) ( , )
1 1( , )64 64i j u v
i j Z u v Z
MDx m n mvx mvx
( , ) ( ( , ) ( , )) 2MD m n MDx m n MDy m n
![Page 11: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/11.jpg)
11
Motion homogeneity determined (4/4)
If MD(m,n) < Tthen the MB is considered with homogeneous motion.
Otherwise, the MB is considered with complex motion.
The threshold T is fixed for each QP level and different sequences, which is set to 0.1.
![Page 12: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/12.jpg)
12
Selective disparity estimation
MB with homogeneous motion is likely to choose temporal prediction.
If a MB satisfies the criterion of spatially homogenous motion, inter-view prediction can be skipped.
![Page 13: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/13.jpg)
13
Selective variable size motion estimation
When a MB is with homogeneous motion, the best mode size of the MB has a very large probability to be 16×16.
![Page 14: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/14.jpg)
14
Proposed fast DE/ME algorithm
1) Derive MV from left, above, left-above MB, and the corresponding MB in the previously coded view.
2) Compute the motion homogeneity for current MB.3) If a MB is a homogeneous motion, perform 16x16
ME, and go to step 6, otherwise, go to step 4.4) Perform variable size DE and ME.5) Perform intra 4x4 prediction.6) Perform intra 16x16 prediction.7) Determine the best prediction direction and
prediction mode. Go to step 1 and proceed with next MB.
![Page 15: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/15.jpg)
15
Experimental results (1/4)Experimental environment
JMVM 6.0 Test sequences (total of 9):
Downflamence2, Flamencol, Golf1, Golf2, Race1, Exit, Ballroom, Jungle, Uli
Full temporal prediction modes and inter-view prediction (FMD)
3 views are coded QP: 20, 24, 28, 32 CABAC, loop filter are enabled
![Page 16: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/16.jpg)
16
Experimental results (2/4)
Comparison between the proposed method and FMD in JMVM:
![Page 17: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/17.jpg)
17
Experimental results (3/4)
![Page 18: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/18.jpg)
18
Experimental results (4/4)
Compares with other method[11]:
[11] X. Li, D. Zhao, X. Ji, Q. Wang, and W. Gao, “A fast inter frame prediction algorithm for multi-view video coding,” in ICIP, 2007.
![Page 19: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/19.jpg)
Fast Mode Decision Using Global Disparity Vector for Multiview Video
coding
Dong-Hoon Han, Yung-Lyul Lee
2008 Second International Conference on Future Generation Communication and Networking Symposia
![Page 20: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/20.jpg)
20
Outline
Goal Proposed algorithm Experimental results
![Page 21: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/21.jpg)
21
Goal
Using both MB-based region segmentation information and global disparity vector (GDV) among view to reduce encoding time.
Fast mode decision using GDV.
![Page 22: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/22.jpg)
22
Region partition (1/2)
The proposed segmentation of the background and objects block modes for fast mode decision in inter-view prediction:
An MB is decided as background block mode if a derive motion vector is smaller than ¼ in integer pixel unit in case of Direct mode , Inter 16x16, P_SKIP or B_SKIP mode.
![Page 23: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/23.jpg)
23
Region partition (2/2)
Black block: object region
White block: background region
![Page 24: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/24.jpg)
24
Fast mode decision forinter-view prediction
Regions of the vies using inter-view prediction are estimated using MB-based GDV and region segmentation map of reference view.
Region segmentation information of base-view
Region segmentation information of non-base view using GDV and (a)
![Page 25: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/25.jpg)
25
Flow chart
![Page 26: Liquan Shen , Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An](https://reader035.fdocuments.us/reader035/viewer/2022062501/56816587550346895dd83e73/html5/thumbnails/26.jpg)
26
Experimental results