High-efficiency video coding: tools and complexity

Oct. 5 2012

HEVC

• Highlights– Latest video coding standard developed by the

JCTVC– 50% bit rate reduction compared with the

H.264/AVC at same visual quality– HEVC decoder complexity is similar to the

H.264/AVC decoder, but encoder is way more complex than the H.264/AVC encoder.

– HEVC extensions are under developing

Hybrid Motion-compensation & Transform Coding

What’s new?

• Recursive coding tree structure (64x64 -> 4x4)• Advanced intra prediction• DCT based interpolation filter• Advanced inter prediction• Discrete Since Transform• Scanning• Sample adaptive offset

Recursive Tree Structure

• H.264/AVC– 16x16 macroblock– 16x16, …, 4x4 prediction block– 8x8 & 4x4 transform block

• HEVC– 64x64, …, 4x4 recursive coding unit (CU)– 64x64, …, 4x4 recursive prediction unit (PU)– 32x32, …, 4x4 recursive transform unit (TU)

If we restricted LCU < 64x64

Intra Prediction

• H.264/AVC– Up to 9 modes

• HEVC– Up to 35 modes– Intra smoothing• [1,2,1]/4

Interpolation

• H.264/AVC – 6-tap Wiener for ½ pixel – Bilinear for ¼ pixel

• HEVC– 8-tap DCT-IF for ½ pixel– 7-tap DCT-IF for ¼ pixel

Inter Prediction

• H.264/AVC– Median motion from neighboring blocks

• HEVC– Spatial motion vector prediction– Temporal motion vector prediction

Discrete Since Transform

• Intra 4x4 only– 1% BD-Rate gain

Scanning

• H.264/AVC– Zigzag

• HEVC (zigzag is the history!)– Diagonal up-right– Horizontal– Vertical

Sample adaptive offset (SAO)

• SAO

History of the tool development

And more ….

• mid-2004 H.264/AVC High Profile• Early-2005, VCEG Key Technology Area (KTA)• 2005-2008, MPEG Call for Evidence• Jan. 2010, Joint CfP• April 2010, 1st JCT-VC meeting, TMuC• Oct. 2010, 3rd JCT-VC meeting, WD1• July 2012, 10th JCT-VC meeting, DIS• Jan. 2013, 12th JCT-VC meeting, FDIS

Performances of the HEVC (1)

• Objectively– IBBP– IPPP

Performances of the HEVC (1)

• Subjectively

Complexity Analysis

• NTT DoCoMo Real-Time HEVC Decoder

HEVC Related R&D

• Fast Mode Decision• Rate Control• HEVC Perceptual Quality Model• Software Implementation– ARM– ARM + GPU

• Hardware implementation

HEVC Scalability

• CfP: July 2012 Stockholm Meeting– Base layer could be compatible with HEVC,

H.264/AVC, MPEG-2– Enhancement layer can use new developed tools

• First JCT-VC meeting on scalability: Oct. 2012 Shanghai Meeting – Select the base software to develop the HEVC

based scalability– FDIS of HEVC scalability: Jan. 2014

References[1]

G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, "Overview of the High Efficiency Video Coding (HEVC) Standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, December 2012.

[2] J.-R. Ohm, G. J. Sullivan, H. Schwarz, T. K. Tan, and T. Wiegand, "Comparison of the Coding Efficiency of Video Coding Standards – Including High Efficiency Video Coding (HEVC)," IEEE Transactions on Circuits and Systems for Video Technology, vol.22,December 2012.

[3] F. Bossen, B. Bross, K. Sühring, and D. Flynn, "HEVC Complexity and Implementation Analysis," IEEE Transactions on Circuits and Systems for Video Technology, vol.22, December 2012.