Priyadarshini Anjanappa Multimedia Processing Lab University of Texas at Arlington November 26, 2012...

Priyadarshini AnjanappaMultimedia Processing Lab

University of Texas at Arlington

November 26, 2012

Advisor: Dr. K. R. Rao

Performance Analysis and Implementation of Mode Dependent DCT/DST in H.264/AVC

OutlineMotivationOverview of H.264Overview of HEVCMode Dependent DCT/DSTImplementationPerformance AnalysisConclusion Future Work

Motivation Transform coding: Basic coding tool in image and video compression to reduce

spatial redundancies

DCT-II used in recent video coding standards due to excellent energy compaction close to the optimal KLT

Integer approximation of DCT-II used in H.264

Alternative Transforms proposed for HEVC

Integer approximation of DST-VII for 4x4 luma intra prediction residuals in HEVC

Revisiting Mode-Dependent DST/DCT implementation in HEVC to analyze the performance in H.264

Block Diagram of H.264 Encoder

Typical H.264 encoder block diagram [1]

Block Diagram of H.264 Decoder

Typical H.264 decoder block diagram [1]

Profiles in H.264

Profiles and corresponding tools in H.264 [2]

(a) Intra prediction samples for a 4x4 block [3]

A through M are previously predicted pixels

(b) Directions for intra prediction modes for 4x4 and 8x8 blocks [3]

Intra prediction

(a) Intra prediction modes for 4x4 blocks [4]

Intra prediction modes

(b) Intra prediction modes for 16x16 blocks [4]

Macroblock partitions for inter prediction in H.264 [4]

Partitions for Inter prediction

Integer DCT matrices in H.264

4x4 Forward Integer DCT Matrix [5] 4x4 Inverse DCT Matrix [5]

Secondary Transform for DC Coefficients in H.264

4x4 and 2x2 Hadamard Transform matrices [6]

The sixteen 4x4 blocks with DC coefficients drawn as smaller blocks on upper left corner [6]

Scan order in H.264

Scan order of transform coefficients [4]

DC

Block Diagram of HEVC Encoder

Typical HEVC encoder block diagram [7]

Block Diagram of HEVC Decoder

Typical HEVC decoder block diagram [8]

Unified Intra prediction in HEVC : 33 directional modes + DC + Planar

Directional mode for an 8x8 block [7]Modes and angular intra prediction

directions [7]

Intra prediction directions in HEVC

Illustration of Intra prediction directions with mode names mapped to the angular directions [9]

Partitions for Inter prediction in HEVC

Partitions for inter prediction [7]

Quadtree coding structure in HEVC

Subdivision of a CTB into CBs and TBs. Solid lines indicate CB boundaries and dotted lines indicate TB boundaries.

Left: the CTB with its partitioning; right: the corresponding quadtree [7]

Core Transform : Integer DCT in HEVC

Length-16 Integer DCT matrix in HEVC [7]

Integer DST in HEVC

4x4 Integer DST matrix used for intra prediction luma residual units in HEVC [7]

Adaptive scanning in HEVC

(a) Diagonal up-right scan (b) Horizontal scan (c) Vertical scan

Scanning methods for 8x8 blocks in HEVC [7]

Mode Dependent DCT/DST Motivation for DST [10]:

When prediction is performed from one-side, the energy in prediction error residuals increases as we go away from the boundary

A sine transform is better adaptable to these prediction residual statistics

Fast implementation of DST-VII is used in HEVC

For block size of NxN with j, j as row and column indices of the 2-D DST matrix for DST-VII is given as

Applied only to 4x4 luma intra prediction residuals

Mapping from intra prediction modes to DCT/DST in HM2.0

Mode Unified Intra prediction direction

Vertical Transform

Horizontal Transform

0,5,6,12,13,22,23,24,25 VER to VER+8 DST DCT

1,8,9,16,17,30,31,32,33 HOR to HOR+8 DCT DST

2 DC DCT DCT

3,4,10,11,18,19,20,21 VER-8 to VER-1 DST DST

7,14,15,26,27,28,29 HOR-8 to HOR-1 DST DST

DCT/DST combinations applied to intra prediction modes[10]

Example of mapping a prediction mode to DCT/DST

• Intra prediction mode: Vertical• DST in vertical direction• DCT in horizontal direction

Categories of unified intra prediction modes in HEVC

Category 1 oblique modes [12]:

Prediction is performed from the decoded pixels from either the top row or the left column.

The vertical mode 0 and the horizontal mode 1 are special cases of this oblique mode

Category 2 oblique modes [12]:

Prediction is performed from both the top row and the left column pixels

DC mode [12]:

Prediction is performed from an average of all available decoded pixels ; DCT is shown to be the optimal transform along both the directions

Category 1 oblique modes

(a) Prediction from top row only [12] (b) Prediction from left-column only [12]

Category 2 Oblique modes

A (dx,dy) pair denotes the horizontal and vertical distance respectively along the prediction direction for the pixel being predicted from the reference pixel [12]

Proposed Mapping from intra prediction modes to DCT/DST in H.264

Mode Intra prediction direction Vertical Transform Horizontal Transform

0,3,7 Vertical, Diagonal Down-Left, Vertical-Right

DST DCT

1,8 Horizontal, Horizontal –Up DCT DST

2 DC DCT DCT

4,5,6 Diagonal Down-Right, Vertical-Left, Horizontal-Down

DST DST

Forward Integer DCT matrix in H.264

Integer DCT matrix =

Norms of the basis vectors along rows :Row 1: 2Row 2: (3.1623)Row 3: 2Row 4: (3.1623)

Inverse DCT matrix in H.264

Inverse Integer DCT matrix =

Norms of the basis vectors along rows :Row 1: 2Row 2: (1.5811)Row 3: 2Row 4: (1.5811)

Integer DST matrix in HEVCInteger DST matrix =

Norms of the basis vectors along rows :Row 1: (128.0547)Row 2: (128.1718)Row 3: Row 4: (128.0547)

Obtaining DST matrices for H.264

To implement the same DST matrix in H.264, the norm along each row of the DST matrix is made equal to the norm of the corresponding row of the DCT matrices.

Divide rows of the DST matrix by the factor :

Forward DST matrix in H.264

Forward Integer DST matrix

= Norms of the basis vectors along rows :Row 1:2Row 2: 3.1622Row 3: 2Row 4:

Inverse DST matrix in H.264

Inverse Integer DST matrix

= Norms of the basis vectors along rows :Row 1:2Row 2: 1.5812Row 3: 2Row 4:

Implementation of Mode Dependent DCT/DST in H.264

Mode dependent DCT/DST is implemented in the H.264 Reference Software JM 18.4 [13] in the Main profile

Matrix multiplication is implemented instead of fast implementation as the new transform matrices obtained are non-integer values

It is verified that the default fast DCT implementation in H.264 reference software and DCT matrix multiplication give the same results

Simulations are run on an i7 quad 4, 2.0 GHz processor, 8 GB RAM

Performance of default fast DST implementation in HEVC is also analyzed using HEVC reference software HM 8.0 [14] in the Intra Main profile

BD PSNR gives the absolute PSNR gain at the same bitrate and BD Bitrate gives the percentage bitrate savings at the same PSNR

BD PSNR and BD Rate [16] [17] are used to analyze the performance of the RD-curves

Video sequences[15] belonging to HD (1920x1080, 1280x720), WVGA (832x480) and WQVGA (416x240) are used for evaluating the performance

WVGA: Wide Video Graphics Array (Touch screen mobile phones support this resolution) [18]

WQVGA: Wide Quarter Video Graphics Array (Hand-held internet-enabled devices support this resolution) [18]

Only the first frame ( I frame) of the video sequences are considered for the analysis

Calculation of BD PSNR and BD Rate BD PSNR and BD Rate represent the average PSNR and bitrate differences respectively between two

rate distortion (RD) curves [16]

Fit a curve through four data points corresponding to QP = 16, 20, 24, 28

The data points are Y-PSNR values for BD PSNR and bitrate values for BD Rate

A good interpolation curve through the four data points can be obtained by a third order polynomial of the form:

SNR = a + b*bit + c*bit2 + d*bit3

where a, b, c, d are determined such that the curve passes through all the data points

The difference between the RD curves is dominated by high bitrates. Hence logarithmic value of bitrate is considered for calculating BD Rate

In the same way, we can do interpolation as a function of SNR as follows:SNR = a + b*SNR + c*SNR2 + d*SNR3

Based on the interpolation , find an expression for the integral of the curve

The average difference is the difference between the integrals divided by the integration interval

Comparing Default H.264 and Default HEVC

BD PSNR: 2.0212 dBBD Rate: -17.735%

Comparing Default H.264 and Mode Dependent DCT/DST

BD PSNR: -0.4913 dBBD Rate: 4.775%

Comparing Default H.264and Default HEVC

BD PSNR: 1.4722 dBBD Rate: -14.2433%


BD PSNR: -0.4852 dBBD Rate: 5.1894%


BD PSNR: 1.8785 dBBD Rate: -14.6427 %


BD PSNR: -0.7315 dBBD Rate: 6.2691 %


BD PSNR: 1.4754 dBBD Rate: -17.985 %


BD PSNR: -0.3462 dBBD Rate: 4.6475 %


BD PSNR: 1.3704 dBBD Rate: -20.3635


BD PSNR: 0.0665 dBBD Rate: -1.1473 %


BD PSNR: 1.2773 dBBD Rate: -10.2026 %


BD PSNR: -0.6996 dBBD Rate: 6.0669 %


BD PSNR: 2.1567 dBBD Rate: -21.9681 %


BD PSNR: -0.6369 dBBD Rate: 7.2692 %


BD PSNR: 1.3025 dBBD Rate: -19.1702 %


BD PSNR: -0.2102 dBBD Rate: 3.5532 %


BD PSNR: 1.4856 dB BD Rate: -33.1268 %


BD PSNR: -0.029 dB BD Rate: -0.4242 %


BD PSNR: 2.053 dBBD Rate: -33.5926 %


BD PSNR: 0.0216 dBBD Rate: -0.5157 %


BD PSNR: 1.9721 dBBD Rate: -33.2696 %


BD PSNR: 0.0851 dBBD Rate: -1.8388 %


BD PSNR: 2.0659 dBBD Rate: -33.2327 %

Comparing Default H.264 andMode Dependent DCT/DST

BD PSNR: 0.0422 dBBD Rate: -0.8327 %

Case BD PSNR (dB)

BD Rate (%)

1 0.0192 -0.18172 -0.0277 0.26273 0.0239 -0.24684 0.0678 -0.75435 -0.0844 0.79426 -0.1595 1.48387 -0.0236 0.19378 -0.4913 4.7759 -6.3851 82.9101

Input sequence: RaceHorses_416x240_30.yuv

Conclusions

HEVC shows almost 1.5 – 2 dB performance improvement in quality over default H.264

Four out of 12 input sequences showed some performance improvement for Mode Dependent DCT/DST in H.264

The performance drop in most cases could be due to many reasons:

Use of non-integer transform coefficients which results in the decrease in accuracy of reconstructed output

The number of prediction modes is just 9 in case of H.264 and 33 in case of HEVC. The DCT/DST combinations are used for less number of prediction directions in the former case

Selection of the best prediction mode and the corresponding mode dependent DCT/DST is not considered here

Future WorkThe scan order of the transformed coefficients can be modified to

obtain better performance depending upon the intra prediction direction for each block

Rate distortion optimization could be used to apply Mode Dependent DCT/DST only to the best performing modes

Higher resolution video sequences can be used for analysis by using the High profiles in H.264

Analysis can be extended to block sizes other than 4x4 luma and also for chroma intra prediction residuals

References[1] S. -K. Kwon et al, "Overview of h.264/MPEG-4 part 10", Journal of Visual Communication and Image Representation, vol. 17, no. 2, pp. 186-216, April 2006[2] D. Marpe et al, “The H.264/MPEG-4 AVC standard and its applications”, IEEE Communications Magazine, vol. 44, pp. 134-143, Aug. 2006 [3] H.Kalva, “The H.264 video coding standard,” IEEE Multimedia, vol. 13, no. 4, pp.86-90, Oct. 2006[4] I.E.Richardson, “The H.264 advanced video compression standard”, Wiley, Second edition, 2010[5] Whitepaper: 4x4 Transform and Quantization in H.264/AVC:http://www.vcodex.com/files/H264_4x4_transform_whitepaper_Nov10.pdf[6] A. Puri et al, "Video coding using the H.264/MPEG-4 AVC compression standard", Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004

http://www.vcodex.com/files/H264_4x4_transform_whitepaper_Nov10.pdf



[7] G. J. Sullivan et al, “Overview of high efficiency video coding (HEVC) standard”, Pre-publication draft, to appear in IEEE Trans. Circuits and Systems for Video Technology, Dec. 2012 [8] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC-J0292r1, July 2012[9] Analysis of coding tools in HEVC Test model (HM 1.0) - Intra Prediction: http://www.h265.net/2010/12/analysis-of-coding-tools-in-hevc-test-model-hm-intra-prediction.html[10] A. Saxena and F. Fernanades, "CE7: Mode-dependent DCT/DST for intra prediction in video coding”, ITU-T / ISO-IEC Document: JCTVC-D033, Jan. 2011[11] V. Britanak et al, "Discrete cosine and sine transforms: General properties, fast algorithms and integer approximations", Academic Press - Elsevier, Oxford, UK, 2007

http://www.h265.net/2010/12/analysis-of-coding-tools-in-hevc-test-model-hm-intra-prediction.html



[12] A. Saxena and F. Fernandes, "Mode dependent DCT/DST for intra prediction in block-based image/video coding", IEEE ICIP, pp. 1685-1688, Sep. 2011 [13] H.264 Reference Software JM 18.4: http://iphome.hhi.de/suehring/tml/download/[14] HEVC Reference Software HM 8.0 : http://hevc.kw.bbc.co.uk/trac/browser/tags/HM-8.0[15] Test video sequences: ftp://ftp.tnt.uni-hannover.de/testsequences/[16] G. Bjontegaard, "Calculation of average PSNR differences between RD curves", VCEG-M33, April 2001[17] MATLAB source code for BD PSNR and BD Rate: http://www.mathworks.com/matlabcentral/fileexchange/27798-bjontegaard-metric/content/bjontegaard.m [18] Graphics Display Resolution: http://en.wikipedia.org/wiki/Graphics_display_resolution

http://iphome.hhi.de/suehring/tml/download/

http://hevc.kw.bbc.co.uk/trac/browser/tags/HM-8.0

ftp://ftp.tnt.uni-hannover.de/testsequences/

http://www.mathworks.com/matlabcentral/fileexchange/27798-bjontegaard-metric/content/bjontegaard.m

http://www.mathworks.com/matlabcentral/fileexchange/27798-bjontegaard-metric/content/bjontegaard.m

http://en.wikipedia.org/wiki/Graphics_display_resolution

Thank you

Priyadarshini Anjanappa Multimedia Processing Lab University of Texas at Arlington November 26, 2012...

Documents

Transcript of Priyadarshini Anjanappa Multimedia Processing Lab University of Texas at Arlington November 26, 2012...