Analysis of Motion Estimation Algorithm (HEVC), using Multi-core processing

Shiba Kuanar

Shiba.kuanar@mavs.uta.edu

1000449352

• OVERVIEW OF HEVC• ALGORITHEM SEARCH• RESULTS• REFERENCES

CONTENTS:

• High Efficiency Video Coding (HEVC) is the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group.

• The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards—in the range of 50% bit-rate reduction for equal perceptual video quality.

• Video Coding Standards: Always have a trade between 1) Picture Quality 2) Compression Ratio 3) Computation complexity

HEVC

Fig 1:HEVC Encoder[2]

• Partitioning – into non overlapping macro blocks• Prediction - Forward/Backward based on current, past and

future frame• Error signal - transformed and quantized • Entropy coded

HEVC

Figure 2: HEVC decoder block diagram [2]

Fig 3: Partitioning of a Macro block

CTB->CBs andTBsSolid lines indicate CB boundariesand dotted lines indicate TB boundaries

Corresponding QUAD TREE

• Motion estimation consumes more than 84% coding complexity and time to encode

• Block Based ME algorithms are proposed to reduce the computation time. Algorithms

will be discussed the current proposal

“Based on search position number reduction”

“Fast Full algorithm”

• Proposed work is to analyze the existing algorithm and try to reduce motion estimation

computation time using multicore programming.

• HM13.0 – latest HEVC software.

• Block Matching Criterions used in BBME

- Mean of Square Error

- Mean of absolute difference (No multiplication)

- Matching Pixel Count.

Motion Estimation(ME)

Fig 4: BBME Algorithm for a) 2DLOG, b) TSS and c) NTSS search

Figure 5: Various Search Patterns: a) Diamond search, b) Square Search, c) Horizontal Hexagonal search, d) Vertical Hexagonal Search, e)Rotating Hexagon

Type - 1 and f) Rotating Hexagon Type - 2 [22]

a) Diamond search b) Square Search c) Horizontal Hexagonal d) Vertical Hexagonal

e) Rotating Hexagon Type – 1 f) Rotating Hexagon Type – 2

• In HEVC video coding, motion estimation (ME) plays a vital role in temporary redundancy reduction between the frames.

• In BBME, the current frame is divided into N ×N pixel size macro blocks (MBs) and for each MB a certain area of the reference frame is searched, to minimize a block difference measure (BDM).

• The block difference measure is usually a sum of absolute differences (SAD) between the current and the reference frame MB.

• The displacement within the search area which gives the minimum BDM value is called a motion vector (MV).

• MVs together with transformed and quantized block differences (residua) are entropy coded into the video bit stream.

• If maximum search range (SR) within the reference frame is set to d, then (2d+ 1)² [fig 6] search points (SPs) have to be evaluated, when the full search (FS) strategy is employed.

• To decrease such a huge computational burden, many fast BBME algorithms have been proposed. On the current proposal only two algorithms are going to be analyzed 1) Algorithm based on the search position number and 2) algorithm based on reduction of fast full search.

Block based Motion Estimation

Figure 6: d - search range, with a reference frame.

(2d+ 1)² search points (SPs)

Full Search Algorithm Process (4 stages)

• 1) Prediction stage – Algorithm uses motion vector of previously coded neighboring blocks to predict initial block.

• 2) Search patterns are employed to find the “Global minimum point”

• 3) Early termination where the ME algorithm uses a “threshold” to terminate the search process (Basically Iterative process)

• 4) ME algorithm refines the motion vector if it does not satisfy the early termination criteria

Results and Plots (Measures)

• The simulation will be conducted using HM Software 13.0 with difference video sequences [28], search range, block sizes and number of frames using GPU multicore computing.

• PSNR (dB), bitrate (kbps) plots and BD (Bjontegaard Delta) [17] results will calculated for different algorithms using various search patterns

• AVC: Advanced Video Coding•  • CABAC: Context Adaptive Binary Arithmetic Coding•  • CB: Coding Block•  • CSVT: Circuits and Systems for Video Technology•  • CTB: Coding Tree Block•  • CTU: Coding Tree Unit•  • CPU: Central Processing Unit•  • CU: Coding Unit•  • CUDA: Compute Unified Device Architecture•  • DCT: Discrete Cosine Transform•  • GPU: Graphic Processing Unit•  • HEVC: High Efficiency Video Coding•  • ISO: International Organization for Standardization•  • ITU-T: International Telecommunication Union – Telecommunication Standardization Sector•  

Acronyms

• MPEG: Moving Picture Experts Group• • OPENMP: Open Multiprocessing• • PB: Prediction Block• • PCM: Pulse Code Modulation • • PU: Prediction Unit• • SIMD: Single Instruction Multiple Data• • TB: Transform Block• • TU: Transform Unit• • VCEG: Video Coding Experts Group• • VBSME: Variable Block Size Motion Estimation

Acronyms

Reference thesis• 1 Thesis by S.Gangavati on “Complexity reduction of H.264 using parallel

programming” which describes significant speed-up in encoding time on GPU using CUDA and CPU combined than on CPU by data and task parallelization, 2012.

http://www-ee.uta.edu/Dip/Courses/EE5359/Sudeep_Thesis_Draft_2.pdf • 2 Thesis proposed by Pratik Meheta on “Complexity reduction for intra mode selection

in HEVC using OpenMP” http://www-ee.uta.edu/Dip/Courses/EE5359/Pratik_Mehta_ThesisProposal.pdf

References• [1] G.J. Sullivan et al, “Overview of the high efficiency

video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22, pp.1649-1668, Dec.2012.

References

• [2] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC). J. Sel. IEEE Journal of topics in Signal Processing” vol. 7, pp.1001-1016, Dec. 2013

• [3] F. Bossen et al, “HEVC complexity and implementation analysis”, IEEE Trans. CSVT, vol. 22, pp.1685-1696, Dec.2012.

• [4] T.L Silva et al,”HEVC intra coding acceleration based on tree inter-level mode correlation”, SPA 2013, Poznan, Poland Sep.2013.

• [5] A. Saxena and F. Fernanades, “Mode dependent DCT/DST for intra prediction in block based image/video coding”, IEEE ICIP, pp. 1685-1688, Sept. 2011.

• [6] H. Zhang and Z. Ma, ”Fast intra prediction for high efficiency video coding ”, Pacific Rim Conf. on Multimedia, PCM2012, Singapore, Dec.2012.

• [7] C.C. Chi et al, “Parallel scalability and efficiency of HEVC parallelization approaches”, IEEE Trans. CSVT, vol. 22, pp.1827-1838, Dec.2012.

References

• [8] Introduction to parallel computing https://computing.llnl.gov/tutorials/parallel_comp/#Whatis

• [9] T. Wiegand, et al, Working Draft 3 of High Efficiency Video Coding, JCTVC-E603, JCTVC of ISO/IEC and ITU-T. Geneva, Swiss, Mar. 2011.

• [10] T. Wiegand and G.J. Sullivan, "Overview of the H.264/AVC video coding standard," IEEE Trans. on Circ. Sys. for Video Tech., vol. 13, no. 7, July 2003.

• [11] Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, "Comments on Motion Estimation Algorithms in Current JM Software (JVT-Q089)", Joint Video Team Document, 17th Meeting: Nice, FR, 14-21 Oct., 2005.

• [12] N. Purnachand, L.N. Alves and A.Navarro, "Improvements to TZ search motion estimation algorithm for multiview video coding", IEEE IWSSIP 2012, Vienna, Apr. 2012.

• [13] HM Reference Software 13.0 https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-13.0rc1/

• [14] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243

• [15] J. Ascenso, C. Brites and F. Pereira, "Improving Frame Interpolation with Spatial Motion Smoothing for Pixel Domain Distributed Video Coding", in Proc. EURASIP Conference on Speech and Image Processing, Multimedia Communication and Services, Slovak Republic, June-July 2005.

• [16] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE International Conference on Consumer Electronics, pp. 165-166, Jan.2010.

References

• [17] G. Bjontegaard, "Calculation of average PSNR difference between RD curves", VCEG-M33, 2001.

• [18] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012.

• [19] P. Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression standard”, SPIE Applications of digital image processing XXXV , vol. 8499, paper 8499 -30, Aug. 2012.

• [20] M. Horowitz et al, “Informal subjective quality comparison of video compression performance of the HEVC and H.264/MPEG - 4 AVC standards for low delay applications”, SPIE Applications of digital image processing XXXV , vol. 8499, paper 8499 - 31, Aug. 2012.

• [21] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012.

• [22] L.N.A. Alves, A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc IEEE International Conf. on Consumer Electronics -ICCE Berlin , Germany , vol.11 , pp. 11 - 14 , Sep. , 2012

References

• [[23] X. Wang et al, Paralleling Variable Block Size Motion Estimation of HEVC on Multicore CPU plus GPU platform, IEEE International Conference on Image Processing (ICIP 2013), Melbourne, Australia, Sep.15-18, 2013.

• [24] “K.R. Rao, D.N. Kim, J.J. Hwang, Video Coding Standards” –Springer 2014 .

• [25] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey,” Journal of Opto-Electronics Review, vol. 21, pp 86-102, Mar. 2013.

• [26] A. Abdelazim,W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, 902905, Feb. 17, 2014, doi:10.1117/12.2041166

• [26] Software repository for HEVC - http://hevc.hhi.fraunhofer.de/

• [27] Video test sequences - http://forum.doom9.org/archive/index.php/t-135034.html or http://media.xiph.org/video/derf/

• [28] HM Software Manual - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/

References

THANK YOU