8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
1/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1137
Adaptivesearchrangedecisionforfastintra-andinter-predictionmodedecisioninH.264/AVC
MehdiJafari
1andShohrehKasaei
2
1DepartmentofElectricalEngineering,KermanBranch,IslamicAzadUniversity,Kerman,Iran
2DepartmentofComputerEngineering,SharifUniversityofTechnology,Tehran,Iran
[email protected];[email protected]
H.264/AVC,thelatestvideocodingstandard,adoptsrate-distortionoptimization(RDO)techniquetoobtainthebestintra- andinter-prediction,whilemaximizingvisualqualityandminimizingthe requiredbitrate.However,the fullRDcostcalculationforallintra-predictionmodes,theexhaustivesearchesforfindingoptimalmotionvectorsforallblocksizes,andthemultiplereferencesframeprocedureconsiderablyincreaseitscomputationalcomplexitywiththeallowednumber of prediction modes. The authors have previously proposed an approach for both inter- and intra-modedecisions that takes into account the several effective parameters, image content type, quantization parametercorrelationofmotionvectorsandmotiondetailsofvideoobjects.Inordertoreducethecomplexity,hereweproposeanewtwo-stepadaptivesearchrangedecisionbydevelopingacombinationofpreviousalgorithm.Thefirststepofouradaptivesearch rangedecisiondetermines thesearchrange forcurrentMBaccording to themotionvectorsof thepreviousMBs.Inthesecondstep,themotionvectorobtainedinthecurrentframeisusedtoadaptivelysearchingthewindowsizeofthepreviousreferenceframeformulti-framemotionestimation.Assuch,onlyasubsetofinter-and
intra-modesis chosen forRDOcalculation.Experimentalresultsshowthattheproposedalgorithmreducesthe totaencodingcost with negligible loss inPSNRand a slightly increase in the requiredbitrate,when compared tooupreviousalgorithmandotheralgorithmsreportedintheliterature.
Keywords:H.264/AVC,intra-andinter-prediction,rate-distortionoptimization,similarpredicted-pixels,split/merge.
IntroductionAs recent multimedia applications are growing
rapidly, videocompression requireshigher performanceas well as new features. The newest video codingstandard is developed by the joint of video teams ofISO/IEC MPEG and ITU_T VCEG as the internationalstandard 14496-10 (MPEG-4 part 10) advanced videocoding (AVC) (Wiegandetal., 2003; Panetal., 2005).
H.264/AVChasgainedmoreandmoreattention;mainlydue to its high coding efficiency, minor increase indecoder complexity compared to existing standards,adaptation to delay constraints, error robustness, andnetwork friendliness (Wiegand et al., 2003; Pan et al.,2005). To achieve an outstanding coding performance,H.264/AVCemploysseveralpowerful coding techniquessuchasdirectionalpredictionofintra-codedblocks,inter-predictionwithvariableblock-sizemotioncompensation,multi-reference framemotion estimation,motion vectorswithquarter-pel accuracy, in-loop deblocking filter, 44integertransform,andtheforth.Accordingtothesenewfeatures, the encoder computational complexity hasextremely increased compared to previous standards.
However, theses functionalities have made H.264/AVCdifficult for applications with low computationalcapabil ities. Thus, until now, the reduction of itscomplexity is a challenging task. The improvement incoding performance comes mainly from the intra- andinter-prediction part. H.264/AVC employs the LagrangeRDOmethod toachieve the best codingmodeofintra-and inter-prediction with highest coding efficiency. TheRDO technique requires a lot of computations since ittests the encoding process with all possible coding
modesofintra-andinter-coding,andcalculatestheirRDcoststo choose themodehaving theminimumrequiredbitrate. The reference software of H.264/AVC, JM7.1adoptsafullsearchforbothmotionestimationandintraprediction. The run-time percentage of each function isshown in Fig.1 (Huang et al., 2006). As shown in this
figure, motion estimation is the most computationallyintensive part. The instruction profile of the referencesoftwareshowsthatreal-timeencodingofCIF30Hzvideo(baseline options, search range [-16.75, +16.75], five
reference frames) requires 314,994 million instructionspersecondandmemoryaccessof471,299MBytes/sec.
Also, H.264/AVC offers a rich set of predictionpatterns for intra-prediction (i.e., nine prediction modesfor44lumablocksandfourpredictionmodesfor16 16lumablocks). Theintra-predictionmodedecisionis verycomplexandthenumberofcomputingRDcostvaluesfolumaandchromaofamacroblockis592(Changsungeal.,2003).Thus,thecomputationalburdenofthesetypesofbruteforce-searchingalgorithmisfarmoredemanding
Fig.1.Run-timepercentagesoffunctionalblocksinH.264/AVCbaselineencoder.
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
2/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1138
thananyexistingvideocodingalgorithm.Thechallengingtopicoffastmodeselectionforintra-
andinter-predictionisconsideredinthispaper.Toreducethe computational complexity, many algorithms havebeenproposed.For fast inter-prediction modedecision,the early termination technique (Tourapis et al., 2003)
reducesthenumberofpotentialpredictionmodes.Lim etal. (2003) proposed a classification method. Later, fastinter-mode selection algorithms were proposed (Kim etal.,2004a,b;Martinetal.,2004)toalleviatetheencodercomplexitydue to themotionestimationandinter-modedecisions. Dai et al. (2004), presented a fast inter-prediction mode decision based on pre-encodingprocess.
Fast intra-mode decision algorithms using edgedetection histogram and local edge detection areproposed by Pan et al. (2003, 2004, 2005). However,theirpreprocessingstagesstillconsumeacodingtimetodetect theedgedirectionand toclassify itintoa limited
direction. Also, there exist fast algorithms to select theoptimal intra-prediction mode using simple directionalmasks (Kim et al., 2005)with saving time of 70%, andstatistical-basedmethods(Garg etal.,2005)withsavingtimeof45%.Anotherfastintra-modedecisionschemeisproposed by Jeon et al. (2003), where the encodingspeedisapproximately30%fasterthanthatoftheRDOmethod. Anewfast intra-predictionalgorithmbasedonmacroblockproperties(FIPAMP)ispresentedbyYangetal. (2004). This algorithm achieves 10% to 40% ofcomputation reduction while maintaining similar PSNRandbitrate performance ofH.264/AVC codes.Mengetal., (2003) presented an efficient intra-prediction (EIP)algorithm based on early termination, selective
computation of highly probable modes, and partialcomputationofthecostfunction.Also,animprovedcostfunctiontoimprovethecodingperformanceisproposedbyTsengetal.(2004;Bentaoetal.,2010;Tamtaouietal.,2010). AlsoHsuetal. (2006) obtaineda fastalgorithmbased on local edge information by calculating edgefeatureparameters.
JafariandKasaei(2008)proposedanewapproachfor both inter-and intra-mode size selection. For intra-prediction mode decision, we improved Pans method(Pan et al., 2003, 2004, 2005), and for inter-predictionmode decision, the split/merge procedure was used.Also, an effective method for accelerating the multiple
reference frames ME without significant loss of videoquality was proposed that was basedon analyzing theavailableinformationobtainedfrompreviouslyprocessedframes.
This paper presents a two-step adaptive searchrange decision algorithm to improve the proposedalgorithm(Jafari&Kasaei,2008;2010).Inthefirststep,theproposedadaptive algorithmdeterminesthe searchrangeforcurrentMBaccordingtoMVsofpreviousMBs.In the second step, the length of MV obtained in the
current reference frame is considered as the maximumsize of the search window for motion estimation in thepreviousreferenceframe.We haveverified thedifferenparts of the proposed algorithm step by step byimplementing it on the JM7.1 reference software andhavecompared itsperformancewith previousandothe
availablefastalgorithms.Experimentalresultsshowthathe proposedmethod reduces the encoding cost up to40% with a negligible loss in the reconstructed videoqualityandanegligibleincreaseintherequiredbitrate.ModedecisionandRDOinH.264/AVC
In commonwith earlier standards, H.264/AVCdoesnot define the encoder, but defines the syntax of anencoded video bitstream together with the method odecoding the bitstream (Jafari &Kasaei, 2005; Jafari &Kasaei, 2006). The codec combines intra-pictureprediction with inter-picture prediction to exploit thespatialandtemporalredundancy,respectively.Intra-predictionmodedecision
Intra-prediction is based on the observation thaadjacent macroblocks tend to have similar propertiesPrediction may be formed for each 44 luma block(I4MB),1616lumaMB(I16MB),and88chromablock.
For prediction of 44 luminance blocks, the 9directional modes consist of a DC prediction (Mode 2and8directionalmodes(labeled0,1,3,4,5,6,7,and8)as shown inFig.2(a). In Fig. 2(b), the block (values opixels a to p) is to be predicted using A to Q pixevalues.
TheDCprediction(mode2)isusefulforthoseblockswithlittleornolocalactivities,theothermodes(1-8)mayonly be used if all required prediction samples areavailable.
For regions with less spatial details (i.e., flat regions)H.264/AVCsupports1616intra-coding;inwhichoneofour prediction modes (DC, vertical, horizontal, andplanar) is chosen for prediction of the entire luminancecomponentsofthemacroblock(Richardson,2005).
H.264/AVC supports four chroma prediction modesfor88chrominanceblocks,similartothatoftheI16MBprediction, except that the order of mode numbers isdifferent: DC (Mode 0), horizontal (Mode 1), vertica(Mode2),andplane(Mode3).
Fig.2.(a)Intra-predictionmodesfor4 4luminanceblocks.(b)Labelingofpredictionsamples.
(a) (b)
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
3/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1139
Inter-predictionmodedecisionInter-prediction is based onusingmotionestimation
and compensation to take advantage of temporalredundanciesthatexistbetweensuccessiveframes.Theimportant differences fromearlier standards include thesupportforarangeofblocksizes(downto4 4),multiple
referenceframes,andfinesub-pixelmotionvectors(1/4pixelinthelumacomponent).
H.264/AVC supports motion compensation blocksizes ranging from 1616 to 44 luminance sampleswith many options between the two. The luminancecomponentofeachmacroblock(1616samples)maybesplitupin4waysas1616,168,816or88.Ifthe88mode is chosen, each of the four 88 macroblockpartitionswithinthemacroblockmaybesplitinafurther4ways as 88, 84, 48 or 44. These partitions andsub-partitions give rise to a large number of possiblecombinationswithineachmacroblock.
A separate motion vector is required for each
partition or sub-partition. Each motion vector must becoded and transmitted; in addition, the choice ofpartition(s) must be encoded in the compressedbitstream.Choosinga largepartitionsize (e.g., 1616,168, 816) means that a small number of bits arerequired to indicate the choice of motion vector(s) andthe typeof partition; however,the motion compensatedresidualmay contain a significant amount of energy inframeareaswithhighdetails.Choosingasmallpartitionsize (e.g., 84, 44, etc.) may give a lower energyresidualaftermotioncompensation,butrequiresalargernumberofbitstosignalthemotionvectorsandthechoiceofpartition(s).Thechoiceofpartitionsizethereforehasasignificantimpactoncompression(asmallpartitionsize
maybebeneficialfordetailedareas).H.264/AVC as an enhanced reference picture
selectionasH.263++enablesefficientcodingbyallowinganencodertoselect(formotioncompensationpurposes)among a large number of pictures that have beendecodedandstoredinthedecoder.RDOprocedure
For I-frames, all MBs are predicted as Intra.H.264/AVC encoder encodes the best mode using allmodecombinationsoflumaandchromaandchoosestheonethatgivesthebestRDOperformance.ForP-frames,intra-andinter-predictionisdoneandRDOisusedtofindthebestprediction.TheRDOproceduretoencodeone
MBin an I-framehasbeen reported(Changsungetal.,2003). According to the procedure, intra-prediction inH.264/AVC, thenumberofmodecombinationsfor lumaand chroma blocks in amacroblock is N8 (16N4 +N16), whereN8, N4, andN16, denote the number ofmodesfor88chromablocks,and4 4and1616lumablocks, respectively(Changsung et al., 2003). Also,accordingtotheRDOprocedureofinter-prediction,forMblockmodes,Nreferenceframes,and W searchrange,
2)12.(. WNM positions should be tested for a single
referenceframeandasingleblockmode.Thismakesthecomplexity of the encoder extremely high. In order toreduce the encoding complexity with little RDperformancedegradation, fast intra- and intra-predictionmodedecisionmethodsareproposed.Proposedfastintra-predictionmodedecisionmethods
This section presents a new fast intra-predictionalgorithm. The proposed method is based on severafactsthatwehaveobservedfromthestatisticsofdifferensequences as: 1)Fig.3shows the total number of 44and 1616 intra-coded macroblocks at differenquantizationparameters(QPs).Ascanbeseenfromthifigure, fast detection of 44 intra-prediction mode cansignificantly improve the encoding speed at low QPswhile1616 intra-predictions can improve thespeed alargeQPs, 2) The prediction modes of each block arecorrelated with those of neighboring 44 luminanceblocks, 3) Normally, pixels along the direction of locaedgeshavesimilarvalues.Therefore, agoodpredictioncan be achieved by predicting the pixels using theneighboringpixelsthatlieinthesameedgedirections,4)The optimal mode (found by a full-search) and othegood (second or third best)modes are most likely tohavesimilardirections,5)Thedirectionalfeaturesof4 4blocks can be preserved roughly after down-samplingand 6) Experimental results show that the referencepixelsofa44lumablockarelikelytobesimilartoeach
other(Jafari&Kasaei,2005).Basedon theseobservations, wehave proposeda
fast intra-prediction mode selection algorithm. In thissection, some new ideas are combined with the fasmode selection algorithm introduced (Lim et al., 2003Kim & Altunbasak, 2004; Pan et al., 2005) to improvetheirefficiency.ImprovedpansmethodforfastdecisionofI4MB
Panetal.(2005)presentedafastmodeselectionfointra-prediction method in which the average edge
Fig.3.Numberof4 4and16 16intra-codedmacroblocksfordifferentquantizationparameters.
10 15 20 25 30 35 40 45 500
1000
2000
3000
4000
5000
6000
Quantization Parameter
Num
berofM
B
16x16 MB
4x4 MB
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
4/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1140
direction of a given block is measured. The Sobeloperatorsarefirstusedtoobtainthedirectionalvectorofeachpixelinablockby
},{ ,,, jijiji dydxD
(1)
WheretheSobeloperatoraredefinedby
,1,11,11,1,11,1,
,11,1,11,11,1,1,
22
22
jijijijijijiji
jijijijijijiji
PPPPPPdy
PPPPPPdx
(2)
The amplitude and angle of each edge vector can becalculatedusing
||||)( ,,, jijiji dydxDAmp
(3)
and,
)arctan(180
)(,
,
,
ji
jio
jidx
dyDAng
(4)
WhereAng(.)isfittedintooneofthe8modes.
Then, the edge direction histogram (EDH) of theblock is found (thatindicates the number ofpixels withsimilar edge directions). Therefore, the cell k with the
maximumamount indicatesthat thereis a strong edgealong that direction in theblockand thus is assignedas the dominant blockdirection. Fig.5 shows theEDHoftheimageshowninFig.4.
InthePansmethod,forI4MBthereare4modes(1DC, 1 from maximum
amplitudeofEDH,andits2neighbors)with2modes
(1DCmodeand1directional)foreach1616lumablockand 88 chroma block. Here, to improve the Pansmethod, we will eliminate the DC mode from thecandidates if the direction of the block is obvious, andotherwise,wechooseonlyDCmode.TocheckwhethertheDCoftheblockisclearornot,thediffvalue,givenin
Eq.(5),iscomputedtocheckwhetheritissmallerthanapredeterminedthresholdornot,using
4)8(
||
15
0
15
0
i
i
i
i
i
pavg
pavgdiff
(5)
TheimprovedPansmethodisproposedasfollows:1.Find the maximum value of the edge directionahistogramH.DenotethecorrespondingmodebyM1.2. Ifdiff>T,carryouttheRDOprocedurefor3modesatthemost(M1anditstwoneighbors).3. Else, ifdiff T1consider only the primary prediction mode decided by
edge direction histogram as a candidate for the bespredictionmode.Thediff1inthiscaseiscomputedby
6)32(
||1
64
0
64
0
i
i
i
i
i
pavg
pavgdiff(6)
5. If diff1
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
5/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1141
1.Find the maximum value of EDH. Denote thecorrespondingmodebyM1.2.Ifthemodes for one ofthe top orleftblocksareM1,then choose M1 as the best candidate mode for thecurrentblock.Gotostep7.3.For 44 luma block, compute themean of absolute
difference (MAD)of its reference pixels. If this value issmallerthanapredeterminedthreshold,selectM1.Gotostep7.Thisresultisyieldedfromthefactthatifthesimilarityofreferencepixelsofablockishigh,thedifferencebetweendifferent prediction modes will be very small. For thiscase,itisnotnecessarytocheckall9predictionmodes,but only one prediction mode is enough (Hsu et al.,2006).4.If the mean of absolute difference of horizontalreferences(MADH)islessthanathresholdandM1isamemberoftheset{mode0,mode3,mode7},selectM1.Gotostep7.
5.Also, if the mean of absolute difference of verticalreferences (MADV)islessthanathresholdandM1isamember the set of{mode1,mode8},selectM1.Go tostep7.It isobvious that if the similarity ofhorizontal referencepixelsofablockishigh,thedifferencebetweenpredictionresultsobtainedwithpredictionmodes0,3and7willbevery small. Also, if the similarity of vertical referencepixelsofablockishigh,thesimilaritybetweenmodes1and8ishigh.6.Ifallaboveconditionsareunsatisfied,usetheimprovedPansmethod(explainedinSection3.1).7.Terminate.Assuch,intheworstcaseonlythreedifferent44intra-
modecostswillbeevaluated.Also,forI16MBand8 8chromablockstheimprovedPansmethodisused.
Toincreasethespeedofthealgorithm,weusedtheearly termination of RDO calculations for all proposedalgorithmsasin(Panetal.,2005).Proposedfastinter-predictionmodedecisionmethod
Themotivatedfacts about theinter-predictionare:1)The block with high motion activities, instead of hightexturaldetails,canbebettercodedusingsmallerblocksizes,whiletheblockwith lessmotion activitiescanbemoreefficientlyencodedusinglargerblocksizes,2)Itisobserved that in natural image sequences, when thevideo objects move, the different parts of the video
objects move in a similar manner. Then, homogenousregionsareencodedusing1616blocksizeswhilenon-homogenous regions are encoded using smaller blocksizesand3)Videobackgroundisnothomogeneous,butbecauseoftemporalredundancyitiscodedusing1616blocksizes.Fast inter-prediction mode decision using split andmergeprocedure
The split procedure partitions the MB into variablesizeblocksusingaquad-treeapproach.Inthismethod,a
macroblock is divided into equal-size quarters. Then,usingthesimilaritiesofmotionvectorsofadjacentblockswe will show how tomerge the sub-blocks for quarterdivisions. The proposed algorithm is summarized as
follows:1.Subtract the current frame
fromits previous frameandforany1616MBcompute
MBdifferenceinpixelsnonzeroofnumberN
iMBsetthetobelongpixelsofnumberN
im
i
2.If iim NN issmallerthanorequaltoapredetermined
threshold, choose direct mode as the finalmacroblocktype.3.Otherwise, split the block into four 88 blocks andconduct a new iteration of blockmatching for each ofthesefourdescendingblocks.4.Ifmotionvectorsof88sub-blockareequalorthreesub-block MVare the same and the forthunequal MVonly differbyone quarter-pixel distance, choose mode1(1616).Terminate.5.IfMV0=MV1andMV2=MV3(videTable2),choose8 16.Terminate.6.IfMV0=MV2andMV1=MV3,choose168.Terminate.7.RepeatSteps2to4foreach88blocks,exceptthatsub-blocksare44.8.Terminate.
Themode selection methodology employed in thispaperisasfollows.ForI4MB,I16MB,andPtheproposedalgorithmsareusedtoextractthebestmodeamongtherelated categoryandatlastRDO isusedto extract thefinalmode.
Analysisandcomplexity reductionofmultiple referenceframesmotionestimation
InH.264/AVC,motionestimationisallowedtosearchmultiple reference frames. Therefore, the requiredcomputationishighlyincreased,anditis inproportiontothenumberofsearchedreferenceframes.Experimental results showsome facts that are used todecideonthenumberofreferencesframes(Huangetal.2006).1. ForQP=20,QP=30andQP=40, it can be seen tha65%, 79%, and 95%, of macroblocks need only onereference frame, respectively. Therefore, the blocmatchingprocessshouldbeproceededfromthenearesreferenceframetothefarthestreferenceframe.2.Another interesting point is that lowbitrate cases armorelikelytohavethebestreferenceframesclosetothecurrentframethanhigherbitratecases.3.We can see that for QP=20, there are 59.84%05.00%,04.88%,28.11%,and02.17%ofthemacroblockselected as P1616, P168, P816, P88, and intrarespectively,whenonly onepreviousframe issearchedForQP=30,thereare75.97%,05.36%,05.45%,11.04%and02.18%ofthemacroblocksselectedasP16 16,P18,P816,P88,andintra,respectively.ForQP=40,th
Table2.MBdivision
MV0 MV1MV2 MV3
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
6/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1142
correspondingpercentagesare89.34%,03.21%,03.07%,01.69%,and02.69%.4. InH.264/AVC,theSKIPmodeisaspecialcaseofP1616. The percentages of SKIP macroblocks aftersearchingonereferenceframeare44.57%,62.69%,and79.14%forQP=20,30,and40,respectively.
ThisresultshowsthatalargepercentofMBsarecodedas 1616 or skip mode that use only one referenceframe,whileforlargeQPthisfactisamplified.Accordingtotheseobservations,inthefollowing,wehavelistedthesteps for each macroblock to check whether it isnecessarytosearchthenextreferenceframeattheendofeachreferenceframeloop.1.After the prediction procedure, residues aretransformed,quantized,andentropycoded.Ifwefacethesituation for which the transformed and quantizedcoefficients are very close tozero in the first referenceframe, stop the matching process for the remainingframes.
Table4.Differentmethodsusedinourexperiments.
2.Calculate the sum of absolute transform difference(SATD).Ifitislessthanathreshold(THSATD),stopthesearchingprocess.3.If the best reference frameis the previous frameandthe bestmotion vector isthe sameas that oftheSKIPmodeor1616modeandQP islargerthanathreshod(THQP),themultiplereferenceframesloopwillbeearlyterminated. The determination of THQP is empiricallyobtainedin(Huangetal.,2006).Intheproposedmethod76%-96% of computations for searching unnecessaryreferenceframescanbeavoided.
Also, similar to intra-prediction we used an earlytermination technique based on early detection of zeroblocks.Adaptivesearchrangedecisionalgorithm
InreferencesoftwareJM7.1,the fixed search rangewindow size equal to16 is used for motion estimation.This full search range is inefficient and increases thecomputationalcomplexity.Therefore,atwo-stepadaptivesearchrangedecisionalgorithmsispresented.Inthefirststep,wehaveadoptedthelengthofMVobtainedinthefirstreferenceframeasthemaximumsizeofthesearch
windowforthemotionestimationinthesecondreferenceframe. And, the length of MV in the second referenceframe is selected as the maximum size of the searchwindow in the third reference frame, and so forth.ThisstepoftheproposedalgorithmreducestheSWsizefor4referenceframes,andforthefirstreferenceframeafulsearchisapplied.Inthesecondstep,anadaptivesearchrangedecisionmethod ispresentedthatdeterminesthesearch window according to theMVs of previousMBs
The primary search window size is selected as theaverageofMVs of the encodedMBs immediatelyonleft, above, above the left, and above the right ocurrentMB,computedasSWR.x=(MVA.x+MVL.x+MVAR.x+MVAL.x)/4SWR.y=(MVA.y+MVL.y+MVAR.y+MVAL.y)/4where SWR.x, MVAx, MVL.x, MVAR.x, and MVL.xdenote the search window, motion vector abovemotion vector left,motionvector above theright,andmotionvectorabovetheleft,inthexaxis,respectivelySimilarly y stands for the y axis. By increasing thesearch window size around the computed SW, thespeed of motion estimation can be improved. The
increment of the SWsize isstoppedwhen the blockmatchinghasnotimprovedwithrespecttotheprimarycomputedSWsize.
ExperimentalresultsOurproposedalgorithmwasimplementedintoJM7.1
providedbyJVTaccordingtothetestconditionsspecifiedinVCEG-N81documentaslistedinTable3(Sullivaneal., 2001). Experiments were carried out on therecommended sequences with various quantizationparameters forIPPPtype.ForIPPPexperiments, thetotalnumberofframeswas300foreachsequence,andtheperiodofI-framewas100.TheusedtestplatformwasPentium IV-2.8 GHz with 256 Mbytes RAM. Wecompared the performance of our proposed algorithm
(fastmotionestimation+fastintra+fastinter+selectiveframe+ twostep adaptivesearch rangedecisions)withother available approaches. To show the impact odifferentpartsofthealgorithm,thesepartswereaddedadifferentstepsandthe resultswereanalyzed.Thus, theexperiments were ordered in seven states as listed inTable4.
Comparisons with the case of exhaustive search(RDO) were performed with respect to the change o
averagePSNR( PSNR),thechangeofaveragedata
Cate-gory
Intra-prediction(I4MB,I16MBChroma)
Inter-Predict ion Mult i-ReferenceAlgorithm
EarlyTermi-nation
RDO RDO RDO RDO YESM1 PansMethod RDO RDO YESM2 ImprovedPans
methodRDO RDO YES
M3 ProposedAlg. RDO RDO YESM4 ProposedAlg. Proposed
Alg.(split/merge)RDO YES
M5 ProposedAlg. ProposedAlg.(split/merge)
ProposedAlg.
YES
M6 ProposedAlg.
Proposed
Alg.(Adaptivesearchrange)
Proposed
Alg.
Yes
Table3.Experimentconditions.GOP IIIIIorIPPP
Codec JM7.1MVSearchRange 16
QuantizationParameter 10,16,24,28,36,40NumberofReferences 5
CommonCodingOption HadamardTransform,CABAC,RDOisenabled
Format CIFandQCIFNumberofFrames 300
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
7/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1143
bits( Bit),andthechangeofaverageencodingtime(Time),respectively.ThePSNRisderivedfrom
Therefore, in the rest ofthispaperwe used the overallPSNRvalueofallthreecomponentsY,U,andVusingEq.(11).
time bedefinedas
100%
ref
refprop
T
TTTime .
(8)
Also,thebitrateincreaseisdefinedas
Agroupofexperimentswerecarriedoutondifferent
sequences.Theencodingbitrates,thePSNRvalues,andthetimesavingfactor(ascomparedwiththeH.264RDOmethod) for four test sequences with differentquantization parameters are l isted in Tables 5-6.Generallyspeaking,ascanbeseenfromthistables,wehave saved 40-50% of the total encoding time at theexpenseofonly0.1~1.5%rateincreasein averageand0.015distortioninaverageforthesetestsequences.
Fig.6, 7, and8 show theexamples of RD and thecomplexity curves of sequences Akiyo (class A),
Foreman (Class B), and Stefan (Class C) for IPPPsequences. From these figures, one can see that theproposed fast decision scheme gives almost identicarate-distortion performance while providing a speed-upfactor (ratio ofencodingtimeusing the RDO techniqueand the proposedscheme)of3-6. In thesefigures, the
RDO,Pansmethod,improvedPansmethod,andthreeformsoffastproposedmethods(Fastintra+Fastinter+Fastmultireferenceframes)arecompared(videTable4)Fig.9-11showexamplesofclassificationresultsandfinalmode decision for Football, Stefan and walkingpersonsequences.Inthesefigures,thered,yellow,andgreencolorsshowtheskip,intra-mode,andinter-moderespectively.Inpart(c)ofthesefigures,theblackblocksare the macroblocks with inter-prediction and part (dshows the motion vector and intra-prediction mode foeachblock.Conclusion
In this paper, an efficient intra- and inter-modedecision algorithm for H.264/AVC standard wasintegrated using a two-step adaptive search windowdecision. Intheproposedalgorithm,we decreased theencoding time by reducing the number of candidatemodes. In order to achieve a better performanceadaptivesearchrangeformultiframereferenceandMVoencodedMBswith somestrengthpointswereproposedtoimprovepreviousalgorithms.
Inordertoevaluatetheimpactofdifferentpartsoftheproposedalgorithm, theywere addedstep by step andtherelatedexperimentalresultswereshown.The
Table6.ExperimentalresultsforIPPPtypesequences,ratecomparison.
%Bit 10 16 22 32 40
Forem
an
M1 1.650 1.540 1.536 1.354 1.230M2 1.050 1.004 0.962 0.987 0.870
M3 1.230 1.210 1.035 0.670 0.345M4 1.032 0.890 0.634 0.425 0.478
M5 1.130 0.735 0.917 0.890 0.098M6 1.624 0.789 0.604 0.346 0.512
News
M1 1.534 1.001 1.022 1.030 0.924M2 0.924 0.940 0.876 0.830 0.910M3 1.021 0.932 0.982 0.760 0.567M4 1.120 0.0942 0.876 0.320 0.314
M5 1.098 0.954 0.897 0.830 0.210M6 0.615 0.941 0.941 0.576 0.735
C
ontainer
M1 1.803 1.902 1.090 0.950 0.921M2 0.983 0.987 0.732 0.510 0.321
M3 1.673 0.982 0.879 0.340 0.450M4 1.345 0.941 0.512 0.710 0.342
M5 2.340 0.980 0.987 0.604 0.324M6 0.924 1.001 0.634 1.021 0.876
Silent
M1 0.923 0.987 0.875 0.875 0.745
M2 1.624 0.720 0.945 0.742 0.439M3 2.100 0.789 0.870 0.615 0.370M4 2.301 0.872 0.425 0.576 0.346M5 1.250 0.99 0.612 0.512 0.367
M6 0.830 0.210 0.830 0.897 0.932
Table5.ExperimentalresultsforIPPPtypesequences,distortioncomparison.
)( dBPSNR
10 16 22 32 40
Foreman
M1 -0.081 -0.079 -0.077 -0.065 -0.061
M2 -0.5 -0.2 -0.1 0.0 0.1M3 -0.09 -0.081 -0.073 -0.05 -0.05M4 -0.30 -0.27 -0.012 -0.015 -0.01
M5 -0.012 -0.01 -0.012 -.002 -.001M6 -0.014 -0.080 -0.032 -0.016 -0.10
News
M1 -0.073 -0.071 -0.067 -0.064 -0.062
M2 -0.047 -0.023 -0.01 0.00 0.0M3 -0.061 -0.060 -0.059 -0.50 -0.001
M4 -0.03 -0.01 -0.00 -0.01 0.01M5 -0.020 -0.13 -0.016 -0.013 -0.120M6 -0.01 -0.061 -0.013 -0.01 -0.03
Co
ntainer
M1 -0.089 -0.083 -0.081 -0.076 -0.074
M2 -0.51 -0.17 -0.1 -0.1 0.00M3 -0.080 -0.065 -0.067 -0.069 -0.032M4 -0.46 -0.21 -0.01 -0.02 -0.03M5 -0.10 -0.204 -0.013 -0.013 -0.032
M6 -0.012 -0.012 -0.05 -0.05 -0.01
Silent
M1 -0.032 -0.035 -0.033 -0.032 -0.029
M2 -0.04 -0.037 -0.023 -0.019 -0.01M3 -0.1 -0.02 -0.01 -0.02 0.00M4 -0.03 -0.01 -0.23 -0.012 -0.013M5 -0.324 -0.014 -0.080 -0.001 -0.035
M6 -0.01 -0.037 -0.1 -0.076 -0.071
MSEPSNR
2
10255log10
(7))
100%
ref
refprop
bitrate
bitratebitrateBitrate .
(9)
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
8/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1144
Fig.6.Akiyosequence(IPPPseq.).(a)Computationalcomplexity.
(a)
(b)
(b)R-Dperformance. (b)R-Dperformance.
(b
Fig.7.Foremansequence(IPPPseq.).(a)Computationalcomplexity.
(a)
(a)
Fig.8.Stefansequence(IPPPseq.).(a)Computationalcomplexity.(b)R-Dperformance.
b
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
9/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1145
(c)Classificationresultofintra-andinter-prediction.
(d)Motionvector,andintra-
predictionmodes.
Fig.9.SamplesofFootballsequence.
(a)MBdivision.(b)MBdivision,without
background.
(b)MBdivision,withoutbackground
Fig.10.SamplesofStefansequence.
(a)MBvision.
(c)Classificationresultofintra-andinter-prediction.
(d)Motionvector,andintra-predictionmodes.
(d)Motionvector,andintra-predictionmodes.(c)Classificationresultofintra-andinter-prediction.
a MBdivision. (b)MBdivision,withoutbackground.
Fig.11.Samplesofwalkingperson.
8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264
10/10
Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846
Research article Video coding M.Jafari & S.kasae
Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno
1146
experimentalresultsshowedthattheproposedalgorithmhasreducedthenumberofRDOcalculationswithrespecttotheoriginalandhasimprovedthepreviousalgorithmswithanegligiblelossinreconstructedvideoqualityandanegligible bitrate increase. As the experimental resultsshow,theproposedalgorithmcanbeusedforchallenging
work of intra- and inter-predictionmode decision in theH.264/AVCvideoencoderswithlowcomputationalcost.References1. PanF,LinX,RahardjaS,LimKP,LiZG,WuDand
WuS(2005)Fastmodedecisionalgorithmforintra-prediction inH.264/AVC video coding. IEEETrans.onCircuits & Systems for Video Tech. 15(7), 813-822.
2. WiegandT,SullivanGJ,BjontegaardGandLuthraA(2003) Overview of the H.264/AVC video codingstandard. IEEE Trans. on Circuits & Systems forVideoTech.13(7),560-576.
3. Yu-wenHuang,Bing-YuHsieh, Shao-YiChien,Shyh-
yihMaandLiang-GeeChen(2006)Analysiscomplexityreduction of multiple reference frames motionestimation in H.264/AVC. IEEE Trans. on Circuits &SystemsforVideoTech.16(4),507-522.
4. Changsung Kim, Qing Li, Jay CC and Kuo (2003)Fast intra-prediction model selection for H.264Codec. In: SPIE Intl. Symposium ITCOM 2003,Orlando,Florida,Sep.7-11.
5. LimKP,WuS,RahardjaDJS,LinX,PanfandLiZG(2003)Fastintermodeselection.9
thJointVideoTeam
(JVT)Mtg,SanDiego,CA,USA.6. KimHandAltunbasakY(2004a)Fastmodedecision
forinterpredictioninH.264.In:ICIP2004,Singapore.7. Kim YH, Yoo JW, Lee SW and Shin J (2004b)
Adaptive mode decision for H.264 encoder. In:ElectronicLett.40(19).
8. Yu CA and Martin G (2004) Advanced block sizeselection algorithm for inter frame coding inH.264/MPEG-4AVC.In:ICIP2004,Singapore.
9. DaiQ,ZhuDandDingR(2004)FastmodedecisionforinterpredictioninH.264.In:ICIP2004,Singapore.
10.PanFandLinX(2003)Fastmodedecisionforintra-prediction, ISO/IEC JTC1/SC29/WG11 and ITU-TSG16Q.6.In:JVT7thMeeting,PattayaII,Thailand.
11.PanF,LinX,RahardjaS,LimKPandLiZG(2004)Adirectional field based fast intra-mode decisionalgorithmforH.264VideoCoding.IEEEIntl.Conf.on
Multimedia&Expo.2,1147-1150.12.KimJandJeongJ(2005)Fastintra-modedecisioninH.264 video coding usingsimple directionalmasks.In:Proc.ofSPIE,Vol.5960,pp:1071-1079.
13.GargR,Jindal M and ChauhanM (2005) Statisticsbased fast intra-mode detection. In: Proc. of SPIE,Vol.5960,pp:2085-2091.
14.Jeon B and lee J (2003) Fast mode decision forH.264, ISO/IECJTC1/SC29/WG11andITU-TSG16Q.6,JVT.In:10
thMeeting,Waikoloa,Hawaii.
15.YangC,PoLandLamW(2004)AfastH.264intra-predictionalgorithmusingmacroblockproperties. InICIP.pp:461-464.
16.MengB,AuOC,WongCandLamH(2003)Efficientintra-predictionalgorithminH.264.In:IEEE.pp:837-840.
17.TsengC,WangHandYangJ(2004)Improvedandfast algorithms for intra-4X4 mode decision in
H.264/AVC.In:IEEE.pp:2128-2131.18.Hsu C, Ho M and Hong J (2006) An efficien
algorithmforintra-predictioninH.264.IEEE.pp:3536.
19.Jafari M and Kasaei S (2008) Fast intra and inteprediction mode decision in H.264 advanced videocoding.IJCSNS.8(5),130-140.
20.Jafari M and Kasaei S (2006) An efficient intpredictionmodedecisionalgorithmforH.263toH.2Transcoding. 10
th IEEE Intl. Conf. on Compu
Systems&Appl.pp:1082-1089.
21.Jafari M and Kasaei S (2005) Prioritisation of dapartitioned MPEG-4 video overGPRS/EGPRSmobnetworks, Asian Internet Engg. conf. (AINTECThailand.pp:68-82.
22.Richardson(2005),H.264/MPEG-4part10whitepapavailable: http://www.vcodex.fsnet.co.uesources.html.
23.SullivanG (2001)Recommended simulation commconditions forH.26L coding efficiencyexperimentslow resolution progressive scan source materiPresented at the 14th VCEG-N81 Meeting, SanBarbara,CA.
24.Jafari M and Kasaei S (2010) Adaptive search randecision for fast intra- and inter-prediction modecision in h.264/AVC. In:The2nd National ElectriEngg.Conf.NEEC,247-252.
25.A.Elyousfi,A.Tamtaoui,andE.Bouyakh(2010)Anefast intra prediction mode decision algorithmH.264/AVC encoder. Intl. J. Electrical & ElectronEngg.4(1),89-95.
26.Bentao, Wu and Minyuan (2010) an improvemenintra prediction algorithm for H.264. J. Natural Sci2010.15(4),340-344.
Top Related