Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264

8/3/2019 Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264

1/10

Indian Journal of Science and Technology Vol. 4 No. 9 (Sep 2011) ISSN: 0974- 6846

Research article Video coding M.Jafari & S.kasae

Indian Society for Education and Environment (iSee) http://www.indjst.org Indian J.Sci.Techno

1137

Adaptivesearchrangedecisionforfastintra-andinter-predictionmodedecisioninH.264/AVC

MehdiJafari

1andShohrehKasaei

2

1DepartmentofElectricalEngineering,KermanBranch,IslamicAzadUniversity,Kerman,Iran

2DepartmentofComputerEngineering,SharifUniversityofTechnology,Tehran,Iran

[email protected];[email protected]

H.264/AVC,thelatestvideocodingstandard,adoptsrate-distortionoptimization(RDO)techniquetoobtainthebestintra- andinter-prediction,whilemaximizingvisualqualityandminimizingthe requiredbitrate.However,the fullRDcostcalculationforallintra-predictionmodes,theexhaustivesearchesforfindingoptimalmotionvectorsforallblocksizes,andthemultiplereferencesframeprocedureconsiderablyincreaseitscomputationalcomplexitywiththeallowednumber of prediction modes. The authors have previously proposed an approach for both inter- and intra-modedecisions that takes into account the several effective parameters, image content type, quantization parametercorrelationofmotionvectorsandmotiondetailsofvideoobjects.Inordertoreducethecomplexity,hereweproposeanewtwo-stepadaptivesearchrangedecisionbydevelopingacombinationofpreviousalgorithm.Thefirststepofouradaptivesearch rangedecisiondetermines thesearchrange forcurrentMBaccording to themotionvectorsof thepreviousMBs.Inthesecondstep,themotionvectorobtainedinthecurrentframeisusedtoadaptivelysearchingthewindowsizeofthepreviousreferenceframeformulti-framemotionestimation.Assuch,onlyasubsetofinter-and

intra-modesis chosen forRDOcalculation.Experimentalresultsshowthattheproposedalgorithmreducesthe totaencodingcost with negligible loss inPSNRand a slightly increase in the requiredbitrate,when compared tooupreviousalgorithmandotheralgorithmsreportedintheliterature.

Keywords:H.264/AVC,intra-andinter-prediction,rate-distortionoptimization,similarpredicted-pixels,split/merge.

IntroductionAs recent multimedia applications are growing

rapidly, videocompression requireshigher performanceas well as new features. The newest video codingstandard is developed by the joint of video teams ofISO/IEC MPEG and ITU_T VCEG as the internationalstandard 14496-10 (MPEG-4 part 10) advanced videocoding (AVC) (Wiegandetal., 2003; Panetal., 2005).

H.264/AVChasgainedmoreandmoreattention;mainlydue to its high coding efficiency, minor increase indecoder complexity compared to existing standards,adaptation to delay constraints, error robustness, andnetwork friendliness (Wiegand et al., 2003; Pan et al.,2005). To achieve an outstanding coding performance,H.264/AVCemploysseveralpowerful coding techniquessuchasdirectionalpredictionofintra-codedblocks,inter-predictionwithvariableblock-sizemotioncompensation,multi-reference framemotion estimation,motion vectorswithquarter-pel accuracy, in-loop deblocking filter, 44integertransform,andtheforth.Accordingtothesenewfeatures, the encoder computational complexity hasextremely increased compared to previous standards.

However, theses functionalities have made H.264/AVCdifficult for applications with low computationalcapabil ities. Thus, until now, the reduction of itscomplexity is a challenging task. The improvement incoding performance comes mainly from the intra- andinter-prediction part. H.264/AVC employs the LagrangeRDOmethod toachieve the best codingmodeofintra-and inter-prediction with highest coding efficiency. TheRDO technique requires a lot of computations since ittests the encoding process with all possible coding

modesofintra-andinter-coding,andcalculatestheirRDcoststo choose themodehaving theminimumrequiredbitrate. The reference software of H.264/AVC, JM7.1adoptsafullsearchforbothmotionestimationandintraprediction. The run-time percentage of each function isshown in Fig.1 (Huang et al., 2006). As shown in this

figure, motion estimation is the most computationallyintensive part. The instruction profile of the referencesoftwareshowsthatreal-timeencodingofCIF30Hzvideo(baseline options, search range [-16.75, +16.75], five

reference frames) requires 314,994 million instructionspersecondandmemoryaccessof471,299MBytes/sec.

Also, H.264/AVC offers a rich set of predictionpatterns for intra-prediction (i.e., nine prediction modesfor44lumablocksandfourpredictionmodesfor16 16lumablocks). Theintra-predictionmodedecisionis verycomplexandthenumberofcomputingRDcostvaluesfolumaandchromaofamacroblockis592(Changsungeal.,2003).Thus,thecomputationalburdenofthesetypesofbruteforce-searchingalgorithmisfarmoredemanding

Fig.1.Run-timepercentagesoffunctionalblocksinH.264/AVCbaselineencoder.


2/10




1138

thananyexistingvideocodingalgorithm.Thechallengingtopicoffastmodeselectionforintra-

andinter-predictionisconsideredinthispaper.Toreducethe computational complexity, many algorithms havebeenproposed.For fast inter-prediction modedecision,the early termination technique (Tourapis et al., 2003)

reducesthenumberofpotentialpredictionmodes.Lim etal. (2003) proposed a classification method. Later, fastinter-mode selection algorithms were proposed (Kim etal.,2004a,b;Martinetal.,2004)toalleviatetheencodercomplexitydue to themotionestimationandinter-modedecisions. Dai et al. (2004), presented a fast inter-prediction mode decision based on pre-encodingprocess.

Fast intra-mode decision algorithms using edgedetection histogram and local edge detection areproposed by Pan et al. (2003, 2004, 2005). However,theirpreprocessingstagesstillconsumeacodingtimetodetect theedgedirectionand toclassify itintoa limited

direction. Also, there exist fast algorithms to select theoptimal intra-prediction mode using simple directionalmasks (Kim et al., 2005)with saving time of 70%, andstatistical-basedmethods(Garg etal.,2005)withsavingtimeof45%.Anotherfastintra-modedecisionschemeisproposed by Jeon et al. (2003), where the encodingspeedisapproximately30%fasterthanthatoftheRDOmethod. Anewfast intra-predictionalgorithmbasedonmacroblockproperties(FIPAMP)ispresentedbyYangetal. (2004). This algorithm achieves 10% to 40% ofcomputation reduction while maintaining similar PSNRandbitrate performance ofH.264/AVC codes.Mengetal., (2003) presented an efficient intra-prediction (EIP)algorithm based on early termination, selective

computation of highly probable modes, and partialcomputationofthecostfunction.Also,animprovedcostfunctiontoimprovethecodingperformanceisproposedbyTsengetal.(2004;Bentaoetal.,2010;Tamtaouietal.,2010). AlsoHsuetal. (2006) obtaineda fastalgorithmbased on local edge information by calculating edgefeatureparameters.

JafariandKasaei(2008)proposedanewapproachfor both inter-and intra-mode size selection. For intra-prediction mode decision, we improved Pans method(Pan et al., 2003, 2004, 2005), and for inter-predictionmode decision, the split/merge procedure was used.Also, an effective method for accelerating the multiple

reference frames ME without significant loss of videoquality was proposed that was basedon analyzing theavailableinformationobtainedfrompreviouslyprocessedframes.

This paper presents a two-step adaptive searchrange decision algorithm to improve the proposedalgorithm(Jafari&Kasaei,2008;2010).Inthefirststep,theproposedadaptive algorithmdeterminesthe searchrangeforcurrentMBaccordingtoMVsofpreviousMBs.In the second step, the length of MV obtained in the

current reference frame is considered as the maximumsize of the search window for motion estimation in thepreviousreferenceframe.We haveverified thedifferenparts of the proposed algorithm step by step byimplementing it on the JM7.1 reference software andhavecompared itsperformancewith previousandothe

availablefastalgorithms.Experimentalresultsshowthathe proposedmethod reduces the encoding cost up to40% with a negligible loss in the reconstructed videoqualityandanegligibleincreaseintherequiredbitrate.ModedecisionandRDOinH.264/AVC

In commonwith earlier standards, H.264/AVCdoesnot define the encoder, but defines the syntax of anencoded video bitstream together with the method odecoding the bitstream (Jafari &Kasaei, 2005; Jafari &Kasaei, 2006). The codec combines intra-pictureprediction with inter-picture prediction to exploit thespatialandtemporalredundancy,respectively.Intra-predictionmodedecision

Intra-prediction is based on the observation thaadjacent macroblocks tend to have similar propertiesPrediction may be formed for each 44 luma block(I4MB),1616lumaMB(I16MB),and88chromablock.

For prediction of 44 luminance blocks, the 9directional modes consist of a DC prediction (Mode 2and8directionalmodes(labeled0,1,3,4,5,6,7,and8)as shown inFig.2(a). In Fig. 2(b), the block (values opixels a to p) is to be predicted using A to Q pixevalues.

TheDCprediction(mode2)isusefulforthoseblockswithlittleornolocalactivities,theothermodes(1-8)mayonly be used if all required prediction samples areavailable.

For regions with less spatial details (i.e., flat regions)H.264/AVCsupports1616intra-coding;inwhichoneofour prediction modes (DC, vertical, horizontal, andplanar) is chosen for prediction of the entire luminancecomponentsofthemacroblock(Richardson,2005).

H.264/AVC supports four chroma prediction modesfor88chrominanceblocks,similartothatoftheI16MBprediction, except that the order of mode numbers isdifferent: DC (Mode 0), horizontal (Mode 1), vertica(Mode2),andplane(Mode3).

Fig.2.(a)Intra-predictionmodesfor4 4luminanceblocks.(b)Labelingofpredictionsamples.

(a) (b)


3/10




1139

Inter-predictionmodedecisionInter-prediction is based onusingmotionestimation

and compensation to take advantage of temporalredundanciesthatexistbetweensuccessiveframes.Theimportant differences fromearlier standards include thesupportforarangeofblocksizes(downto4 4),multiple

referenceframes,andfinesub-pixelmotionvectors(1/4pixelinthelumacomponent).

H.264/AVC supports motion compensation blocksizes ranging from 1616 to 44 luminance sampleswith many options between the two. The luminancecomponentofeachmacroblock(1616samples)maybesplitupin4waysas1616,168,816or88.Ifthe88mode is chosen, each of the four 88 macroblockpartitionswithinthemacroblockmaybesplitinafurther4ways as 88, 84, 48 or 44. These partitions andsub-partitions give rise to a large number of possiblecombinationswithineachmacroblock.

A separate motion vector is required for each

partition or sub-partition. Each motion vector must becoded and transmitted; in addition, the choice ofpartition(s) must be encoded in the compressedbitstream.Choosinga largepartitionsize (e.g., 1616,168, 816) means that a small number of bits arerequired to indicate the choice of motion vector(s) andthe typeof partition; however,the motion compensatedresidualmay contain a significant amount of energy inframeareaswithhighdetails.Choosingasmallpartitionsize (e.g., 84, 44, etc.) may give a lower energyresidualaftermotioncompensation,butrequiresalargernumberofbitstosignalthemotionvectorsandthechoiceofpartition(s).Thechoiceofpartitionsizethereforehasasignificantimpactoncompression(asmallpartitionsize

maybebeneficialfordetailedareas).H.264/AVC as an enhanced reference picture

selectionasH.263++enablesefficientcodingbyallowinganencodertoselect(formotioncompensationpurposes)among a large number of pictures that have beendecodedandstoredinthedecoder.RDOprocedure

For I-frames, all MBs are predicted as Intra.H.264/AVC encoder encodes the best mode using allmodecombinationsoflumaandchromaandchoosestheonethatgivesthebestRDOperformance.ForP-frames,intra-andinter-predictionisdoneandRDOisusedtofindthebestprediction.TheRDOproceduretoencodeone

MBin an I-framehasbeen reported(Changsungetal.,2003). According to the procedure, intra-prediction inH.264/AVC, thenumberofmodecombinationsfor lumaand chroma blocks in amacroblock is N8 (16N4 +N16), whereN8, N4, andN16, denote the number ofmodesfor88chromablocks,and4 4and1616lumablocks, respectively(Changsung et al., 2003). Also,accordingtotheRDOprocedureofinter-prediction,forMblockmodes,Nreferenceframes,and W searchrange,

2)12.(. WNM positions should be tested for a single

referenceframeandasingleblockmode.Thismakesthecomplexity of the encoder extremely high. In order toreduce the encoding complexity with little RDperformancedegradation, fast intra- and intra-predictionmodedecisionmethodsareproposed.Proposedfastintra-predictionmodedecisionmethods

This section presents a new fast intra-predictionalgorithm. The proposed method is based on severafactsthatwehaveobservedfromthestatisticsofdifferensequences as: 1)Fig.3shows the total number of 44and 1616 intra-coded macroblocks at differenquantizationparameters(QPs).Ascanbeseenfromthifigure, fast detection of 44 intra-prediction mode cansignificantly improve the encoding speed at low QPswhile1616 intra-predictions can improve thespeed alargeQPs, 2) The prediction modes of each block arecorrelated with those of neighboring 44 luminanceblocks, 3) Normally, pixels along the direction of locaedgeshavesimilarvalues.Therefore, agoodpredictioncan be achieved by predicting the pixels using theneighboringpixelsthatlieinthesameedgedirections,4)The optimal mode (found by a full-search) and othegood (second or third best)modes are most likely tohavesimilardirections,5)Thedirectionalfeaturesof4 4blocks can be preserved roughly after down-samplingand 6) Experimental results show that the referencepixelsofa44lumablockarelikelytobesimilartoeach

other(Jafari&Kasaei,2005).Basedon theseobservations, wehave proposeda

fast intra-prediction mode selection algorithm. In thissection, some new ideas are combined with the fasmode selection algorithm introduced (Lim et al., 2003Kim & Altunbasak, 2004; Pan et al., 2005) to improvetheirefficiency.ImprovedpansmethodforfastdecisionofI4MB

Panetal.(2005)presentedafastmodeselectionfointra-prediction method in which the average edge

Fig.3.Numberof4 4and16 16intra-codedmacroblocksfordifferentquantizationparameters.

10 15 20 25 30 35 40 45 500

1000

2000

3000

4000

5000

6000

Quantization Parameter

Num

berofM

B

16x16 MB

4x4 MB


4/10




1140

direction of a given block is measured. The Sobeloperatorsarefirstusedtoobtainthedirectionalvectorofeachpixelinablockby

},{ ,,, jijiji dydxD

(1)

WheretheSobeloperatoraredefinedby

,1,11,11,1,11,1,

,11,1,11,11,1,1,

22

22

jijijijijijiji

jijijijijijiji

PPPPPPdy

PPPPPPdx

(2)

The amplitude and angle of each edge vector can becalculatedusing

||||)( ,,, jijiji dydxDAmp

(3)

and,

)arctan(180

)(,

,

,

ji

jio

jidx

dyDAng

(4)

WhereAng(.)isfittedintooneofthe8modes.

Then, the edge direction histogram (EDH) of theblock is found (thatindicates the number ofpixels withsimilar edge directions). Therefore, the cell k with the

maximumamount indicatesthat thereis a strong edgealong that direction in theblockand thus is assignedas the dominant blockdirection. Fig.5 shows theEDHoftheimageshowninFig.4.

InthePansmethod,forI4MBthereare4modes(1DC, 1 from maximum

amplitudeofEDH,andits2neighbors)with2modes

(1DCmodeand1directional)foreach1616lumablockand 88 chroma block. Here, to improve the Pansmethod, we will eliminate the DC mode from thecandidates if the direction of the block is obvious, andotherwise,wechooseonlyDCmode.TocheckwhethertheDCoftheblockisclearornot,thediffvalue,givenin

Eq.(5),iscomputedtocheckwhetheritissmallerthanapredeterminedthresholdornot,using

4)8(

||

15

0

15

0

i

i

i

i

i

pavg

pavgdiff

(5)

TheimprovedPansmethodisproposedasfollows:1.Find the maximum value of the edge directionahistogramH.DenotethecorrespondingmodebyM1.2. Ifdiff>T,carryouttheRDOprocedurefor3modesatthemost(M1anditstwoneighbors).3. Else, ifdiff T1consider only the primary prediction mode decided by

edge direction histogram as a candidate for the bespredictionmode.Thediff1inthiscaseiscomputedby

6)32(

||1

64

0

64

0

i

i

i

i

i

pavg

pavgdiff(6)

5. If diff1


5/10




1141

1.Find the maximum value of EDH. Denote thecorrespondingmodebyM1.2.Ifthemodes for one ofthe top orleftblocksareM1,then choose M1 as the best candidate mode for thecurrentblock.Gotostep7.3.For 44 luma block, compute themean of absolute

difference (MAD)of its reference pixels. If this value issmallerthanapredeterminedthreshold,selectM1.Gotostep7.Thisresultisyieldedfromthefactthatifthesimilarityofreferencepixelsofablockishigh,thedifferencebetweendifferent prediction modes will be very small. For thiscase,itisnotnecessarytocheckall9predictionmodes,but only one prediction mode is enough (Hsu et al.,2006).4.If the mean of absolute difference of horizontalreferences(MADH)islessthanathresholdandM1isamemberoftheset{mode0,mode3,mode7},selectM1.Gotostep7.

5.Also, if the mean of absolute difference of verticalreferences (MADV)islessthanathresholdandM1isamember the set of{mode1,mode8},selectM1.Go tostep7.It isobvious that if the similarity ofhorizontal referencepixelsofablockishigh,thedifferencebetweenpredictionresultsobtainedwithpredictionmodes0,3and7willbevery small. Also, if the similarity of vertical referencepixelsofablockishigh,thesimilaritybetweenmodes1and8ishigh.6.Ifallaboveconditionsareunsatisfied,usetheimprovedPansmethod(explainedinSection3.1).7.Terminate.Assuch,intheworstcaseonlythreedifferent44intra-

modecostswillbeevaluated.Also,forI16MBand8 8chromablockstheimprovedPansmethodisused.

Toincreasethespeedofthealgorithm,weusedtheearly termination of RDO calculations for all proposedalgorithmsasin(Panetal.,2005).Proposedfastinter-predictionmodedecisionmethod

Themotivatedfacts about theinter-predictionare:1)The block with high motion activities, instead of hightexturaldetails,canbebettercodedusingsmallerblocksizes,whiletheblockwith lessmotion activitiescanbemoreefficientlyencodedusinglargerblocksizes,2)Itisobserved that in natural image sequences, when thevideo objects move, the different parts of the video

objects move in a similar manner. Then, homogenousregionsareencodedusing1616blocksizeswhilenon-homogenous regions are encoded using smaller blocksizesand3)Videobackgroundisnothomogeneous,butbecauseoftemporalredundancyitiscodedusing1616blocksizes.Fast inter-prediction mode decision using split andmergeprocedure

The split procedure partitions the MB into variablesizeblocksusingaquad-treeapproach.Inthismethod,a

macroblock is divided into equal-size quarters. Then,usingthesimilaritiesofmotionvectorsofadjacentblockswe will show how tomerge the sub-blocks for quarterdivisions. The proposed algorithm is summarized as

follows:1.Subtract the current frame

fromits previous frameandforany1616MBcompute

MBdifferenceinpixelsnonzeroofnumberN

iMBsetthetobelongpixelsofnumberN

im

i

2.If iim NN issmallerthanorequaltoapredetermined

threshold, choose direct mode as the finalmacroblocktype.3.Otherwise, split the block into four 88 blocks andconduct a new iteration of blockmatching for each ofthesefourdescendingblocks.4.Ifmotionvectorsof88sub-blockareequalorthreesub-block MVare the same and the forthunequal MVonly differbyone quarter-pixel distance, choose mode1(1616).Terminate.5.IfMV0=MV1andMV2=MV3(videTable2),choose8 16.Terminate.6.IfMV0=MV2andMV1=MV3,choose168.Terminate.7.RepeatSteps2to4foreach88blocks,exceptthatsub-blocksare44.8.Terminate.

Themode selection methodology employed in thispaperisasfollows.ForI4MB,I16MB,andPtheproposedalgorithmsareusedtoextractthebestmodeamongtherelated categoryandatlastRDO isusedto extract thefinalmode.

Analysisandcomplexity reductionofmultiple referenceframesmotionestimation

InH.264/AVC,motionestimationisallowedtosearchmultiple reference frames. Therefore, the requiredcomputationishighlyincreased,anditis inproportiontothenumberofsearchedreferenceframes.Experimental results showsome facts that are used todecideonthenumberofreferencesframes(Huangetal.2006).1. ForQP=20,QP=30andQP=40, it can be seen tha65%, 79%, and 95%, of macroblocks need only onereference frame, respectively. Therefore, the blocmatchingprocessshouldbeproceededfromthenearesreferenceframetothefarthestreferenceframe.2.Another interesting point is that lowbitrate cases armorelikelytohavethebestreferenceframesclosetothecurrentframethanhigherbitratecases.3.We can see that for QP=20, there are 59.84%05.00%,04.88%,28.11%,and02.17%ofthemacroblockselected as P1616, P168, P816, P88, and intrarespectively,whenonly onepreviousframe issearchedForQP=30,thereare75.97%,05.36%,05.45%,11.04%and02.18%ofthemacroblocksselectedasP16 16,P18,P816,P88,andintra,respectively.ForQP=40,th

Table2.MBdivision

MV0 MV1MV2 MV3


6/10




1142

correspondingpercentagesare89.34%,03.21%,03.07%,01.69%,and02.69%.4. InH.264/AVC,theSKIPmodeisaspecialcaseofP1616. The percentages of SKIP macroblocks aftersearchingonereferenceframeare44.57%,62.69%,and79.14%forQP=20,30,and40,respectively.

ThisresultshowsthatalargepercentofMBsarecodedas 1616 or skip mode that use only one referenceframe,whileforlargeQPthisfactisamplified.Accordingtotheseobservations,inthefollowing,wehavelistedthesteps for each macroblock to check whether it isnecessarytosearchthenextreferenceframeattheendofeachreferenceframeloop.1.After the prediction procedure, residues aretransformed,quantized,andentropycoded.Ifwefacethesituation for which the transformed and quantizedcoefficients are very close tozero in the first referenceframe, stop the matching process for the remainingframes.

Table4.Differentmethodsusedinourexperiments.

2.Calculate the sum of absolute transform difference(SATD).Ifitislessthanathreshold(THSATD),stopthesearchingprocess.3.If the best reference frameis the previous frameandthe bestmotion vector isthe sameas that oftheSKIPmodeor1616modeandQP islargerthanathreshod(THQP),themultiplereferenceframesloopwillbeearlyterminated. The determination of THQP is empiricallyobtainedin(Huangetal.,2006).Intheproposedmethod76%-96% of computations for searching unnecessaryreferenceframescanbeavoided.

Also, similar to intra-prediction we used an earlytermination technique based on early detection of zeroblocks.Adaptivesearchrangedecisionalgorithm

InreferencesoftwareJM7.1,the fixed search rangewindow size equal to16 is used for motion estimation.This full search range is inefficient and increases thecomputationalcomplexity.Therefore,atwo-stepadaptivesearchrangedecisionalgorithmsispresented.Inthefirststep,wehaveadoptedthelengthofMVobtainedinthefirstreferenceframeasthemaximumsizeofthesearch

windowforthemotionestimationinthesecondreferenceframe. And, the length of MV in the second referenceframe is selected as the maximum size of the searchwindow in the third reference frame, and so forth.ThisstepoftheproposedalgorithmreducestheSWsizefor4referenceframes,andforthefirstreferenceframeafulsearchisapplied.Inthesecondstep,anadaptivesearchrangedecisionmethod ispresentedthatdeterminesthesearch window according to theMVs of previousMBs

The primary search window size is selected as theaverageofMVs of the encodedMBs immediatelyonleft, above, above the left, and above the right ocurrentMB,computedasSWR.x=(MVA.x+MVL.x+MVAR.x+MVAL.x)/4SWR.y=(MVA.y+MVL.y+MVAR.y+MVAL.y)/4where SWR.x, MVAx, MVL.x, MVAR.x, and MVL.xdenote the search window, motion vector abovemotion vector left,motionvector above theright,andmotionvectorabovetheleft,inthexaxis,respectivelySimilarly y stands for the y axis. By increasing thesearch window size around the computed SW, thespeed of motion estimation can be improved. The

increment of the SWsize isstoppedwhen the blockmatchinghasnotimprovedwithrespecttotheprimarycomputedSWsize.

ExperimentalresultsOurproposedalgorithmwasimplementedintoJM7.1

providedbyJVTaccordingtothetestconditionsspecifiedinVCEG-N81documentaslistedinTable3(Sullivaneal., 2001). Experiments were carried out on therecommended sequences with various quantizationparameters forIPPPtype.ForIPPPexperiments, thetotalnumberofframeswas300foreachsequence,andtheperiodofI-framewas100.TheusedtestplatformwasPentium IV-2.8 GHz with 256 Mbytes RAM. Wecompared the performance of our proposed algorithm

(fastmotionestimation+fastintra+fastinter+selectiveframe+ twostep adaptivesearch rangedecisions)withother available approaches. To show the impact odifferentpartsofthealgorithm,thesepartswereaddedadifferentstepsandthe resultswereanalyzed.Thus, theexperiments were ordered in seven states as listed inTable4.

Comparisons with the case of exhaustive search(RDO) were performed with respect to the change o

averagePSNR( PSNR),thechangeofaveragedata

Cate-gory

Intra-prediction(I4MB,I16MBChroma)

Inter-Predict ion Mult i-ReferenceAlgorithm

EarlyTermi-nation

RDO RDO RDO RDO YESM1 PansMethod RDO RDO YESM2 ImprovedPans

methodRDO RDO YES

M3 ProposedAlg. RDO RDO YESM4 ProposedAlg. Proposed

Alg.(split/merge)RDO YES

M5 ProposedAlg. ProposedAlg.(split/merge)

ProposedAlg.

YES

M6 ProposedAlg.

Proposed

Alg.(Adaptivesearchrange)

Proposed

Alg.

Yes

Table3.Experimentconditions.GOP IIIIIorIPPP

Codec JM7.1MVSearchRange 16

QuantizationParameter 10,16,24,28,36,40NumberofReferences 5

CommonCodingOption HadamardTransform,CABAC,RDOisenabled

Format CIFandQCIFNumberofFrames 300


7/10




1143

bits( Bit),andthechangeofaverageencodingtime(Time),respectively.ThePSNRisderivedfrom

Therefore, in the rest ofthispaperwe used the overallPSNRvalueofallthreecomponentsY,U,andVusingEq.(11).

time bedefinedas

100%

ref

refprop

T

TTTime .

(8)

Also,thebitrateincreaseisdefinedas

Agroupofexperimentswerecarriedoutondifferent

sequences.Theencodingbitrates,thePSNRvalues,andthetimesavingfactor(ascomparedwiththeH.264RDOmethod) for four test sequences with differentquantization parameters are l isted in Tables 5-6.Generallyspeaking,ascanbeseenfromthistables,wehave saved 40-50% of the total encoding time at theexpenseofonly0.1~1.5%rateincreasein averageand0.015distortioninaverageforthesetestsequences.

Fig.6, 7, and8 show theexamples of RD and thecomplexity curves of sequences Akiyo (class A),

Foreman (Class B), and Stefan (Class C) for IPPPsequences. From these figures, one can see that theproposed fast decision scheme gives almost identicarate-distortion performance while providing a speed-upfactor (ratio ofencodingtimeusing the RDO techniqueand the proposedscheme)of3-6. In thesefigures, the

RDO,Pansmethod,improvedPansmethod,andthreeformsoffastproposedmethods(Fastintra+Fastinter+Fastmultireferenceframes)arecompared(videTable4)Fig.9-11showexamplesofclassificationresultsandfinalmode decision for Football, Stefan and walkingpersonsequences.Inthesefigures,thered,yellow,andgreencolorsshowtheskip,intra-mode,andinter-moderespectively.Inpart(c)ofthesefigures,theblackblocksare the macroblocks with inter-prediction and part (dshows the motion vector and intra-prediction mode foeachblock.Conclusion

In this paper, an efficient intra- and inter-modedecision algorithm for H.264/AVC standard wasintegrated using a two-step adaptive search windowdecision. Intheproposedalgorithm,we decreased theencoding time by reducing the number of candidatemodes. In order to achieve a better performanceadaptivesearchrangeformultiframereferenceandMVoencodedMBswith somestrengthpointswereproposedtoimprovepreviousalgorithms.

Inordertoevaluatetheimpactofdifferentpartsoftheproposedalgorithm, theywere addedstep by step andtherelatedexperimentalresultswereshown.The

Table6.ExperimentalresultsforIPPPtypesequences,ratecomparison.

%Bit 10 16 22 32 40

Forem

an

M1 1.650 1.540 1.536 1.354 1.230M2 1.050 1.004 0.962 0.987 0.870

M3 1.230 1.210 1.035 0.670 0.345M4 1.032 0.890 0.634 0.425 0.478

M5 1.130 0.735 0.917 0.890 0.098M6 1.624 0.789 0.604 0.346 0.512

News

M1 1.534 1.001 1.022 1.030 0.924M2 0.924 0.940 0.876 0.830 0.910M3 1.021 0.932 0.982 0.760 0.567M4 1.120 0.0942 0.876 0.320 0.314

M5 1.098 0.954 0.897 0.830 0.210M6 0.615 0.941 0.941 0.576 0.735

C

ontainer

M1 1.803 1.902 1.090 0.950 0.921M2 0.983 0.987 0.732 0.510 0.321

M3 1.673 0.982 0.879 0.340 0.450M4 1.345 0.941 0.512 0.710 0.342

M5 2.340 0.980 0.987 0.604 0.324M6 0.924 1.001 0.634 1.021 0.876

Silent

M1 0.923 0.987 0.875 0.875 0.745

M2 1.624 0.720 0.945 0.742 0.439M3 2.100 0.789 0.870 0.615 0.370M4 2.301 0.872 0.425 0.576 0.346M5 1.250 0.99 0.612 0.512 0.367

M6 0.830 0.210 0.830 0.897 0.932

Table5.ExperimentalresultsforIPPPtypesequences,distortioncomparison.

)( dBPSNR

10 16 22 32 40

Foreman

M1 -0.081 -0.079 -0.077 -0.065 -0.061

M2 -0.5 -0.2 -0.1 0.0 0.1M3 -0.09 -0.081 -0.073 -0.05 -0.05M4 -0.30 -0.27 -0.012 -0.015 -0.01

M5 -0.012 -0.01 -0.012 -.002 -.001M6 -0.014 -0.080 -0.032 -0.016 -0.10

News

M1 -0.073 -0.071 -0.067 -0.064 -0.062

M2 -0.047 -0.023 -0.01 0.00 0.0M3 -0.061 -0.060 -0.059 -0.50 -0.001

M4 -0.03 -0.01 -0.00 -0.01 0.01M5 -0.020 -0.13 -0.016 -0.013 -0.120M6 -0.01 -0.061 -0.013 -0.01 -0.03

Co

ntainer

M1 -0.089 -0.083 -0.081 -0.076 -0.074

M2 -0.51 -0.17 -0.1 -0.1 0.00M3 -0.080 -0.065 -0.067 -0.069 -0.032M4 -0.46 -0.21 -0.01 -0.02 -0.03M5 -0.10 -0.204 -0.013 -0.013 -0.032

M6 -0.012 -0.012 -0.05 -0.05 -0.01

Silent

M1 -0.032 -0.035 -0.033 -0.032 -0.029

M2 -0.04 -0.037 -0.023 -0.019 -0.01M3 -0.1 -0.02 -0.01 -0.02 0.00M4 -0.03 -0.01 -0.23 -0.012 -0.013M5 -0.324 -0.014 -0.080 -0.001 -0.035

M6 -0.01 -0.037 -0.1 -0.076 -0.071

MSEPSNR

2

10255log10

(7))

100%

ref

refprop

bitrate

bitratebitrateBitrate .

(9)


8/10




1144

Fig.6.Akiyosequence(IPPPseq.).(a)Computationalcomplexity.

(a)

(b)

(b)R-Dperformance. (b)R-Dperformance.

(b

Fig.7.Foremansequence(IPPPseq.).(a)Computationalcomplexity.

(a)

(a)

Fig.8.Stefansequence(IPPPseq.).(a)Computationalcomplexity.(b)R-Dperformance.

b


9/10




1145

(c)Classificationresultofintra-andinter-prediction.

(d)Motionvector,andintra-

predictionmodes.

Fig.9.SamplesofFootballsequence.

(a)MBdivision.(b)MBdivision,without

background.

(b)MBdivision,withoutbackground

Fig.10.SamplesofStefansequence.

(a)MBvision.

(c)Classificationresultofintra-andinter-prediction.

(d)Motionvector,andintra-predictionmodes.

(d)Motionvector,andintra-predictionmodes.(c)Classificationresultofintra-andinter-prediction.

a MBdivision. (b)MBdivision,withoutbackground.

Fig.11.Samplesofwalkingperson.


10/10




1146

experimentalresultsshowedthattheproposedalgorithmhasreducedthenumberofRDOcalculationswithrespecttotheoriginalandhasimprovedthepreviousalgorithmswithanegligiblelossinreconstructedvideoqualityandanegligible bitrate increase. As the experimental resultsshow,theproposedalgorithmcanbeusedforchallenging

work of intra- and inter-predictionmode decision in theH.264/AVCvideoencoderswithlowcomputationalcost.References1. PanF,LinX,RahardjaS,LimKP,LiZG,WuDand

WuS(2005)Fastmodedecisionalgorithmforintra-prediction inH.264/AVC video coding. IEEETrans.onCircuits & Systems for Video Tech. 15(7), 813-822.

2. WiegandT,SullivanGJ,BjontegaardGandLuthraA(2003) Overview of the H.264/AVC video codingstandard. IEEE Trans. on Circuits & Systems forVideoTech.13(7),560-576.

3. Yu-wenHuang,Bing-YuHsieh, Shao-YiChien,Shyh-

yihMaandLiang-GeeChen(2006)Analysiscomplexityreduction of multiple reference frames motionestimation in H.264/AVC. IEEE Trans. on Circuits &SystemsforVideoTech.16(4),507-522.

4. Changsung Kim, Qing Li, Jay CC and Kuo (2003)Fast intra-prediction model selection for H.264Codec. In: SPIE Intl. Symposium ITCOM 2003,Orlando,Florida,Sep.7-11.

5. LimKP,WuS,RahardjaDJS,LinX,PanfandLiZG(2003)Fastintermodeselection.9

thJointVideoTeam

(JVT)Mtg,SanDiego,CA,USA.6. KimHandAltunbasakY(2004a)Fastmodedecision

forinterpredictioninH.264.In:ICIP2004,Singapore.7. Kim YH, Yoo JW, Lee SW and Shin J (2004b)

Adaptive mode decision for H.264 encoder. In:ElectronicLett.40(19).

8. Yu CA and Martin G (2004) Advanced block sizeselection algorithm for inter frame coding inH.264/MPEG-4AVC.In:ICIP2004,Singapore.

9. DaiQ,ZhuDandDingR(2004)FastmodedecisionforinterpredictioninH.264.In:ICIP2004,Singapore.

10.PanFandLinX(2003)Fastmodedecisionforintra-prediction, ISO/IEC JTC1/SC29/WG11 and ITU-TSG16Q.6.In:JVT7thMeeting,PattayaII,Thailand.

11.PanF,LinX,RahardjaS,LimKPandLiZG(2004)Adirectional field based fast intra-mode decisionalgorithmforH.264VideoCoding.IEEEIntl.Conf.on

Multimedia&Expo.2,1147-1150.12.KimJandJeongJ(2005)Fastintra-modedecisioninH.264 video coding usingsimple directionalmasks.In:Proc.ofSPIE,Vol.5960,pp:1071-1079.

13.GargR,Jindal M and ChauhanM (2005) Statisticsbased fast intra-mode detection. In: Proc. of SPIE,Vol.5960,pp:2085-2091.

14.Jeon B and lee J (2003) Fast mode decision forH.264, ISO/IECJTC1/SC29/WG11andITU-TSG16Q.6,JVT.In:10

thMeeting,Waikoloa,Hawaii.

15.YangC,PoLandLamW(2004)AfastH.264intra-predictionalgorithmusingmacroblockproperties. InICIP.pp:461-464.

16.MengB,AuOC,WongCandLamH(2003)Efficientintra-predictionalgorithminH.264.In:IEEE.pp:837-840.

17.TsengC,WangHandYangJ(2004)Improvedandfast algorithms for intra-4X4 mode decision in

H.264/AVC.In:IEEE.pp:2128-2131.18.Hsu C, Ho M and Hong J (2006) An efficien

algorithmforintra-predictioninH.264.IEEE.pp:3536.

19.Jafari M and Kasaei S (2008) Fast intra and inteprediction mode decision in H.264 advanced videocoding.IJCSNS.8(5),130-140.

20.Jafari M and Kasaei S (2006) An efficient intpredictionmodedecisionalgorithmforH.263toH.2Transcoding. 10

th IEEE Intl. Conf. on Compu

Systems&Appl.pp:1082-1089.

21.Jafari M and Kasaei S (2005) Prioritisation of dapartitioned MPEG-4 video overGPRS/EGPRSmobnetworks, Asian Internet Engg. conf. (AINTECThailand.pp:68-82.

22.Richardson(2005),H.264/MPEG-4part10whitepapavailable: http://www.vcodex.fsnet.co.uesources.html.

23.SullivanG (2001)Recommended simulation commconditions forH.26L coding efficiencyexperimentslow resolution progressive scan source materiPresented at the 14th VCEG-N81 Meeting, SanBarbara,CA.

24.Jafari M and Kasaei S (2010) Adaptive search randecision for fast intra- and inter-prediction modecision in h.264/AVC. In:The2nd National ElectriEngg.Conf.NEEC,247-252.

25.A.Elyousfi,A.Tamtaoui,andE.Bouyakh(2010)Anefast intra prediction mode decision algorithmH.264/AVC encoder. Intl. J. Electrical & ElectronEngg.4(1),89-95.

26.Bentao, Wu and Minyuan (2010) an improvemenintra prediction algorithm for H.264. J. Natural Sci2010.15(4),340-344.

Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264

Documents

Transcript of Adaptive Search Range Decision for Fast Intra and Inter Prediction Mode Decision in h264