Gradient Methods April 2004. Preview Background Steepest Descent Conjugate Gradient.
Research Article An Efficient Hybrid Conjugate Gradient...
Transcript of Research Article An Efficient Hybrid Conjugate Gradient...
Research ArticleAn Efficient Hybrid Conjugate Gradient Method withthe Strong Wolfe-Powell Line Search
Ahmad Alhawarat1 Mustafa Mamat2 Mohd Rivaie3 and Zabidin Salleh1
1School of Informatics and Applied Mathematics Universiti Malaysia Terengganu 21030 Kuala Terengganu Terengganu Malaysia2Faculty of Informatics and Computing Universiti Sultan Zainal Abidin 21300 Kuala Terengganu Terengganu Malaysia3Department of Computer Science and Mathematics Universiti Teknologi MARA (UITM) Terengganu Campus Kuala Terengganu21080 Kuala Terengganu Terengganu Malaysia
Correspondence should be addressed to Ahmad Alhawarat abadee2010yahoocom
Received 11 March 2015 Revised 6 July 2015 Accepted 7 July 2015
Academic Editor Haipeng Peng
Copyright copy 2015 Ahmad Alhawarat et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited
Conjugate gradient (CG) method is an interesting tool to solve optimization problems in many fields such as design economicsphysics and engineering In this paper we depict a new hybrid of CG method which relates to the famous Polak-Ribiere-Polyak(PRP) formula It reveals a solution for the PRP case which is not globally convergent with the strong Wolfe-Powell (SWP) linesearch The new formula possesses the sufficient descent condition and the global convergent properties In addition we furtherexplained about the cases where PRP method failed with SWP line search Furthermore we provide numerical computations forthe new hybrid CG method which is almost better than other related PRP formulas in both the number of iterations and the CPUtime under some standard test functions
1 Introduction
The nonlinear conjugate gradient (CG) method is a usefultool to find the minimum value for unconstrained optimiza-tion problems Consider the following form
min 119891 (119909) | 119909 isin 119877119899 (1)
where 119891 119877119899 rarr 119877 is continuously differentiable functionand its gradient is denoted by 119892(119909) = nabla119891(119909) The CGmethodis to find a sequence of points 119909
119896 119896 ge 1 starting from
initial point 1199091 isin 119877119899 which is given as the following iterative
formula
119909119896+1 = 119909119896 +120572119896119889119896 (2)
where 119909119896is the current iteration point and 120572
119896gt 0 is the step
length obtained by some line search The search direction 119889119896
is defined by
119889119896=
minus119892119896 119896 = 1
minus119892119896+ 120573119896119889119896minus1 119896 gt 1
(3)
where 119892119896= 119892(119909
119896) and 120573
119896is known as CG method formula
or coefficientTo find the step length (120572
119896) we can use the exact line
search which is given by
119891 (119909119896+120572119896119889119896) = min119891 (119909
119896+120572119889119896) 120572 ge 0 (4)
In fact (4) is not an effective line search since it needsheavy computations in function and gradient evaluationsTherefore we prefer to use inexpensive line searchThe strongWolfe-Powell (SWP) line search [1 2] which is given asfollows
119891 (119909119896+120572119896119889119896) le 119891 (119909
119896) + 120575120572
119896119892119879119896119889119896 (5)
100381610038161003816100381610038161003816119892 (119909119896+120572119896119889119896)119879
119889119896
100381610038161003816100381610038161003816le minus 120590
10038161003816100381610038161003816119892119879
119896119889119896
10038161003816100381610038161003816 (6)
where 0 lt 120575 lt 120590 lt 1 is to find an approximation of 120572119896where
the descent property (see (14))must be satisfied and no longersearching in the direction when 119909
119896is far from the solution
Thus by using SWP line search we inherit the advantagesof exact line search with inexpensive and low computational
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2015 Article ID 103517 7 pageshttpdxdoiorg1011552015103517
2 Mathematical Problems in Engineering
cost However different choices of 120572119896and 119889
119896imply different
CG methods In fact the SWP line search is a strong versionof weak Wolfe-Powell (WWP) line search where the latter isgiven by (5) and
119892 (119909119896+120572119896119889119896)119879
119889119896ge 120590119892119879119896119889119896 (7)
The CG method has been developed recently based on itssimplicity numerical efficiency and low memory require-ments Thus it is used widely in engineering medical scienceand other fields As an application in engineering we canuse CG method to solve some real life problem similar tothat mentioned in [3] The CG method is limited for thefunctions where their gradient is availableThus the heuristicalgorithm [4] can be used as an alternative method to findthe solution for general functions A heuristic algorithm isto find an approximation solution for the objective functionswith accepted time In addition the heuristic algorithms canbe applied without using computers We refer the reader tosee some applications for this algorithm in [5ndash7]
The most popular formulas for 120573119896are Hestenes-Stiefel
(HS) [8] Fletcher-Reeves (FR) [9] Polak-Ribiere-Polyak(PRP) [10] andWei et al (WYL) [11] respectively as follows
120573HS119896=119892119879119896(119910119896minus1)
119889119879119896minus1119910119896minus1
(8)
120573FR119896=100381710038171003817100381711989211989610038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172 (9)
120573PRP119896=119892119879119896119910119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 (10)
120573WYL119896
=119892119879119896(119892119896minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817) 119892119896minus1)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 (11)
where 119910119896minus1 = 119892119896 minus 119892119896minus1
Hestenes-Stiefel [8] proposed the first formula for solvingthe quadratic functions in 1952 Fletcher and Reeves [9]presented the first formula (9) for nonlinear functions in1964 The convergence properties of FR method with exactline search were obtained by Zoutendijk [12] Al-Baali [13]proved that FR method is globally convergent with the SWPline search when 120590 lt 12 Later on Guanghui et al [14]extended the result to120590 le 12The global convergence of PRPmethod (10) with the exact line search was proved by Elijahand Ribiere in [10] Powell [15] gave out a counterexampleshowing that there exists nonconvex function where PRPmethod does not converge globally even when the exactline search is used Powell suggested using nonnegative PRPmethod to reveal this problem Gilbert and Nocedal [16]proved that nonnegative PRP (PRP+) method that is 120573
119896=
max120573PRP119896 0 is globally convergent under complicated line
searches However there is no guarantee that PRP+ is conver-gent with SWP line search for general nonlinear functions
Touati-Ahmed and Storey [17] suggest the following hybridmethod
120573TS119896=
120573PRP119896 if 0 le 120573PRP
119896le 120573FR119896
120573FR119896 otherwise
(12)
In 2006 Wei et al [11] presented a new positive CG method(11) which is quite similar to original PRP method whichhas been studied in both exact and inexact line search Manymodifications have appeared such as the following [18ndash20]respectively
120573VHS119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817) 119892119896119892119896minus1119889119879119896minus1119910119896minus1
120573NPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
10038161003816100381610038161003817100381710038171003817119892119896minus1
10038171003817100381710038172
120573DPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
1003816100381610038161003816119898 1003816100381610038161003816119892119879
119896119889119896minus11003816100381610038161003816 +1003817100381710038171003817119892119896minus1
10038171003817100381710038172
where 119898 ge 1
(13)
Recently many CG formulas were constructed in order to getthe efficiency and robustness For more about the latest CGmethods we refer the reader to see [21 22]
One of the important rules in CG methods is the descentcondition that is if one can prove
119892119879119896119889119896lt 0 (14)
then we have a guarantee for119891(119909119896+1) lt 119891(119909119896) If we extended
(14) to the following form
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 119896 ge 0 119888 gt 0 (15)
then (15) is called the sufficient descent conditionThis paper is organized as follows in Section 2 we will
present the current problem for PRP and nonnegative PRPmethod with SWP line search Later on we will suggest thenew hybrid CG formula and its simplifications Furthermorewe will establish the global convergence properties withthe SWP line search in Section 3 Numerical results withconclusion will be presented in Sections 4 and 5 respectively
2 Motivation and the Hybrid Formula
The PRP formula is one of the best CG methods in thiscentury However as we mentioned before this methodfails to solve some standard test problems for nonconvexfunctions even the exact line search is used Thus the maincontribution of this paper is to extend using PRP formula inseveral cases with SWP line search under mild condition andrestart the CG algorithm by using NPRP CG formula whenPRP failed to satisfy that condition
The following discussion illustrates the cases in whichPRP method fails and succeeds with SWP line search to
Mathematical Problems in Engineering 3
obtain the convergence propertiesThe PRPmethod could besimplified as follows
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 =
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 (16)
Therefore we have the following cases
Case A If 119892119879119896119892119896minus1 gt 0 then we have the following two
possibilities
Case A1 If 1198921198962 gt 119892119879
119896119892119896minus1 then we have 0 le 120573PRP
119896le 120573FR119896 In
this case PRPmethod is efficient and has global convergenceproperties
Case A2 If 1198921198962 lt 119892119879
119896119892119896minus1 then we have 120573PRP
119896lt 0 In this
case based on [16] we fail to obtain the global convergenceproperties for nonconvex functions
Case B If 119892119879119896119892119896minus1 lt 0 then
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 =
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
= 120573FR119896+
10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
(17)
In Case B there is no guarantee that this method will satisfythe sufficient descent condition
For the next discussion we will discuss the nonnegativePRP method which is given as follows
120573PRP+119896
= max 120573PRP119896 0 ie
120573PRP+119896
= max Case A1 Case A2 Case B 0 (18)
Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+
119896= 0 the
CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that
120573TS119896=
Case A1
120573FR119896 otherwise
(19)
So from PRP method we can use only Case A1To improve the above ideas we suggest the following
hybrid method
120573PRPlowastlowast
119896=
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 if 1003817100381710038171003817119892119896
10038171003817100381710038172gt10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
120573NPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
10038161003816100381610038161003817100381710038171003817119892119896minus1
10038171003817100381710038172 otherwise
(20)
If 120573PRPlowastlowast
119896= (119892
1198962 minus 119892119879
119896119892119896minus1)119892119896minus1
2 then under thecondition 119892
1198962 gt |119892119879
119896119892119896minus1| we obtain
0 lt 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
lt2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(21)
And if
120573PRPlowastlowast
119896= 120573NPRP119896
=
100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172 (22)
we obtain
0 le 120573PRPlowastlowast
119896lt100381710038171003817100381711989211989610038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172 = 120573
FR119896 (23)
One of the advantages for 120573PRPlowastlowast
119896is that we can use Case A1
and Case B under the condition 1198921198962 gt |119892119879
119896119892119896minus1|
Choosing 120573NPRP119896
as a restart CG formula in (20) if thecondition 119892
1198962 gt |119892119879
119896119892119896minus1| is not satisfied that means
100381710038171003817100381711989211989610038171003817100381710038172le10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816 (24)
Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method
with the new coefficient 120573PRPlowastlowast
119896
Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573
119896based on (20)
Step 3 Compute 119889119896based on (3)
Step 4 Compute 120572119896based on (5) and (6)
Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892
119896 le 120576 then
stop otherwise go to Step 2 with 119896 = 119896 + 1
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
sum119896=1
cos2120579119896
100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)
where
cos 120579119896= minus
(119892119879119896119889119896)
1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)
Equation (25) is known as Zoutendijk condition
The following discussion will discuss the global conver-gence properties for 120573PRP
lowastlowast
119896with SWP line search
Case 1 (120573PRPlowastlowast
119896= 120573NPRP119896
) Since 0 le 120573NPRP119896
le 120573FR119896
the proof ofthe global convergence properties is similar to 120573FR
119896 We refer
the reader to see [13 14]
Case 2 (120573PRPlowastlowast
119896= 120573PRP119896
where 1198921198962 gt |119892119879
119896119892119896minus1|) In this case
we have
0 le 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(27)
The following theoremdemonstrates120573PRPlowastlowast
119896in Case 2 satisfies
the sufficient descent condition with SWP line search
Theorem 3 Consider the sequences 119892119896and 119889
119896are constructed
by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin
(0 14] Then (15) holds
Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879
119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892
119896
we obtain
119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus
100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)
Dividing both sides by 119892119896minus1
2 and by using Case 2 and (6)then we obtain
minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590
119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 (29)
From (29) we obtain
minus119896minus1sum119895=0(2120590)119895 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
(IV) 119875119903119900119901119890119903119905119910lowast holds
(V) Assumptions 1 and 2 hold
Then the iterates are globally convergent
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
21205742
1205742= 119887 gt 1
(38)
Using 119904119896 le 120582 we have
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
10038171003817100381710038171003817100381710038171003817119892119896minus1
10038171003817100381710038172 (39)
By Assumption 2
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le119871120582120574
1205742le
12119887 (40)
The proof is complete
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Number Function Dimensions Initial points
1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)
5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)
9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)
11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)
12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)
20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)
24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
120588119904(119905) =
1119899119901
size 119901 isin119875 119903119901119904le 119905 (42)
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
2 Mathematical Problems in Engineering
cost However different choices of 120572119896and 119889
119896imply different
CG methods In fact the SWP line search is a strong versionof weak Wolfe-Powell (WWP) line search where the latter isgiven by (5) and
119892 (119909119896+120572119896119889119896)119879
119889119896ge 120590119892119879119896119889119896 (7)
The CG method has been developed recently based on itssimplicity numerical efficiency and low memory require-ments Thus it is used widely in engineering medical scienceand other fields As an application in engineering we canuse CG method to solve some real life problem similar tothat mentioned in [3] The CG method is limited for thefunctions where their gradient is availableThus the heuristicalgorithm [4] can be used as an alternative method to findthe solution for general functions A heuristic algorithm isto find an approximation solution for the objective functionswith accepted time In addition the heuristic algorithms canbe applied without using computers We refer the reader tosee some applications for this algorithm in [5ndash7]
The most popular formulas for 120573119896are Hestenes-Stiefel
(HS) [8] Fletcher-Reeves (FR) [9] Polak-Ribiere-Polyak(PRP) [10] andWei et al (WYL) [11] respectively as follows
120573HS119896=119892119879119896(119910119896minus1)
119889119879119896minus1119910119896minus1
(8)
120573FR119896=100381710038171003817100381711989211989610038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172 (9)
120573PRP119896=119892119879119896119910119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 (10)
120573WYL119896
=119892119879119896(119892119896minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817) 119892119896minus1)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 (11)
where 119910119896minus1 = 119892119896 minus 119892119896minus1
Hestenes-Stiefel [8] proposed the first formula for solvingthe quadratic functions in 1952 Fletcher and Reeves [9]presented the first formula (9) for nonlinear functions in1964 The convergence properties of FR method with exactline search were obtained by Zoutendijk [12] Al-Baali [13]proved that FR method is globally convergent with the SWPline search when 120590 lt 12 Later on Guanghui et al [14]extended the result to120590 le 12The global convergence of PRPmethod (10) with the exact line search was proved by Elijahand Ribiere in [10] Powell [15] gave out a counterexampleshowing that there exists nonconvex function where PRPmethod does not converge globally even when the exactline search is used Powell suggested using nonnegative PRPmethod to reveal this problem Gilbert and Nocedal [16]proved that nonnegative PRP (PRP+) method that is 120573
119896=
max120573PRP119896 0 is globally convergent under complicated line
searches However there is no guarantee that PRP+ is conver-gent with SWP line search for general nonlinear functions
Touati-Ahmed and Storey [17] suggest the following hybridmethod
120573TS119896=
120573PRP119896 if 0 le 120573PRP
119896le 120573FR119896
120573FR119896 otherwise
(12)
In 2006 Wei et al [11] presented a new positive CG method(11) which is quite similar to original PRP method whichhas been studied in both exact and inexact line search Manymodifications have appeared such as the following [18ndash20]respectively
120573VHS119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817) 119892119896119892119896minus1119889119879119896minus1119910119896minus1
120573NPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
10038161003816100381610038161003817100381710038171003817119892119896minus1
10038171003817100381710038172
120573DPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
1003816100381610038161003816119898 1003816100381610038161003816119892119879
119896119889119896minus11003816100381610038161003816 +1003817100381710038171003817119892119896minus1
10038171003817100381710038172
where 119898 ge 1
(13)
Recently many CG formulas were constructed in order to getthe efficiency and robustness For more about the latest CGmethods we refer the reader to see [21 22]
One of the important rules in CG methods is the descentcondition that is if one can prove
119892119879119896119889119896lt 0 (14)
then we have a guarantee for119891(119909119896+1) lt 119891(119909119896) If we extended
(14) to the following form
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 119896 ge 0 119888 gt 0 (15)
then (15) is called the sufficient descent conditionThis paper is organized as follows in Section 2 we will
present the current problem for PRP and nonnegative PRPmethod with SWP line search Later on we will suggest thenew hybrid CG formula and its simplifications Furthermorewe will establish the global convergence properties withthe SWP line search in Section 3 Numerical results withconclusion will be presented in Sections 4 and 5 respectively
2 Motivation and the Hybrid Formula
The PRP formula is one of the best CG methods in thiscentury However as we mentioned before this methodfails to solve some standard test problems for nonconvexfunctions even the exact line search is used Thus the maincontribution of this paper is to extend using PRP formula inseveral cases with SWP line search under mild condition andrestart the CG algorithm by using NPRP CG formula whenPRP failed to satisfy that condition
The following discussion illustrates the cases in whichPRP method fails and succeeds with SWP line search to
Mathematical Problems in Engineering 3
obtain the convergence propertiesThe PRPmethod could besimplified as follows
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 =
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 (16)
Therefore we have the following cases
Case A If 119892119879119896119892119896minus1 gt 0 then we have the following two
possibilities
Case A1 If 1198921198962 gt 119892119879
119896119892119896minus1 then we have 0 le 120573PRP
119896le 120573FR119896 In
this case PRPmethod is efficient and has global convergenceproperties
Case A2 If 1198921198962 lt 119892119879
119896119892119896minus1 then we have 120573PRP
119896lt 0 In this
case based on [16] we fail to obtain the global convergenceproperties for nonconvex functions
Case B If 119892119879119896119892119896minus1 lt 0 then
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 =
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
= 120573FR119896+
10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
(17)
In Case B there is no guarantee that this method will satisfythe sufficient descent condition
For the next discussion we will discuss the nonnegativePRP method which is given as follows
120573PRP+119896
= max 120573PRP119896 0 ie
120573PRP+119896
= max Case A1 Case A2 Case B 0 (18)
Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+
119896= 0 the
CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that
120573TS119896=
Case A1
120573FR119896 otherwise
(19)
So from PRP method we can use only Case A1To improve the above ideas we suggest the following
hybrid method
120573PRPlowastlowast
119896=
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 if 1003817100381710038171003817119892119896
10038171003817100381710038172gt10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
120573NPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
10038161003816100381610038161003817100381710038171003817119892119896minus1
10038171003817100381710038172 otherwise
(20)
If 120573PRPlowastlowast
119896= (119892
1198962 minus 119892119879
119896119892119896minus1)119892119896minus1
2 then under thecondition 119892
1198962 gt |119892119879
119896119892119896minus1| we obtain
0 lt 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
lt2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(21)
And if
120573PRPlowastlowast
119896= 120573NPRP119896
=
100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172 (22)
we obtain
0 le 120573PRPlowastlowast
119896lt100381710038171003817100381711989211989610038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172 = 120573
FR119896 (23)
One of the advantages for 120573PRPlowastlowast
119896is that we can use Case A1
and Case B under the condition 1198921198962 gt |119892119879
119896119892119896minus1|
Choosing 120573NPRP119896
as a restart CG formula in (20) if thecondition 119892
1198962 gt |119892119879
119896119892119896minus1| is not satisfied that means
100381710038171003817100381711989211989610038171003817100381710038172le10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816 (24)
Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method
with the new coefficient 120573PRPlowastlowast
119896
Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573
119896based on (20)
Step 3 Compute 119889119896based on (3)
Step 4 Compute 120572119896based on (5) and (6)
Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892
119896 le 120576 then
stop otherwise go to Step 2 with 119896 = 119896 + 1
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
sum119896=1
cos2120579119896
100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)
where
cos 120579119896= minus
(119892119879119896119889119896)
1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)
Equation (25) is known as Zoutendijk condition
The following discussion will discuss the global conver-gence properties for 120573PRP
lowastlowast
119896with SWP line search
Case 1 (120573PRPlowastlowast
119896= 120573NPRP119896
) Since 0 le 120573NPRP119896
le 120573FR119896
the proof ofthe global convergence properties is similar to 120573FR
119896 We refer
the reader to see [13 14]
Case 2 (120573PRPlowastlowast
119896= 120573PRP119896
where 1198921198962 gt |119892119879
119896119892119896minus1|) In this case
we have
0 le 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(27)
The following theoremdemonstrates120573PRPlowastlowast
119896in Case 2 satisfies
the sufficient descent condition with SWP line search
Theorem 3 Consider the sequences 119892119896and 119889
119896are constructed
by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin
(0 14] Then (15) holds
Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879
119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892
119896
we obtain
119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus
100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)
Dividing both sides by 119892119896minus1
2 and by using Case 2 and (6)then we obtain
minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590
119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 (29)
From (29) we obtain
minus119896minus1sum119895=0(2120590)119895 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
(IV) 119875119903119900119901119890119903119905119910lowast holds
(V) Assumptions 1 and 2 hold
Then the iterates are globally convergent
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
21205742
1205742= 119887 gt 1
(38)
Using 119904119896 le 120582 we have
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
10038171003817100381710038171003817100381710038171003817119892119896minus1
10038171003817100381710038172 (39)
By Assumption 2
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le119871120582120574
1205742le
12119887 (40)
The proof is complete
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Number Function Dimensions Initial points
1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)
5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)
9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)
11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)
12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)
20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)
24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
120588119904(119905) =
1119899119901
size 119901 isin119875 119903119901119904le 119905 (42)
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 3
obtain the convergence propertiesThe PRPmethod could besimplified as follows
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 =
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 (16)
Therefore we have the following cases
Case A If 119892119879119896119892119896minus1 gt 0 then we have the following two
possibilities
Case A1 If 1198921198962 gt 119892119879
119896119892119896minus1 then we have 0 le 120573PRP
119896le 120573FR119896 In
this case PRPmethod is efficient and has global convergenceproperties
Case A2 If 1198921198962 lt 119892119879
119896119892119896minus1 then we have 120573PRP
119896lt 0 In this
case based on [16] we fail to obtain the global convergenceproperties for nonconvex functions
Case B If 119892119879119896119892119896minus1 lt 0 then
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 =
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
= 120573FR119896+
10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
(17)
In Case B there is no guarantee that this method will satisfythe sufficient descent condition
For the next discussion we will discuss the nonnegativePRP method which is given as follows
120573PRP+119896
= max 120573PRP119896 0 ie
120573PRP+119896
= max Case A1 Case A2 Case B 0 (18)
Therefore we have a problem inCase B To solve this problemGilbert and Nocedal [16] used another line search to satisfythe convergence properties In addition if 120573PRP+
119896= 0 the
CG method returns to the steepest descent method whichis sometimes a weak tool to find the optimum point forfunctions Furthermore we can notice that
120573TS119896=
Case A1
120573FR119896 otherwise
(19)
So from PRP method we can use only Case A1To improve the above ideas we suggest the following
hybrid method
120573PRPlowastlowast
119896=
120573PRP119896=119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172 if 1003817100381710038171003817119892119896
10038171003817100381710038172gt10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
120573NPRP119896
=100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003816100381610038161003816119892119896119892119896minus1
10038161003816100381610038161003817100381710038171003817119892119896minus1
10038171003817100381710038172 otherwise
(20)
If 120573PRPlowastlowast
119896= (119892
1198962 minus 119892119879
119896119892119896minus1)119892119896minus1
2 then under thecondition 119892
1198962 gt |119892119879
119896119892119896minus1| we obtain
0 lt 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
lt2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(21)
And if
120573PRPlowastlowast
119896= 120573NPRP119896
=
100381710038171003817100381711989211989610038171003817100381710038172minus (1003817100381710038171003817119892119896
1003817100381710038171003817 1003817100381710038171003817119892119896minus1
1003817100381710038171003817)10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172 (22)
we obtain
0 le 120573PRPlowastlowast
119896lt100381710038171003817100381711989211989610038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172 = 120573
FR119896 (23)
One of the advantages for 120573PRPlowastlowast
119896is that we can use Case A1
and Case B under the condition 1198921198962 gt |119892119879
119896119892119896minus1|
Choosing 120573NPRP119896
as a restart CG formula in (20) if thecondition 119892
1198962 gt |119892119879
119896119892119896minus1| is not satisfied that means
100381710038171003817100381711989211989610038171003817100381710038172le10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816 (24)
Thus NPRP method is a suitable nonnegative value to useThe following algorithm is an algorithm of CG method
with the new coefficient 120573PRPlowastlowast
119896
Algorithm 1 Consider the followingStep 1 Initialization given 1199091 set 119896 = 1Step 2 Compute 120573
119896based on (20)
Step 3 Compute 119889119896based on (3)
Step 4 Compute 120572119896based on (5) and (6)
Step 5 Update new point based on (2)Step 6 Convergent test and stopping criteria if 119892
119896 le 120576 then
stop otherwise go to Step 2 with 119896 = 119896 + 1
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
sum119896=1
cos2120579119896
100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)
where
cos 120579119896= minus
(119892119879119896119889119896)
1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)
Equation (25) is known as Zoutendijk condition
The following discussion will discuss the global conver-gence properties for 120573PRP
lowastlowast
119896with SWP line search
Case 1 (120573PRPlowastlowast
119896= 120573NPRP119896
) Since 0 le 120573NPRP119896
le 120573FR119896
the proof ofthe global convergence properties is similar to 120573FR
119896 We refer
the reader to see [13 14]
Case 2 (120573PRPlowastlowast
119896= 120573PRP119896
where 1198921198962 gt |119892119879
119896119892119896minus1|) In this case
we have
0 le 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(27)
The following theoremdemonstrates120573PRPlowastlowast
119896in Case 2 satisfies
the sufficient descent condition with SWP line search
Theorem 3 Consider the sequences 119892119896and 119889
119896are constructed
by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin
(0 14] Then (15) holds
Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879
119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892
119896
we obtain
119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus
100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)
Dividing both sides by 119892119896minus1
2 and by using Case 2 and (6)then we obtain
minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590
119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 (29)
From (29) we obtain
minus119896minus1sum119895=0(2120590)119895 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
(IV) 119875119903119900119901119890119903119905119910lowast holds
(V) Assumptions 1 and 2 hold
Then the iterates are globally convergent
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
21205742
1205742= 119887 gt 1
(38)
Using 119904119896 le 120582 we have
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
10038171003817100381710038171003817100381710038171003817119892119896minus1
10038171003817100381710038172 (39)
By Assumption 2
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le119871120582120574
1205742le
12119887 (40)
The proof is complete
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Number Function Dimensions Initial points
1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)
5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)
9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)
11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)
12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)
20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)
24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
120588119904(119905) =
1119899119901
size 119901 isin119875 119903119901119904le 119905 (42)
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
4 Mathematical Problems in Engineering
3 The Global Convergence Properties for120573PRP
lowastlowast
119896with SWP Line Search
The following standard assumptions are necessary for thiswork
Assumption 1 The level set Χ = 119909 isin 119877119899 119891(119909) le 119891(1199091)with 1199091 to the starting point of the iterative method (2) isbounded
Assumption 2 In some open convex neighborhood119873 ofΧ119891is continuous and differentiable and its gradient is Lipschitzcontinuous that is for any 119909 119910 isin 119873 there exists a constant119871 gt 0 such that 119892(119909) minus 119892(119910) le 119871119909 minus 119910
The following lemma is one of the most important lem-maswhich is used to prove the global convergence properties
Lemma 2 (see [12]) Suppose Assumptions 1 and 2 are trueConsider any form of (2) and (3) with 120572
119896computed by WWP
line search direction 119889119896 is descent for all 119896 ge 1 theninfin
sum119896=1
cos2120579119896
100381710038171003817100381711989211989610038171003817100381710038172lt infin (25)
where
cos 120579119896= minus
(119892119879119896119889119896)
1003817100381710038171003817119889119896100381710038171003817100381710038171003817100381710038171198921198961003817100381710038171003817 (26)
Equation (25) is known as Zoutendijk condition
The following discussion will discuss the global conver-gence properties for 120573PRP
lowastlowast
119896with SWP line search
Case 1 (120573PRPlowastlowast
119896= 120573NPRP119896
) Since 0 le 120573NPRP119896
le 120573FR119896
the proof ofthe global convergence properties is similar to 120573FR
119896 We refer
the reader to see [13 14]
Case 2 (120573PRPlowastlowast
119896= 120573PRP119896
where 1198921198962 gt |119892119879
119896119892119896minus1|) In this case
we have
0 le 120573PRPlowastlowast
119896=100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172 le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le2 1003817100381710038171003817119892119896
10038171003817100381710038172
1003817100381710038171003817119892119896minus110038171003817100381710038172
(27)
The following theoremdemonstrates120573PRPlowastlowast
119896in Case 2 satisfies
the sufficient descent condition with SWP line search
Theorem 3 Consider the sequences 119892119896and 119889
119896are constructed
by Algorithm 1 and 120572119896is computed by (5) and (6) if 120590 isin
(0 14] Then (15) holds
Proof From (3) for 119896 = 1 it is obvious Suppose it is trueuntil 119896 minus 1 that is 119892119879
119896minus1119889119896minus1 = minus119892119896minus12 Multiply (3) by 119892
119896
we obtain
119892119879119896119889119896= 119892119879119896(minus119892119879119896+120573119896119889119896minus1) = minus
100381710038171003817100381711989211989610038171003817100381710038172+120573119896119892119879119896119889119896minus1 (28)
Dividing both sides by 119892119896minus1
2 and by using Case 2 and (6)then we obtain
minus 1+ 2120590119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 1minus 2120590
119892119879119896minus1119889119896minus11003817100381710038171003817119892119896minus1
10038171003817100381710038172 (29)
From (29) we obtain
minus119896minus1sum119895=0(2120590)119895 le
119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
119896minus1sum119895=0(2120590)119895 (30)
Since119896minus1sum119895=0(2120590)119895 le 1 minus (2120590)119896
1 minus 2120590 (31)
we have
minus1 minus (2120590)119896
1 minus 2120590le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 2+
1 minus (2120590)119896
1 minus 2120590 (32)
If 120590 le 14 we get
1 minus (2120590)119896
1 minus 2120590lt 2 (33)
Let
119888 = 2minus 1 minus (120590)119896
1 minus 120590 (34)
then
119888 minus 2 le119892119879119896119889119896
100381710038171003817100381711989211989610038171003817100381710038172 le minus 119888 (35)
Thus
119892119879119896119889119896le minus 119888 1003817100381710038171003817119892119896
10038171003817100381710038172 (36)
where 119888 isin (0 1) The proof is complete
Gilbert andNocedal [16] presented an important theorem(Theorem 4) to find the global convergence properties fornonnegative PRPmethod if the descent condition is satisfiedFurthermore [16] presented nice property called Propertylowast asfollows
119875119903119900119901119890119903119905119910lowast Consider a CG method of forms (2) and (3) andsuppose 0 lt 120574 le 119892
119896 le 120574 we say that the CG method
possesses Propertylowast if there exists constant 119887 gt 1 and 120582 gt 0such that for all 119896 ge 1 we get |120573
119896| le 119887 and if 119909
119896minus119909119896minus1 le 120582
we obtain
10038161003816100381610038161205731198961003816100381610038161003816 le
12119887 (37)
Theorem 4 Consider any CG method of forms (2) and (3)achieves the following properties
(I) 120573119896ge 0
(II) The sufficient descent condition (15) holds
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
(IV) 119875119903119900119901119890119903119905119910lowast holds
(V) Assumptions 1 and 2 hold
Then the iterates are globally convergent
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
21205742
1205742= 119887 gt 1
(38)
Using 119904119896 le 120582 we have
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
10038171003817100381710038171003817100381710038171003817119892119896minus1
10038171003817100381710038172 (39)
By Assumption 2
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le119871120582120574
1205742le
12119887 (40)
The proof is complete
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Number Function Dimensions Initial points
1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)
5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)
9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)
11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)
12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)
20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)
24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
120588119904(119905) =
1119899119901
size 119901 isin119875 119903119901119904le 119905 (42)
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 5
(III) The Zoutendijk condition (25) is satisfied by the linesearch
(IV) 119875119903119900119901119890119903119905119910lowast holds
(V) Assumptions 1 and 2 hold
Then the iterates are globally convergent
The next lemma shows that if the gradients are boundedaway from zero and Propertylowast holds then a certain fractionof steps cannot be too small The proof is given in [16]However we state it for readability
Lemma 5 Consider a CG algorithm as defined in (2) and (3)with the parameter 120573PRP
lowastlowast
119896 If Assumptions 1 and 2 are satisfied
then 119875119903119900119901119890119903119905119910lowast holds
Proof Let 119887 = 212057421205742 ge 1 and 120582 le 12057422119871120574119887
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
100381710038171003817100381711989211989610038171003817100381710038172minus 119892119879119896119892119896minus1
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le
100381710038171003817100381711989211989610038171003817100381710038172+10038161003816100381610038161003816119892119879
119896119892119896minus110038161003816100381610038161003816
1003817100381710038171003817119892119896minus110038171003817100381710038172
le10038171003817100381710038171198921198961003817100381710038171003817 (10038171003817100381710038171198921198961003817100381710038171003817 +1003817100381710038171003817119892119896minus1
1003817100381710038171003817)1003817100381710038171003817119892119896minus1
10038171003817100381710038172 le
21205742
1205742= 119887 gt 1
(38)
Using 119904119896 le 120582 we have
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le
1003816100381610038161003816100381610038161003816100381610038161003816
119892119879119896(119892119896minus 119892119896minus1)
1003817100381710038171003817119892119896minus110038171003817100381710038172
1003816100381610038161003816100381610038161003816100381610038161003816le100381710038171003817100381711989211989610038171003817100381710038171003817100381710038171003817119892119896 minus 119892119896minus1
10038171003817100381710038171003817100381710038171003817119892119896minus1
10038171003817100381710038172 (39)
By Assumption 2
100381610038161003816100381610038161003816120573PRP
lowastlowast
119896
100381610038161003816100381610038161003816le119871120582120574
1205742le
12119887 (40)
The proof is complete
Lemma 6 The CG formula presented in Case 2 has thefollowing properties
(1) 120573PRPlowastlowast
119896gt 0 since the condition 119892
1198962 gt |119892119879
119896119892119896minus1| forces
the CG formula in (20) to be nonnegative
(2) 120573PRPlowastlowast
119896satisfies 119875119903119900119901119890119903119905119910lowast based on Lemma 5
(3) 120573PRPlowastlowast
119896satisfies the sufficient descent condition based on
Theorem 3 and 119888 isin (0 1)
By using Theorems 3 and 4 and Lemma 6 we havethe following convergence result The proof is similar toTheorem 43 which is presented in [16]
Theorem 7 Suppose that Assumption 1 holds Consider theCG method of forms (2) and (3) and 120573
119896as in Case 2
where 120572119896is computed by (5) and (6) with 120590 le 14 then
lim119896rarrinfin
inf 119892119896 = 0
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRPWYLVHS
VHSNPRPWYL
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 1 Performance profile based on the number of iterations
4 Numerical Results and Discussions
To evaluate the efficiency of the new method we selectedsome of the test functions in Table 1 from CUTEr [24]Neculai [23] and Adorio and Diliman [25] We performed acomparison with other CG methods including VHS NPRPPRP+ FR and PRPlowastlowast formulas The tolerance 120576 is selectedto 10minus6 for all algorithms to investigate the rapidity of theiterationmethods towards the optimal solutionThe gradientvalue is used as the stopping criteria Here the stoppingcriteria considered 119892
119896 le 10minus6 We considered the method
failed if the number of iterations exceeds 1000 timesIn Table 1 we selected different initial points for every
function Thus this demonstrated that this method can beused in several real life functions from other fields such asengineering and medical science as we mentioned before Inaddition different dimensions from 500 until 10000 are usedWe also choose from different group of functions We usedMatlab 79 subroutine program with CPU processor Intel (R)Core (TM) i3 CPU and 2GB DDR2 RAM under SWP linesearch to find the optimumpointThe efficiency comparisonsresults are shown in Figures 1 and 2 respectively using aperformance profile introduced by Dolan and More [26]
This performance measure was introduced to compare aset of solvers 119878 on a set of problems 119875 Assuming 119899
119904solvers
and 119899119901problems in 119878 and 119875 respectively the measure 119905
119901119904
is defined as the computation time (eg the number ofiterations or the CPU time) required to solve problem 119901 bysolver 119904
To create a baseline for comparison the performance ofsolver 119904 on problem119901 is scaled by the best performance of anysolver in 119878 on the problem using the ratio
119903119901119904=
119905119901119904
min 119905119901119904 119904 isin 119878
(41)
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Number Function Dimensions Initial points
1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)
5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)
9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)
11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)
12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)
20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)
24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
120588119904(119905) =
1119899119901
size 119901 isin119875 119903119901119904le 119905 (42)
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
6 Mathematical Problems in Engineering
Table 1 A list of problem functions used with the SWP condition with 120575 = 001 and 120590 = 01
Number Function Dimensions Initial points
1 Extended White amp Holstfunction [23] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
2 Extended Rosenbrockfunction [24] 500 1000 5000 10000 (minus12 1 minus12 1 minus12 1) (5 5 5) (10 10 10) (15 15 15)
3 Six-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
4 Extended Beale function[23] 500 1000 5000 10000 (minus1 minus1 minus1) (05 05 05) (10 10 10) (1 1 1)
5 Three-hump function [25] 2 (1 1) (5 5) (10 10) (15 15)
6 Extended Himmelblaufunction [23] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
7 Diagonal 2 function [23] 500 1000 5000 10000 (02 02 02) (025 025 025) (05 05 05) (1 1 1)8 NONSCOMP function [24] 500 1000 5000 10000 (1 1 1) (minus1 minus1 minus1) (minus2 minus2 minus2) (minus5 minus5 minus5)
9 Extended DENSCHNBfunction [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
10 Booth function [25] 2 (1 1) (5 5) (10 10) (15 15)
11 Extended quadratic penaltyfunction QP2 [24] 100 200 300 400 (2 2 2) (5 5 5) (10 10 10) (15 15 15)
12 DIXMAANA function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)13 DIXMAANB function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)14 DIXMAANC function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)15 DIXMAAND function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)16 DIXMAANE function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)17 DIXMAANF function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)18 DIXMAANG function [24] 1500 3000 6000 9000 (1 1 1) (2 2 2) (3 3 3) (5 5 5)19 NONDIA function [24] 500 1000 5000 10000 (minus2 minus2 minus2) (minus1 minus1 minus1) (0 0 0) (1 1 1)
20 Extended Tridiagonal 1function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
21 DQDRTIC function [24] 500 1000 5000 10000 (minus1 minus1 minus1) (1 1 1) (2 2 2) (3 3 3)22 Diagonal 4 function [24] 500 1000 5000 10000 (1 1 1) (5 5 5) (10 10 10) (15 15 15)23 DQDRTIC function [24] 10 20 30 40 (1 1 1) (minus1 minus1 minus1) (1 1 1) (2 2 2)
24 Extended Cliff function[24] 100 200 300 400 (1 1 1) (5 5 5) (10 10 10) (15 15 15)
Suppose that a parameter 119903119872ge 119903119901119904
for all 119901 119904 is chosen and119903119901119904= 119903119872
if and only if solver 119904 does not solve problem 119901Because we would like to obtain an overall assessment of theperformance of a solver we defined the measure
120588119904(119905) =
1119899119901
size 119901 isin119875 119903119901119904le 119905 (42)
Thus 120588119904(119905) is the probability for solver 119904 isin 119878 that the
performance ratio 119903119901119904
is within a factor 119905 isin 119877 of the bestpossible ratio If we define the function 119901
119904as the cumulative
distribution function for the performance ratio then theperformance measure 119901
119904 119877 rarr [0 1] for a solver is
nondecreasing and piecewise continuous from the right Thevalue of 119901
119904(1) is the probability that the solver has the best
performance of all of the solvers In general a solverwith highvalues of 119901(119905) which would appear in the upper right cornerof the figure is preferable
Based on the left side of Figures 1 and 2 the PRPlowastlowastformula is above the other curves Therefore it is the most
efficient method among related PRP methods in terms ofefficiency and robustness In Figure 2 we see that the curveof PRPlowastlowast is still the best but the efficiency is not goodas the number of iterations since we use the complicatedhybrid algorithm leads to high CPU time Thus using highprocessors computers to find the solution will be moreefficient since the number of iterations decreased rapidlyunder PRPlowastlowast method
5 Conclusion
In this paper we proposed hybrid conjugate gradient methodby using nonnegative PRP and NPRP formulas with the SWPline search which extended the cases of using PRP methodunder mild condition The global convergence property isestablished and it is very simple Our numerical results hadshown that the hybrid method is the best when compared toother related PRP CG methods
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 7
t
e0
e1
e2
e3
Ps(t)
00
02
04
06
08
10
FR
FR
NPRP
VHSNPRPWYL
WYLVHS
PRPlowastlowast
PRPlowastlowast
PRP+
PRP+
Figure 2 Performance profile based on the CPU time
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful to the editor and the anony-mous reviewers for their comments and suggestions whichimproved this paper substantially They would also like tothank The Ministry of Education Malaysia (MOE) for fund-ing this research under The Fundamental Research GrantScheme (Grant no 59256)
References
[1] P Wolfe ldquoConvergence conditions for ascent methodsrdquo SIAMReview vol 11 no 2 pp 226ndash235 1969
[2] PWolfe ldquoConvergence conditions for ascent methods II somecorrectionsrdquo SIAM Review vol 13 no 2 pp 185ndash188 1971
[3] J CaiQ Li L LiH Peng andYYang ldquoA fuzzy adaptive chaoticant swarm optimization for economic dispatchrdquo InternationalJournal of Electrical Power and Energy Systems vol 34 no 1 pp154ndash160 2012
[4] KNatalliaAn Introduction toHeuristic Slgorithms Departmentof Informatics and Telecommunications 2005
[5] L LiH Peng J Kurths Y Yang andH J Schellnhuber ldquoChaos-order transition in foraging behavior of antsrdquo Proceedings of theNational Academy of Sciences of the United States of Americavol 111 no 23 pp 8392ndash8397 2014
[6] M Wan L Li J Xiao C Wang and Y Yang ldquoData clusteringusing bacterial foraging optimizationrdquo Journal of IntelligentInformation Systems vol 38 no 2 pp 321ndash341 2012
[7] M Wan C Wang L Li and Y Yang ldquoChaotic ant swarmapproach for data clusteringrdquo Applied Soft Computing Journalvol 12 no 8 pp 2387ndash2393 2012
[8] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 no 6 pp 409ndash436 1952
[9] R Fletcher and C M Reeves ldquoFunction minimization byconjugate gradientsrdquo The Computer Journal vol 7 no 2 pp149ndash154 1964
[10] P Elijah and G Ribiere ldquoNote sur la convergence de methodesde directions conjugueesrdquo Revue francaise drsquoinformatique et derecherche operationnelle vol 3 no 1 pp 35ndash43 1969
[11] Z Wei S Yao and L Liu ldquoThe convergence properties of somenew conjugate gradient methodsrdquo Applied Mathematics andComputation vol 183 no 2 pp 1341ndash1350 2006
[12] G Zoutendijk ldquoNonlinear programming computationalmeth-odsrdquo Integer and Nonlinear Programming vol 143 no 1 pp 37ndash86 1970
[13] M Al-Baali ldquoDescent property and global convergence ofthe FletchermdashReeves method with inexact line searchrdquo IMAJournal of Numerical Analysis vol 5 no 1 pp 121ndash124 1985
[14] L Guanghui H Jiye and Y Hongxia ldquoGlobal convergence ofthe fletcher-reeves algorithm with inexact linesearchrdquo AppliedMathematics-A Journal of Chinese Universities vol 10 no 1 pp75ndash82 1995
[15] M J D Powell ldquoNonconvex minimization calculations and theconjugate gradient methodrdquo in Numerical Analysis vol 1066of Lecture Notes in Mathematics pp 122ndash141 Springer BerlinGermany 1984
[16] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992
[17] D Touati-Ahmed and C Storey ldquoEfficient hybrid conjugategradient techniquesrdquo Journal of OptimizationTheory and Appli-cations vol 64 no 2 pp 379ndash397 1990
[18] Y Shengwei ZWei andH Huang ldquoA note aboutWYLrsquos conju-gate gradientmethod and its applicationsrdquoAppliedMathematicsand Computation vol 191 no 2 pp 381ndash388 2007
[19] L Zhang ldquoAn improved Wei-Yao-Liu nonlinear conjugategradient method for optimization computationrdquoApplied Math-ematics and Computation vol 215 no 6 pp 2269ndash2274 2009
[20] Z Dai and F Wen ldquoAnother improvedWeindashYaondashLiu nonlinearconjugate gradient method with sufficient descent propertyrdquoAppliedMathematics andComputation vol 218 no 14 pp 7421ndash7430 2012
[21] M Rivaie M Mamat L W June and I Mohd ldquoA new classof nonlinear conjugate gradient coefficients with global conver-gence propertiesrdquo Applied Mathematics and Computation vol218 no 22 pp 11323ndash11332 2012
[22] A AlhawaratMMamatM Rivaie and IMohd ldquoA newmodi-fication of nonlinear conjugate gradient coefficients with globalconvergence propertiesrdquo International Journal of MathematicalComputational Statistical Natural and Physical Engineeringvol 8 no 1 pp 54ndash60 2014
[23] A Neculai ldquoAn unconstrained optimization test functionscollectionrdquo Advanced Modeling and Optimization vol 10 no 1pp 147ndash161 2008
[24] I Bongartz A R Conn N Gould P L Toint and I Bon-gartz Constrained and Unconstrained Testing EnvironmentDepartement de Mathematique 1993
[25] E P Adorio and U Diliman ldquoMvf-multivariate test functionslibrary in c for unconstrained global optimizationrdquo 2005
[26] E D Dolan and J J More ldquoBenchmarking optimization soft-ware with performance profilesrdquo Mathematical Programmingvol 91 no 2 pp 201ndash213 2002
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of