OPTIMAL USE OF INFORMATION IN

CERTAIN ITERATIVE PROCESSES

Robert MEERSMAN

Vrije Universiteit Brussel

I. Introduction

To approximate numerically a zero _ of a real ana-

lytic function f,

f(e) = 0,

iteration is most widely used. We will discuss so-

called k-point stationary iterative processes

without memory, _, defined as follows• Let there

exist an interval around a such that for all x in1

this interval, the following functions

_i' i=2,...,k+l are well-defined :

z I = x 1

z2 = _2(Zl'N(z l;f))

z 3 = _3(Zl,Z2,N(Zl,Z2;f))

x2:=_(Xl)=Zk+l - _k+l (Zl'''''Zk'N(Zl'''''Zk ;f)) '

where

a) x 2 is called the new approximation to e ; we

assume that x 2 lies within the same interval a

and that the process is converging, i.e. put-

ting Xl:= x 2 and repeating the iterative step

produces a sequence which converges to _. Also

we assume f' (_) _ 0.

b) N(zl,...,zi;f) is called the information set ati iiii iJ • u ii lINK

This research was supported in part by the National ScienceFoundation under Grant GJ321]] and the Office of Naval Re-

search under Contract N00014-67-A-0314-00]0, NR 044-422.

the points Zl,...,z i and consists of the set of

all evaluations of f and/or its derivatives

used at those points when computing zi+ I. It is

not necessary to give all derivatives up to a

certain degree at any particular point z i.

Examples of specific N abound in the rest of

the paper. Other kinds of information sets can

be considered, but are not the topic of this

paper ; see for instance Kacewicz [75]. The read-

er should also be more or less familiar with the work off .

Woznlakowski [74] [75a].

2. Some known results and conjectures

Definition 2. I.

Let N(z l,...,zk;f) be given. We say that£%_ £_'%

= f mod N when for all f_JJ (z i) 6 N,

f(J) _zi) = _(J)(zi)

/As in Woznlakows_i [T5a] we adopt the following definitions"

Definition 2.2.

The order of iteration p(_) is the largest number

such that for all f with f(e) = 0,

for all _ - f mod N and having a zero e near e,

we have

[¢(xl;N) -lim sup

Xl._ IXl - elP (¢) <

It can be shown this definition coincides with the

usual definition in the literature (e.g. Traub[64])

under weak assumptions on the asymptotical constant

Definition 2.3.• . i i i

The order of information p(N) is the largest num-

ber such that for all f with f(a) = 0,

for all _ - f mod N and having a zero a near a,

we have

lim sup la - al _) <Ix -Xl 1

Remark that choosing a special f or _ - f mod N

will give upper bounds on these orders, by defini-

tion. See also Wo_niakowski [755]. Extensive use will

be made of the following two theorems.

Theorem 2.1. (Maximal Order Theorem)

The order p(¢) is bounded above by p(N) for all

using information N.

Theorem 2.2.

The maximal order is reached for the generalized

interpolating methods I N : p(I N) = p(N) .

For definitions and proofs, see Wo{niakoWski [75a].

Two kinds of problems arise now.

Problem i. Given N, compute p(N) . (In other words,

what is the maximal order achievable with informa-

tion N).

Problem 2. Given n = @N, the number of elements in| _ i __

N, determine P_ = max max p(_), where ¢ uses in-n @N=n

formation N.

Problem 2 is much harder than problem i. Kung and

Traub conjectured (Kung and Traub [74a]) that

(2.3) P_ = 2n-In

and exhibited two families of methods which rea-

lize this bound for each n. In a later paper, Kung

and Traub [74b] , they proved (2.3) for n = 1 and 2.

Remark that in view of Theorems 2.1 and 2.2, (2.3)

is equivalent to

2n-iQn =

where Qn = max p(N).N=n

The conjecture has been settled by Wo_niakowski in

one very important case.

Definition 2.4.

N is hermitean iff for all i, 1 _ i _ k, we have

f(k) (zi)6N_f(k-l) (zi)eN for all k > 0.

Theorem 2.3. (Wo_niakowski [755])

The conjecture of (2.3) is true if the maximum is taken over

hermitean /V. We will show later that a partial converse is

not true, i.e. that p(A/) = 2_v-] does not imply that A is

hermitean.

3. A solution to problem 1 for n = 3

We will prove in this section that P_3 = 4, showing

the correctness of the conjecture in this case.

Our proof uses special cases of some general re-

sults on certain n-evaluations iterations one of

which is to be treated in a later paper, namely

Theorem 3.1.

If N = {f(zl) ,fl(z2) ,...,f(n) (Zn+l) } then

p (N) _ 2n

Note : N is the so-called Abel-Con_arov informa-

tion.

In the proof of the following lemma and the rest

of the paper we assume that at each z i used in _,

some new information is computed. This is not a

restriction since otherwise we can substitute the

expressions for these z i in the other _j, obtai-

ning an equivalent iteration (with less points).

Lemma 3.1., ii

Let # be an iteration using two pieces of informa-

tion, i.e.

z = x1 1

z 2 = _2(Zl,N(Zl;f))

x2:=_(Xl)=Z3 = _3(Zl,Z2,N(Zl,Z2;f)) with @N = 2

(By the above convention, _ is a I- or 2-point

iteration) .

Then, if there exists a (known) constant C, C # 0

and C _ i, such that for all f

- z2 = C(_-z I) + 0(_-Zl) 2, (3.1)

then _ cannot be of second order.

Proof : If C _ I, then from (3.1) we could solve

for _ : r

z 2 - C z I 2= 1 - C + 0 (_-z I)

Z - C Z2 1

And z 2 = 1 - C would therefore produce a

second order approximation to _. Since P1 = I,

both pieces of information must be used then at z i'

but then _ is a one-point iteration, i.e. z 3 = z 2

by the convention, and from (3.1) and C _ 0 it fol-

lows that _ is only of first order.

Theorem 3.2.

P3 = 4

Proof : We prove that p_(N) _ 4 for all N with

_N = 3, where

p (N)=max {p lfor all _=f+G, G-0 mod N, _(_)=0,

and G monic polynomial of degree ( 3,

lim sup I_-_1 < _,}%

x Ix1- IPthereby restricting the class of _ such that

- f rood N.

Step 1

we need one evaluation of f at z to assure con-1

vergence so N is of the form

N = {f(zl) , f(i) (z2) , f(j) (z3) }

(Kung and Traub [74a]).

We will suppose z 2 and z 3 not necessarily diffe-

rent, unless of course i or j equals 0 or i = j.

It is clear this does not affect the bounds on the

optimal order.

Since now G is a monic polynomial of degree _ 3,

we can take i and j _ 2. Indeed, if i or j > 3,

G H 0 mod N is automatically satisfied ; if

i < j = 3, we can interpolate the zero function

for this information at z I and z 2 _ith a monicpolynomial of degree • 2 - from P2 = 2 it follows

that the optimal order is 2 < 4, and similarly if

j < i = 3. If i = j = 3 we can even take

G(z) = z - z I in which case the order of informa-

tion evidently is equal to 1 < 4.

Remark

The above argument can of course be generalized to

any n : it is closely related to the P61ya condi-

tions on the set N, see Wo_niakowski [75b], Sharma

[72].

The different cases for the information N.

With an obvious notation, in the following cases

the answer is already known :

Case 1 : {fl,f2,f3} : Hermitean N, order • 23-1=4

' f_} .- "Brant information with m=0"Case 2 : {fl,f2,

applying the results of Sac.

4, we find

if Zl_Z2:P(N)•m+2(k-l)-l=0+4-1=3<4 ;

if Zl=Z2:P(N)=m+2(k-l)+l=l+2+l=4

Case 3 : {fl,f_,f_} : "Abel-Goncarov" N, by Theo-rem 3.1. : order • 2.2 = 4

Case 4 • {fl' '• f_ f_} : Take again G(z)=z-z I as in

Step 1 ; order • 1 < 4 (here

the P61ya conditions are not

satisfied).

Step 3

Exhaustive checking of the remaining cases. Let us

set

G(z) = (z-z I) (z2+az+b)

Case 5 : {fl'f2'f3 }

Now G(z) = (z-zI)(z-z2) (z-c).

The condition _' (z 3) = 0 gives

(2z3-zl-z2) (z3-c)+(z3-zl) (z3-z 2) = 0

So in general, c is a function of z I, z 2 and z 3

which itself is also a function of z I and z 2.

Since P_ = 2, e-c cannot be of higher order than

(e-Zl) 22. Therefore

- _ = 0(G(_)) = 0(_(_)) = 0(_-z I) (e-z 2) (s-c)

4cannot be of higher order than (e-z I) since e-z 2

1 Thusis at most of order (_-z I) because P l = "

p (N) _ 4 for this N.

Case 6 : {fl'f2'f_ }

Completely analogous to case 5, the condition at

z 3 now is

(z3-c) + (2z3-zl-z 2) = 0.

Case 7 : {fl'f_'f3 }

Now G(z) = (z-zI) (z-z3)(z-c).

The condition at z2 : c = 3z 2 - z I - z 3, again

gives that (e-c) is at most of second order in

(_-zI).

Now (e-z 3) is at most of first order in (e-z I)

since the function

_(z) = z- z1

interpolates the zero function at the points z1

and z 2 for the given information. Thus with an ob-

vious notation,

_( ) - _ = 0(e-Zl) , and by theorem 2.1 and 2.2,

(_-z3) = 0(_-z I) at most.

Consequently,

e(G)-e = 0(_(e))=0(e-Zl) (e-z3) (e-c)

4is again at most of order (e-z I) .

Case 8 : {fl,f_,f_}

This is a permutation of the Abel-Gon_arov infor-

mation. It is an easy consequence of the proof of

Theorem 3.1 that also here we have,

p(N) _ 2 - 2 = 4.

A direct proof for this case can also be found,

and is left to the reader.

, f }Case 9 : {fl'f2' 3

G(z) = (z-zI) (z-z3) (z-c).

Again, (s-c) being of 3rd order or more in (e-Zl)

_ = 2 so if (e-z 3) is of orderwould contradict P2

1 in (e-z I) we are done. However (e-z 3) and (e-c)

cannot be both of second order, since the condi-

tion G' (z2) = 0 reads

(2z2-zl-z3) (z2-c)+(z2-z I) (z2-z 3) = 0

which is easily seen to be equivalent to

e 3(s-c)=e 2 (2el-3e 2)-(el-2e 2) (e3+(s-c))

i0

• = a - z i ; i=1,2,3.where e 1

Now e 3(a-c) cannot be of order 4 while by lemma

2 -3e ) is at most of Order 23.1 with C = 7' e2 (2el 2

and (el-2e2) (e3+(a-c)) is at least of third order.

Note that if 2z 2 - z I - z 3 = 0, this case becomes

non-poised (Sharma [72]) and the function

G(z) = (Z-Zl) (z-z 3) interpolates zero with respect

to this N, giving a maximal order of 3.

Theorem 3.3.

Hermitean information is not uniquely optimal.

Proof : We exhibit two examples for n = 3

a) Consider again case (6) of Theorem 3.9.

We have G(a) = (a-z I) (a-z 2) (a-c)

where c = 3z 3 - z I z 2.

To obtain order 4, this suggests we must have

2- c = 0(a-z I) , or

Zl+Z2 1 2(3.1) z3 - 3 + _y with y-a=8 (a-z I)

To find such a y, take

z I = x 1

z 2 = z I + f(z I)

and y the root of the interpolating (Ist degree)

polynomial at these two points, i.e.

z 2 - z 1

y = zI - f(zI) - f(zi)f(zI)

Then construct z 3 by (3.1) and

x2:= ¢(x I) := z 4

as the root of the (2nd degree) polynomial inter-

ii

polating f with respect to the information at Zl,

z2 and z 3. This _ethod is easily seen to be offourth order.

b) Case 3 with z I = z 2. Now G(z) = (Z-Zl)2(z-c)

and G" (z 3) = 0 gives 3z 3 = 2z I + c. By taking2 1

z3 = _Zl + _ Y where again _ - y = 0(_-Zl ) 2, for

example by a Newton-step, it is easy to show that

the following method has fourth order :

z I = x I (= z 2)

2 + 1 f (Zl)

z 3 : _ z I _ (z I - f,(zi))

X2:=#(x I) := z4 = zero of the second degree polyno-

mial interpolating f with respect to the informa-

tion at z I and z 3.

Remarks

The previous arguments permit the determination of

all arrangements of the information (_N = 3) which

can give optimal order. They are denoted by their

incidence matrices (Sharma [72]) as follows :

IOo0i]°IE) I 0 0 F) i 0 G) i 0 0]

100 10 01 0]001 01 00

The cases A) and C) have optimal generalizations

for all n > 3, it will be shown in a later paper

that also case F) can be generalized to a non-her-

mitean optimal case for all n.

12

4. A solution to a "problem 2" for N = Brent infor-mation.

Definition 4.1.

- i) i)Nm,£,k_{f(z ,f, (Zl) ,...,f(m) (z ;

f(£) (z2) ,...,f(£) (Zk) }

is called Brent-information, where the zi are dis-

tinct, m >. i, k >. 2 and _ _ i.

Brent has shown the following

Theorem 4.1. (Brent [ 74])

Assume £ _ m + I.

There exist methods using Nm Z k of order m+2k-l.f f

We will now prove that the Brent methods make op-

timal use of the information N Z k (with respectm, ,to order).

Theorem 4.2.

Let N _ k be as in definition 4.1. Then if Z_m+l,mt ,

p(N) = m + 2k - 1

If Z > m + i, p(N) = m + I.

Proof : A technique is used similar to theorem 3.2.

If _ > m + i, a function G(z) - 0 rood N is given

bym+ 1

G(z) = (z-z I)

And consequently I_- = •O{I = 0(_(e)) 0(e-Zl )m+l

By theorem 2.1 the maximal order is not more than

m + i. Methods realizing this order exist and are

trivial to find. Thus p(N) = m + i.

If £ ( m + i, we construct G(z) as follows.

13

To satisfy the conditions at z2,...,z k we must have

G (£) (z) = (Z-Zl)m-£+l (z-z2) (z-z3) ... (Z-Zk) H(z)

where H(z) is any (sufficiently regular) function.

Integrating £ times,

G(z) = (t-z) £-I (t-zl)m-£+l (t-z2) ... (t-Zk) H(t)dt.z 1

According to the remark at (2.2), we obtain an

upper bound by choosing a special H.

Take H(t) = (t-z 2) ... (t-zk) , making

G(z) = (t-z) £-i (t_zl)m-£+l [ (t_z2) ... (t_zk) I 2 dt.z1

Consider now G(_) = _(e). Transform G(e) to the

interval [-i,+I] ; after some easy calculations we

obtain

(_) = K(e-z I) £-I (__zl)m-£+l (__Zl) 2(k-l) (__Zl) .I(e)

where K M 0 does not depend on e or any of the z.1and where

+ 1 k z -z.I (e) = (l-T) £-l(l+T)m-£+l H (T+-I -i)2 _dT.

-I i=2 _-Zl

As is well known, I(_) is minimized when

k z -z.

H (T + 1 l) is equal to the (k-1) st monici=2 e-Zl

Jacobi polynomial corresponding to the weight

function (l-T) £-i (l+T)m-£+l.

Then I (e) >, c where c is independent of the zi.

(See for example G. Natanson : "Konstruktive Funk-

tionentheorie ") .

so = o = o p

14

with p = (£-i) + (m-Z+l) + 2(k-l) + 1

= m+ 2k- I.

Thus p(N) _ m + 2k - I, but by Brent's theorem and

theorem 2.1, we have equality.

Note : The previous theorem was independently dis-

covered by Wo{niakowski. A generalization of this

theorem is possible•

Theorem 4.3.

Let now N = {f(z I) ,f' (z I) ,... ,f(ml) (z I) ;

f(£) (z2) ,...,f(£+m2) (z2) ;

f(£) (z3) ,...,f(£+m3) (z3) ;

f(_) (Zk),...,f (£+mk) (zk) } ml>.l,

£_i.

Then if £ > m I + i, p(N) = m I + I, and if £_ml+l ,

(4•1) p(N) _ 1 + m I + 2. I ii=l

Proof : The proof runs analogously, with H replaced

byk e.

H(t) = H (t_zl) x with E i = f0 if m i odd

i=2 _i if m i even

(The Jacobi polynomial is now of degree 7 i ! )i=2

Remark 4.1.

Let m i = 1 for i = 2,...,k. Then p(N) _ m + 2k - I,

so we gain nothing compared to the Brent informa-

tion case ! In general the order cannot be raised

if all m i are even and we add the pieces of infor-

15

mation f (Z+mi+l) (zi) for i = 2,...,k.

Remark 4.2.

Contrary to Theorem 4.2 the inequality (4.1) is not

yet known to be an equality in general. If all m i

are equal, methods can be constructed by means of

so-called "s-polynomials" realizing the bound. For

a definition of these, see Ghizetti and Ossicini,

"Quadrature Formulae". Again, for details we refer

to a forthcoming paper on this subject.

Remark 4.3.

Kung and Traub's conjecture states that the optimal

order for a given number of pieces of information

will double by adding one extra piece of informa-

tion. That it is however possible to increase the

order more than twofold when it is not optimal, is

shown by the following example :

Let N = {f(zl) ,f'(zl) ,f"(z I) ; f'(z2) ,f"(z2) }.

By theorem 4.3 and remark 4.1 any method using this

information must have order at most 5. Adding the

element {f(z2)} to N, we get however an information

at which allows us to obtain order 12, as is easi-

ly shown. Although of course not a counterexample

to the conjecture - we believe it is true - it will

complicate any possible proof by induction.

Finally, we state without proof the following re-

sult, used in the proof of Theorem 3.1 :

16

Theorem 4.4.

If in Theorem 4.3 m I = 0 (and consequently, to

avoid trivial cases, 1 = i) the order of informa-

tion is bounded by

p(N) _ i + m 2 + I .i=3

References

Brent [74] Brent. R.. "Efficient Methods for Finding Zeroesof Functions whose Derivatives are Easy to Evalu-

ate," Department of Computer Science Report,

Carnegie-Mellon University, December, ]974.

Kacewicz [75] Kacewicz, B., "An Integral-interpolatory Meth-of for the Solution of Non-linear Scalar Equa-

tions," Department of Computer Science Report,Carnegie-Mellon University, January, ]975.

Kung and Traub [74a] Kung, H. T. and Traub, J. F., "OptimalOrder of One-point and Multipoint Iteration,"JACM 2], 643-65].

Kung and Traub [74b] Kung, H. T. and Traub, J. F., "OptimalOrder and Efficiency for Iterations with Two

Evaluations," to appear in SIAM J. Numer. Anal.

Sharma [72] Sharma, A., "Some Poised and Non-poised Problems

of Interpolation," SIAM Review ]4, ]972.

Traub [64] Traub, J. F., Iterative Methods for the Solution

of Equations, Prentice-Hall, ]964.

Woz'niakowski [74] Woz'niakowski, H., "Maximal StationaryIterative Methods for the Solution of Operator

Equations," SlAM J. Numer. Anal. ]], ]974, 934-949.

Wozniakowskl [75a] WozSniakowski, H., "Generalized Informa-tion and Maximal Order of Iteration for Operator

Equations," SlAM J. Numer. Anal. ]2, ]975, ]2]-]35.

17

/ i "Maximal Order of MultiWozniakowski [75b] Wozniakowski, H.,I!

point Iterations Using N Evaluations, to appear

in Analytic Computational Complexity, edited byJ. F. Traub, Academic Press, 1975.

UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE (When Oila Entered)

REPORT DOCUMENTATION PAGE READINSTRUCTIONSBEFORE COMPLETING FORMt. RE_ORT

12 GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBERNUMBER

ii

4. TITLE (and Subtitle) 5. TYPE OF REPOR _ & _ERIOO COVERED

OPTIMAL USE OF INFORMATION IN Interim

CERTAIN ITERATIVE PROCESSES 6 PERFORMING ORG DEDOR_ NU.REP

7. AUTHOR(m) 8. CONTRACT OR GRAN _ NUMBER_s_

Robert Meer sman NO 014-67 -A-0314-00 I0

NR 044-422.9 PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT TASK

AREA & WORK UNIT NUMBERS

Carnegie-Mellon University

Computer Science Dept.Pittsburgh, PA 15213

1 CONTROLLING ")_ICE NAME AND ADDRESS 12. REPORT DATE

Office of Naval Research ,, July 1975Arlington, VA 22207 ,_ NUMBER O_ PAGES

1814 MONITORING 4GENCv NAME & ADDRESSft/different from Contrnllini_ Office) 15. SECURITY CLASS (of this reDort)

UNCLAS S IFIED

15a. D_CLASSIFICATION DOWNGRADINGSCHEDULE

16 DISTRIBU_'_ON STATEMENT (of thts Report)

Approved for public release; distribution unlimited.

17 DIS'rRIBUTION STATEMENT (o! the abstract entered In Block 20, l! dl!!erent !zorn Report)

18. SUPDt EMENTARY NOTES

19. KEY WO_DS !Continue on reverse side tf necessary and Identity by block number)

20. A BS ? _ a __" ((_ontinuQ on reverse side If necese_P/ attd Idontifv b v block number)

none submitted

_OPM

_uu ,.,,,, ,_ 1473 EO,TION OF ' NOV 65 IS OBSOLETE UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE (When ]ol, t_ Ent,_r_d)