Chap. 11Numerical Differentiation and Integration

Computer Theory and Formal Methods LAB HWANG Dae-Yon (dyhwang@formal.korea.ac.kr)

SIM Jae-Hwan (jhsim@formal.korea.ac.kr)YANG Jin-Seok (jsyang@formal.korea.ac.kr)

May. 18, 2005

2Page

Computer Theory and Formal Methods Lab.

Contents

11.1 DIFFERENTIATION 11.1.1 First Derivatives 11.1.2 Higher Derivatives 11.1.3 Richardson Extrapolation

11.2 BASIC NUMERICAL INTEGRATION 11.2.1 Trapezoid Rule 11.2.2 Simpson Rule 11.2.3 Midpoint Rule 11.2.4 Other Newton-Cotes Open Formulas

3Page

Computer Theory and Formal Methods Lab.

11.1 DIFFERENTIATION

11.1.1 First Derivatives

Forward difference formula

Backward difference formula

Central difference formula

4Page

Computer Theory and Formal Methods Lab.

Example 11.1

Forward, Backward and Central DifferencesData Points(x0, y0) = (1,2) (x1, y1) = (2,4) (x2, y2) = (3,8) (x3, y3) = (4,16) (x4, y4) = (5,32)

Forward

Backward

Central

5Page

Computer Theory and Formal Methods Lab.

Three-point difference formula

Three-point forward difference formula

Three-point backward difference formula

6Page

Computer Theory and Formal Methods Lab.

Discussion

Taylor polynomial

Forward : h = xi+1 – xi

Backward : h = xi-1 – xi

7Page

Computer Theory and Formal Methods Lab.

Discussion (cont’)

Central : h = xi+1 – xi = xi – xi-1

8Page

Computer Theory and Formal Methods Lab.

General Three-Point Formula

Based on Lagrange interpolation polynomial(x1 , y1) , (x2 , y2) , (x3 , y3)

9Page

Computer Theory and Formal Methods Lab.

General Three-Point Formula(2)

10Page

Computer Theory and Formal Methods Lab.

General Three-Point Formula(3)

If h = xi+1 – xi = xi – xi-1

11Page

Computer Theory and Formal Methods Lab.

General Three-Point Formula(4)

If h = xi+1 – xi = xi – xi-1

12Page

Computer Theory and Formal Methods Lab.

11.1.2 Higher Derivative

Formula for higher derivative can be founded by Differentiating the interpolating polynomial repeatedly. Using Taylor expansions.

For example, Three equally spaced abscissas xi-1,xi, xi+1 Formula for the second derivative is

1 12

1( ) [ ( ) 2 ( ) ( )]i i i if x f x f x f x

h with truncation error O(h2)

13Page

Computer Theory and Formal Methods Lab.

11.1.2 Higher Derivative- Derivation of Second-Derivative Formula

If we assume that fourth derivative is continuous on [x-h, x+h], we can

write the error term as for some point

From the Taylor polynomial

2 3 4(4)

2( ) ( ) ( ) ( ) ( ) ( )2! 3! 4!

h h hf x h f x hf x f x f x f

2 3 4(4)

1( ) ( ) ( ) ( ) ( ) ( )2! 3! 4!

h h hf x h f x hf x f x f x f

where

1x x h and

2x h x . adding gives4

2 (4) (4)1 2( ) ( ) 2 ( ) ( ) [ ( ) ( )]

4!

hf x h f x h f x h f x f f

or 2

1( ) [ ( ) 2 ( ) ( )]f x f x h f x f x h

h with truncation error O(h4)

2(4) ( )

12

hf x h x h

14Page

Computer Theory and Formal Methods Lab.

11.1.2 Higher Derivative- Derivation of Second-Derivative Formula

Table 11.1 Centered difference formulas, all O(h2)

1 1

1( ) [ ( ) ( )]

2i i if x f x f xh

1 12

1( ) [ ( ) 2 ( ) ( )]i i i if x f x f x f x

h

2 1 1 23

1( ) [ ( ) 2 ( ) 2 ( ) ( )]

2i i i i if x f x f x f x f xh

Example 11.3 Second DerivativeEstimate the second derivative at x2 = 3, using point (x1, y1) = (2, 4),

(x2, y2) = (3, 8), and (x3, y3) = (4, 6); for this example, h=1

(4)2 1 1 24

1( ) [ ( ) 4 ( ) 6 ( ) 4 ( ) ( )]i i i i i if x f x f x f x f x f x

h

(3) [ (4) 2 (3) (2)] [16 2(8) 4] 4f f f f

15Page

Computer Theory and Formal Methods Lab.

11.1.2 Higher Derivative- Partial Derivatives

Partial derivative of a function of two variables General point as (xi, yj ) The value of the function u(x, y) at that point as u i, j

The spacing in the x and y directions is the same, h Using subscripts to indicate partial differentiation

1, 1,

1 1[ ]

2 2x i j i ju u uh h -1 0 1

i-1 i i+1

j

1, , 1,2 2

1 1[ 2 ]xx i j i j i ju u u u

h h 1 -2 1

i-1 i i+1

j

16Page

Computer Theory and Formal Methods Lab.

11.1.2 Higher Derivative- Partial Derivatives

For the mixed second partial derivative and higher derivatives, the schematic form is especially convenient.The Laplacian operatorThe bi-harmonic operator

2xx yyu u u

4 2xxxx xxyy yyyyu u u u

2

1

4xyu h 2

2

1u

h

44

1u

h

-1 0 1

0

-1

00

1 0

1

1-41

1i-1 i i+1

j-1

j

j+1

i-1 i i+1j-1

j

j+1

1 -8 20 -8 1

2 -8 2

1

2 -8 2

1

17Page

Computer Theory and Formal Methods Lab.

11.1.3 Richardson Extrapolation

Method of improving the accuracy of a low order approximation formula A(h) whose error can be expressed as

To apply Richardson extrapolation, we form approximations to A separately using the step size h and h/2

2 42 4( ) ...A A h a h a h

4 ( / 2) ( )

3

A h A hA

To continue the extrapolation process, consider4 6 8

4 6 8( ) ...A B h b h b h b h Where B(h) is simply the extrapolated approximation to A, using step sizes h/2 and h/4, this would correspond to B(h/2). we get

16 ( / 2) ( )( )

15

B h B hC h

which has error O(h6)

18Page

Computer Theory and Formal Methods Lab.

11.1.3 Richardson Extrapolation

The central difference formula can be written as2

41( ) ( ) [ ( ) ( )] ( ) ( )

2 6

hD h f x f x h f x h f x O h

h

We can also find f’(x) using one-half the previous value of h2

41( / 2) ( ) [ ( / 2) ( / 2)] ( ) ( )

24

hD h f x f x h f x h f x O h

h

Since the coefficient of the h2 term does not change , the two estimates can be combined to give

4 ( / 2) ( )

3

D h D hD

19Page

Computer Theory and Formal Methods Lab.

11.1.3 Richardson Extrapolation- Example 11.4

Improved Estimate of the Derivative From Example 11.1 h=2 The approximation to f’(x2) is based on D(h)=7.5 and D(h/2)

4(6) 7.516.5 5.5

3D

The data in the example are points on the curve f(x)=2x. The actual value of f’(x) is (ln2)2x, which gives

(3) 5.54f

20Page

Computer Theory and Formal Methods Lab.

11.1.3 Richardson Extrapolation- Discussion

Richardson extrapolation Forms a linear combination of approximation A(h) and A(h/2) Dominant error term, which depends on h2, cancels

2 42 4( ) ...A A h a h a h

2 4

2 4( / 2) ...4 16

h hA A h a a or

42

2 44 4 ( / 2) ...4

hA A h a h a

(11.2)

(11.3)

Subtracting eq. (11.2) from eq. (11.3) gives

43 4 ( / 2) ( ) ( )A A h A h O h or 414 ( / 2) ( ) ( )

3A A h A h O h

O(h2) O(h4)

21Page

Computer Theory and Formal Methods Lab.

11.1.3 Richardson Extrapolation- Discussion

To continue the extrapolation, we write4 6 8

4 6 8( ) ...A B h b h b h b h where B(h) is simply the extrapolated approximation to A, using step size h, h/2.using step size h/2 and h/4, this would corresponding to B(h/2).

4 6 8 84 6 8( / 2) ( /16) ( / 64) ( / 2 ) ...A B h b h b h b h

Therefore,6 8

6 815 16 ( / 2) ( )A B h B h c h c h Define the second level extrapolated approximation to A as

16 ( / 2) ( )( )

15

B h B hC h

22Page

Computer Theory and Formal Methods Lab.

Contents

Trapezoid Rule Example 11.5 Discussion about Trapezoid Rule

Simpson Rule Example 11.6 Example 11.7 Discussion about Simpson Rule

Midpoint Rule Example 11.8

Other Newton-Cotes Open Formulas

23Page

Computer Theory and Formal Methods Lab.

Overview (1/2)

Numerical integration rules are very important. Functions may not have exact formulas for their antiderivatives (inde

finite integrals). An exact formula for the antiderivative dose exist, it may be difficult

to find.

A numerical integration rule has the form

24Page

Computer Theory and Formal Methods Lab.

Overview (2/2)

Two basic types of Newton-Cotes formulas. “Closed” formulas : the endpoints values are used.

Trapezoid Rule Simpson Rule

“Open” formulas : the endpoints are not used. Midpoint Rule

Each of these formulas can be derived by approximating the function to be integrated by its Lagrange interpolating polynomial.

25Page

Computer Theory and Formal Methods Lab.

Trapezoid Rule

This rule approximates the curve by the straight line that passes through the points (a,f(a)) and (b, f(b)).

26Page

Computer Theory and Formal Methods Lab.

Trapezoid Rule – Example 11.5

Integral of Using the Trapezoid Rule

27Page

Computer Theory and Formal Methods Lab.

Trapezoid Rule – Discussion (1/2)

It derived from the Lagrange form of linear interpolation of f(x) using the endpoints of the interval of integration.

28Page

Computer Theory and Formal Methods Lab.

Trapezoid Rule – Discussion (2/2)

29Page

Computer Theory and Formal Methods Lab.

Simpson Rule

Approximating the function to be integrated by a quadratic polynomial leads to the basic Simpon Rule:

The approximate integral is given by

30Page

Computer Theory and Formal Methods Lab.

Simpson Rule – Example 11.6

31Page

Computer Theory and Formal Methods Lab.

Simpson Rule – Example 11.7

32Page

Computer Theory and Formal Methods Lab.

Simpson Rule – Discussion (1/2)

Simpson’s rule is found by integrating the Lagrange interpolating polynomial for f(x).

33Page

Computer Theory and Formal Methods Lab.

Simpson Rule – Discussion (2/2)

34Page

Computer Theory and Formal Methods Lab.

Midpoint Rule

If we use only function evaluations at points within the interval, the simplest formula is the midpoint rule.

This formula uses only one function evaluation (so n = 1),at the midpoint of the interval, xm=(a+b)/2.

Interpolating the function by the constant value f(xm) :

35Page

Computer Theory and Formal Methods Lab.

Midpoint Rule – Example 11.8

36Page

Computer Theory and Formal Methods Lab.

Other Newton-Cotes Open Formulas (1/2)

Using two function evaluations

Trapezoid Rule

37Page

Computer Theory and Formal Methods Lab.

Other Newton-Cotes Open Formulas (2/2)

Using three function evaluations

Simpson Rule