Overview

1 Root findingRecapNewton–Raphson

2 Numerical integrationNewton–Cotes formulaeAccuracy analysisRomberg integration

3 Ordinary differential equationsThe Euler methodHigher-order ODEs

4 Summary

Next quiz 24/26 March!

Recap / Outline

Start by bracketing the root

Bisection: halve the interval — always works!

False position: interpolate to guess next point

Secant: similar to false position, but does not rely on bracketing.May fly off to outer space!

Bisection has linear convergence

Secant is superlinear

False position is usually superlinear

Recap / Outline

Start by bracketing the root

Bisection: halve the interval — always works!

False position: interpolate to guess next point

Secant: similar to false position, but does not rely on bracketing.May fly off to outer space!

Bisection has linear convergence

Secant is superlinear

False position is usually superlinear

Recap / Outline

Start by bracketing the root

Bisection: halve the interval — always works!

False position: interpolate to guess next point

Secant: similar to false position, but does not rely on bracketing.May fly off to outer space!

Bisection has linear convergence

Secant is superlinear

False position is usually superlinear

Recap / Outline

Start by bracketing the root

Bisection: halve the interval — always works!

False position: interpolate to guess next point

Secant: similar to false position, but does not rely on bracketing.May fly off to outer space!

Bisection has linear convergence

Secant is superlinear

False position is usually superlinear

Newton–Raphson

Use Taylor expansion to estimate the root

Very good if you are close enough

Most useful if you have analytical expression for f ′(x)

Much less useful in 1-d if you must use numerical derivative:Secant is better, Ridders is much better!

Almost only general method on the market in n-dim

Can go horribly wrong!

Simple principle:

f (x + ε) = f (x) + εf ′(x) +ε2

2f ′′(x) + . . .

Ignore higher order terms, find ε : f (x + ε) = 0

=⇒ ε = −f (x)/f ′(x)

Newton–Raphson

Use Taylor expansion to estimate the root

Very good if you are close enough

Most useful if you have analytical expression for f ′(x)

Much less useful in 1-d if you must use numerical derivative:Secant is better, Ridders is much better!

Almost only general method on the market in n-dim

Can go horribly wrong!

Simple principle:

f (x + ε) = f (x) + εf ′(x) +ε2

2f ′′(x) + . . .

Ignore higher order terms, find ε : f (x + ε) = 0

=⇒ ε = −f (x)/f ′(x)

Newton–Raphson

Use Taylor expansion to estimate the root

Very good if you are close enough

Most useful if you have analytical expression for f ′(x)

Much less useful in 1-d if you must use numerical derivative:Secant is better, Ridders is much better!

Almost only general method on the market in n-dim

Can go horribly wrong!

Simple principle:

f (x + ε) = f (x) + εf ′(x) +ε2

2f ′′(x) + . . .

Ignore higher order terms, find ε : f (x + ε) = 0

=⇒ ε = −f (x)/f ′(x)

Newton–Raphson

Use Taylor expansion to estimate the root

Very good if you are close enough

Most useful if you have analytical expression for f ′(x)

Much less useful in 1-d if you must use numerical derivative:Secant is better, Ridders is much better!

Almost only general method on the market in n-dim

Can go horribly wrong!

Simple principle:

f (x + ε) = f (x) + εf ′(x) +ε2

2f ′′(x) + . . .

Ignore higher order terms, find ε : f (x + ε) = 0

=⇒ ε = −f (x)/f ′(x)

Newton–Raphson

Use Taylor expansion to estimate the root

Very good if you are close enough

Most useful if you have analytical expression for f ′(x)

Much less useful in 1-d if you must use numerical derivative:Secant is better, Ridders is much better!

Almost only general method on the market in n-dim

Can go horribly wrong!

Simple principle:

f (x + ε) = f (x) + εf ′(x) +ε2

2f ′′(x) + . . .

Ignore higher order terms, find ε : f (x + ε) = 0

=⇒ ε = −f (x)/f ′(x)

Newton–Raphson in more dimensions

We have N equations for N variables:

fi (x1, . . . , xN) = 0 , i = 1, . . . ,N

Taylor expansion:

fi (~x + ~δ) = fi (~x) +N∑

j=1

∂fi∂xj

δj +O(δ2)

In vector–matrix notation:

F (x + δ) = F (x) + J · δ , Jij ≡∂fi∂xj

= Jacobian

Solve matrix eq Jδ = −F for the vector δ.

Newton–Raphson in more dimensions

We have N equations for N variables:

fi (x1, . . . , xN) = 0 , i = 1, . . . ,N

Taylor expansion:

fi (~x + ~δ) = fi (~x) +N∑

j=1

∂fi∂xj

δj +O(δ2)

In vector–matrix notation:

F (x + δ) = F (x) + J · δ , Jij ≡∂fi∂xj

= Jacobian

Solve matrix eq Jδ = −F for the vector δ.

Newton–Raphson in more dimensions

We have N equations for N variables:

fi (x1, . . . , xN) = 0 , i = 1, . . . ,N

Taylor expansion:

fi (~x + ~δ) = fi (~x) +N∑

j=1

∂fi∂xj

δj +O(δ2)

In vector–matrix notation:

F (x + δ) = F (x) + J · δ , Jij ≡∂fi∂xj

= Jacobian

Solve matrix eq Jδ = −F for the vector δ.

Numerical integration

Two main approaches

1 Quadrature (classical numerical integration):add up integrand at a series of points

2 Transform integral into an ode:

I (y) =

∫ y

y0

f (x)dx ⇐⇒ dI

dy= f (y) , I (y0) = 0 .

→ our main business this semester!

Other approaches

use polynomial or rational approximations

Fourier-type integrals: Fast Fourier Transform (next semester)

Multidimensional integrals: Monte Carlo (next semester)

Numerical integration

Two main approaches

1 Quadrature (classical numerical integration):add up integrand at a series of points

2 Transform integral into an ode:

I (y) =

∫ y

y0

f (x)dx ⇐⇒ dI

dy= f (y) , I (y0) = 0 .

→ our main business this semester!

Other approaches

use polynomial or rational approximations

Fourier-type integrals: Fast Fourier Transform (next semester)

Multidimensional integrals: Monte Carlo (next semester)

Numerical integration: quadrature

History dates back to Newton, mainstay of numerical methods up to 19th cNewton–Cotes formulae (trapezium, Simpson, Boole,. . . )

Trapezium rule

∫ b

af (x)dx =

N∑n=1

∫ xn

xn−1

f (x)dx

≈N∑

n=1

(1

2fn−1 +

1

2fn

)· δ

This is the most useful of all quadrature rules!

Numerical integration: quadrature

History dates back to Newton, mainstay of numerical methods up to 19th cNewton–Cotes formulae (trapezium, Simpson, Boole,. . . )

Trapezium rule

∫ b

af (x)dx =

N∑n=1

∫ xn

xn−1

f (x)dx

≈N∑

n=1

(1

2fn−1 +

1

2fn

)· δ

This is the most useful of all quadrature rules!

Numerical integration: quadrature

History dates back to Newton, mainstay of numerical methods up to 19th cNewton–Cotes formulae (trapezium, Simpson, Boole,. . . )

Trapezium rule

∫ b

af (x)dx =

N∑n=1

∫ xn

xn−1

f (x)dx

≈N∑

n=1

(1

2fn−1 +

1

2fn

)· δ

This is the most useful of all quadrature rules!

Quadrature

Midpoint rule

∫ b

af (x)dx =

N∑n=1

∫ xn

xn−1

f (x)dx

≈N∑

n=1

fn−1/2 · δ

Simpson’s rule∫ xn+1

xn−1

f (x)dx ≈ δ ·(1

3fn−1 +

4

3fn +

1

3fn+1

)∫ b

af (x)dx ≈ δ ·

(1

3f0 +

4

3f1 +

2

3f2 +

4

3f3 + . . . +

4

3fN−1 +

1

3fN

)

Quadrature

Midpoint rule

∫ b

af (x)dx =

N∑n=1

∫ xn

xn−1

f (x)dx

≈N∑

n=1

fn−1/2 · δ

Simpson’s rule∫ xn+1

xn−1

f (x)dx ≈ δ ·(1

3fn−1 +

4

3fn +

1

3fn+1

)∫ b

af (x)dx ≈ δ ·

(1

3f0 +

4

3f1 +

2

3f2 +

4

3f3 + . . . +

4

3fN−1 +

1

3fN

)

Accuracy

Consider one step of the integral using trapezium rule:∫ xn

xn−1

f (x)dx ≈ δ

2(fn−1 + fn)

Taylor expand the integrand around xn:

f (xn + ε) = f (xn) + εf ′(xn) +ε2

2f ′′(xn) +O(ε3)

The sub-integral is∫ xn

xn−1

f (x)dx =

∫ 0

−δf (xn + ε)dε =

∫ 0

−δ

[fn + εf ′n +

ε2

2f ′′n +O(ε3)

]= δfn −

δ2

2f ′n +

δ3

6f ′′n +O(δ4)

(I)

Accuracy

Consider one step of the integral using trapezium rule:∫ xn

xn−1

f (x)dx ≈ δ

2(fn−1 + fn)

Taylor expand the integrand around xn:

f (xn + ε) = f (xn) + εf ′(xn) +ε2

2f ′′(xn) +O(ε3)

The sub-integral is∫ xn

xn−1

f (x)dx =

∫ 0

−δf (xn + ε)dε =

∫ 0

−δ

[fn + εf ′n +

ε2

2f ′′n +O(ε3)

]= δfn −

δ2

2f ′n +

δ3

6f ′′n +O(δ4)

(I)

Accuracy

Consider one step of the integral using trapezium rule:∫ xn

xn−1

f (x)dx ≈ δ

2(fn−1 + fn)

Taylor expand the integrand around xn:

f (xn + ε) = f (xn) + εf ′(xn) +ε2

2f ′′(xn) +O(ε3)

The sub-integral is∫ xn

xn−1

f (x)dx =

∫ 0

−δf (xn + ε)dε =

∫ 0

−δ

[fn + εf ′n +

ε2

2f ′′n +O(ε3)

]= δfn −

δ2

2f ′n +

δ3

6f ′′n +O(δ4)

(I)

Accuracy

Now Taylor expand fn−1 around xn:

f (xn − δ) = f (xn)− δf ′(xn) +δ2

2f ′′(xn) +O(δ3)

The trapezium approximation then becomes

δ

2(fn−1 + fn) =

δ

2fn −

δ

2δf ′n +

δ

2

δ2

2f ′n +

δ

2fn +O(δ4)

= δfn −δ2

2f ′n +

δ3

4f ′′n +O(δ4)

(T)

Compare with (I):∫ xn

xn−1

f (x)dx = δfn −δ2

2f ′n +

δ3

6f ′′n +O(δ4)

Accuracy

Now Taylor expand fn−1 around xn:

f (xn − δ) = f (xn)− δf ′(xn) +δ2

2f ′′(xn) +O(δ3)

The trapezium approximation then becomes

δ

2(fn−1 + fn) =

δ

2fn −

δ

2δf ′n +

δ

2

δ2

2f ′n +

δ

2fn +O(δ4)

= δfn −δ2

2f ′n +

δ3

4f ′′n +O(δ4)

(T)

Compare with (I):∫ xn

xn−1

f (x)dx = δfn −δ2

2f ′n +

δ3

6f ′′n +O(δ4)

Accuracy

Now Taylor expand fn−1 around xn:

f (xn − δ) = f (xn)− δf ′(xn) +δ2

2f ′′(xn) +O(δ3)

The trapezium approximation then becomes

δ

2(fn−1 + fn) =

δ

2fn −

δ

2δf ′n +

δ

2

δ2

2f ′n +

δ

2fn +O(δ4)

= δfn −δ2

2f ′n +

δ3

4f ′′n +O(δ4)

(T)

Compare with (I):∫ xn

xn−1

f (x)dx = δfn −δ2

2f ′n +

δ3

6f ′′n +O(δ4)

Accuracy of trapezium rule

Accuracy of single step

δ

2(fn−1 + fn) =

∫ xn

xn−1

f (x)dx +δ3

12f ′′n +O(δ4) =

∫ xn

xn−1

f (x)dx +O(δ3)

Accuracy of entire integral

What is the error on I =∫ ba f (x)dx?

— sum up the error on each step!— we cannot assume any cancellations

IT = I +N∑

n=1

O(δ3) = I + NO(δ3)

But N = (b − a)/δ, so NO(δ3) = (b − a)O(δ2) = O(δ2)

Accuracy of trapezium rule

Accuracy of single step

δ

2(fn−1 + fn) =

∫ xn

xn−1

f (x)dx +δ3

12f ′′n +O(δ4) =

∫ xn

xn−1

f (x)dx +O(δ3)

Accuracy of entire integral

What is the error on I =∫ ba f (x)dx?

— sum up the error on each step!— we cannot assume any cancellations

IT = I +N∑

n=1

O(δ3) = I + NO(δ3)

But N = (b − a)/δ, so NO(δ3) = (b − a)O(δ2) = O(δ2)

Accuracy of trapezium rule

Accuracy of single step

δ

2(fn−1 + fn) =

∫ xn

xn−1

f (x)dx +δ3

12f ′′n +O(δ4) =

∫ xn

xn−1

f (x)dx +O(δ3)

Accuracy of entire integral

What is the error on I =∫ ba f (x)dx?

— sum up the error on each step!— we cannot assume any cancellations

IT = I +N∑

n=1

O(δ3) = I + NO(δ3)

But N = (b − a)/δ, so NO(δ3) = (b − a)O(δ2) = O(δ2)

Accuracy of trapezium rule

Accuracy of single step

δ

2(fn−1 + fn) =

∫ xn

xn−1

f (x)dx +δ3

12f ′′n +O(δ4) =

∫ xn

xn−1

f (x)dx +O(δ3)

Accuracy of entire integral

What is the error on I =∫ ba f (x)dx?

— sum up the error on each step!— we cannot assume any cancellations

IT = I +N∑

n=1

O(δ3) = I + NO(δ3)

But N = (b − a)/δ, so NO(δ3) = (b − a)O(δ2) = O(δ2)

Accuracy of trapezium rule

Accuracy of single step

δ

2(fn−1 + fn) =

∫ xn

xn−1

f (x)dx +δ3

12f ′′n +O(δ4) =

∫ xn

xn−1

f (x)dx +O(δ3)

Accuracy of entire integral

What is the error on I =∫ ba f (x)dx?

— sum up the error on each step!— we cannot assume any cancellations

N = (b − a)/δ =⇒ NO(δ3) = (b − a)O(δ2) = O(δ2)

IT = I + NO(δ3) = O(δ2)

Accuracy: Midpoint and Simpson

Midpoint rule

The best here is to Taylor expand integrand around midpoint, which give∫ xn

xn−1

f (x)dx = δ · fn−1/2 +δ3

24f ′′n−1/2 +O(δ5)

=⇒ IM = I +O(δ2)

Midpoint and trapezium are both accurate up to O(δ2) or O(1/N2).

Simpson’s rule

∫ xn+1

xn−1

f (x)dx = 2δ · fn +δ3

3f ′′n +

δ5

60f (4) +O(δ7)

δ ·(1

3fn−1 +

4

3fn +

1

3fn+1

)= 2δ · fn +

δ3

3f ′′n +

δ5

36f (4) +O(δ7)

IS = I +O(δ4) = I +O(1/N4)

Accuracy: Midpoint and Simpson

Midpoint rule

The best here is to Taylor expand integrand around midpoint, which give∫ xn

xn−1

f (x)dx = δ · fn−1/2 +δ3

24f ′′n−1/2 +O(δ5)

=⇒ IM = I +O(δ2)

Midpoint and trapezium are both accurate up to O(δ2) or O(1/N2).

Simpson’s rule

∫ xn+1

xn−1

f (x)dx = 2δ · fn +δ3

3f ′′n +

δ5

60f (4) +O(δ7)

δ ·(1

3fn−1 +

4

3fn +

1

3fn+1

)= 2δ · fn +

δ3

3f ′′n +

δ5

36f (4) +O(δ7)

IS = I +O(δ4) = I +O(1/N4)

Romberg integration

Basic idea:

evaluate IT (trapezium rule) with N points (stepsize δ)

then evaluate with 2N points (include all midpoints) — stepsize δ/2

then evaluate with 4N points, etc.

This gives a series of numbers depending on δ→ extrapolate as polynomial in δ to δ → 0.

The series is in fact a polynomial in δ2 (all odd terms cancel)

The first order extrapolation gives Simpsons rule!

Romberg integration

Basic idea:

evaluate IT (trapezium rule) with N points (stepsize δ)

then evaluate with 2N points (include all midpoints) — stepsize δ/2

then evaluate with 4N points, etc.

This gives a series of numbers depending on δ→ extrapolate as polynomial in δ to δ → 0.

The series is in fact a polynomial in δ2 (all odd terms cancel)

The first order extrapolation gives Simpsons rule!

Ordinary differential equations

Consider again the integral I =∫ ba f (x)dx .

Introduce the function

I (x) =

∫ x

af (ξ)dξ =⇒ I (a) = 0, I (b) = I .

This function obeys the ODE

dI

dx= f (x) , I (a) = 0

Our integral has been transformed into an ODE inital-value problem!

Main advantage

Powerful methods exist for solving ODEs

we will acquaint ourselves with these methods

they apply to general, first-order or higher-order odes

dy

dx= f (x , y) ,

d2y

dx2+ b(x , y)

dy

dx+ c(x , y) = 0 , etc

Ordinary differential equations

Consider again the integral I =∫ ba f (x)dx . Introduce the function

I (x) =

∫ x

af (ξ)dξ =⇒ I (a) = 0, I (b) = I .

This function obeys the ODE

dI

dx= f (x) , I (a) = 0

Our integral has been transformed into an ODE inital-value problem!

Main advantage

Powerful methods exist for solving ODEs

we will acquaint ourselves with these methods

they apply to general, first-order or higher-order odes

dy

dx= f (x , y) ,

d2y

dx2+ b(x , y)

dy

dx+ c(x , y) = 0 , etc

Ordinary differential equations

Consider again the integral I =∫ ba f (x)dx . Introduce the function

I (x) =

∫ x

af (ξ)dξ =⇒ I (a) = 0, I (b) = I .

This function obeys the ODE

dI

dx= f (x) , I (a) = 0

Our integral has been transformed into an ODE inital-value problem!

Main advantage

Powerful methods exist for solving ODEs

we will acquaint ourselves with these methods

they apply to general, first-order or higher-order odes

dy

dx= f (x , y) ,

d2y

dx2+ b(x , y)

dy

dx+ c(x , y) = 0 , etc

Ordinary differential equations

Consider again the integral I =∫ ba f (x)dx . Introduce the function

I (x) =

∫ x

af (ξ)dξ =⇒ I (a) = 0, I (b) = I .

This function obeys the ODE

dI

dx= f (x) , I (a) = 0

Our integral has been transformed into an ODE inital-value problem!

Main advantage

Powerful methods exist for solving ODEs

we will acquaint ourselves with these methods

they apply to general, first-order or higher-order odes

dy

dx= f (x , y) ,

d2y

dx2+ b(x , y)

dy

dx+ c(x , y) = 0 , etc

The Euler method

Let us try to solve the equation

dy

dx= f (x) starting from y(a) = c x ∈ [a, b]

We divide the interval into N segments with length ε = (b − a)/NNumber the points

x0 = a, x1 = a + ε, . . . , xN = b

We want to compute the values

yn = y(xn) = y(x0 + nε) ∀n = 1, . . . ,N

We know y(x0) = y0 = c .

The Euler method

Let us try to solve the equation

dy

dx= f (x) starting from y(a) = c x ∈ [a, b]

We divide the interval into N segments with length ε = (b − a)/NNumber the points

x0 = a, x1 = a + ε, . . . , xN = b

We want to compute the values

yn = y(xn) = y(x0 + nε) ∀n = 1, . . . ,N

We know y(x0) = y0 = c .

The Euler methodWe know y(x0) = y0 = c .We can calculate v0 = y ′(x0) = f (x0)

Take a small step ε:

y1 = y(x1) = y(x0 + ε) = y(x0) + εy ′(x0) = y0 + εv0

v1 = y ′(x1) = f (x1)

Iterate this to get y2, y3, . . . , yN

Algorithm

Set x0 = a, y0 = c , choose N or stepsize εCalculate v0 = f (x0).Then, for each n = 1, . . . ,N do

1 xn = xn−1 + ε

2 yn = yn−1 + εvn−1

3 vn = f (xn)

The Euler methodWe know y(x0) = y0 = c .We can calculate v0 = y ′(x0) = f (x0)

Take a small step ε:

y1 = y(x1) = y(x0 + ε) = y(x0) + εy ′(x0) = y0 + εv0

v1 = y ′(x1) = f (x1)

Iterate this to get y2, y3, . . . , yN

Algorithm

Set x0 = a, y0 = c , choose N or stepsize εCalculate v0 = f (x0).Then, for each n = 1, . . . ,N do

1 xn = xn−1 + ε

2 yn = yn−1 + εvn−1

3 vn = f (xn)

The Euler methodWe know y(x0) = y0 = c .We can calculate v0 = y ′(x0) = f (x0)

Take a small step ε:

y1 = y(x1) = y(x0 + ε) = y(x0) + εy ′(x0) = y0 + εv0

v1 = y ′(x1) = f (x1)

Iterate this to get y2, y3, . . . , yN

Algorithm

Set x0 = a, y0 = c , choose N or stepsize εCalculate v0 = f (x0).Then, for each n = 1, . . . ,N do

1 xn = xn−1 + ε

2 yn = yn−1 + εvn−1

3 vn = f (xn)

Error estimateTaylor expand as before:

y true(xn) = y(xn−1) + εy ′(xn−1) +ε2

2y ′′(xn−1) + . . .

= yEn +O(ε2)

(1)

Each step induces an error of O(ε2).The number of steps is N = (b − a)/ε [or n = (xn − a)/ε]

Real error

y true − yE ∼ N · O(ε2) ∼ O(ε)

This analysis holds in exactly the same way for a general ODE

dy

dx= f (x , y)

The Euler method is accurate up to O(ε)

Error estimateTaylor expand as before:

y true(xn) = y(xn−1) + εy ′(xn−1) +ε2

2y ′′(xn−1) + . . .

= yEn +O(ε2)

(1)

Each step induces an error of O(ε2).The number of steps is N = (b − a)/ε [or n = (xn − a)/ε]

Real error

y true − yE ∼ N · O(ε2) ∼ O(ε)

This analysis holds in exactly the same way for a general ODE

dy

dx= f (x , y)

The Euler method is accurate up to O(ε)

Error estimateTaylor expand as before:

y true(xn) = y(xn−1) + εy ′(xn−1) +ε2

2y ′′(xn−1) + . . .

= yEn +O(ε2)

(1)

Each step induces an error of O(ε2).The number of steps is N = (b − a)/ε [or n = (xn − a)/ε]

Real error

y true − yE ∼ N · O(ε2) ∼ O(ε)

This analysis holds in exactly the same way for a general ODE

dy

dx= f (x , y)

The Euler method is accurate up to O(ε)

Higher order ODEs

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

How would we solve this numerically?

I. Try discretising both derivatives

Use symmetric derivatives for both:

yn+1 − 2yn + yn−1

ε2+ f (xn, yn)

yn+1 − yn−1

2ε+ g(xn, yn) = 0

=⇒ yn+1 =2yn −

(1− ε

2 f (xn, yn))yn−1 − g(xn, yn)

1 + ε2 f (xn, yn)

Higher order ODEs

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

How would we solve this numerically?

I. Try discretising both derivatives

Use symmetric derivatives for both:

yn+1 − 2yn + yn−1

ε2+ f (xn, yn)

yn+1 − yn−1

2ε+ g(xn, yn) = 0

=⇒ yn+1 =2yn −

(1− ε

2 f (xn, yn))yn−1 − g(xn, yn)

1 + ε2 f (xn, yn)

Higher order ODEs

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

How would we solve this numerically?

I. Try discretising both derivatives

Use symmetric derivatives for both:

yn+1 − 2yn + yn−1

ε2+ f (xn, yn)

yn+1 − yn−1

2ε+ g(xn, yn) = 0

=⇒ yn+1 =2yn −

(1− ε

2 f (xn, yn))yn−1 − g(xn, yn)

1 + ε2 f (xn, yn)

Need to know both yn and yn−1 to compute yn+1

Can get y1 from y0, y′0 using forward derivative, then iterate

Higher order ODEs

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

How would we solve this numerically?

I. Try discretising both derivatives

Use symmetric derivatives for both:

yn+1 − 2yn + yn−1

ε2+ f (xn, yn)

yn+1 − yn−1

2ε+ g(xn, yn) = 0

=⇒ yn+1 =2yn −

(1− ε

2 f (xn, yn))yn−1 − g(xn, yn)

1 + ε2 f (xn, yn)

Error in derivatives: O(ε2) Error in each step: ε2O(ε2) = O(ε4)

Total error: O(ε3)

Higher order ODEs

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

How would we solve this numerically?

I. Try discretising both derivatives

Use symmetric derivatives for both:

yn+1 − 2yn + yn−1

ε2+ f (xn, yn)

yn+1 − yn−1

2ε+ g(xn, yn) = 0

=⇒ yn+1 =2yn −

(1− ε

2 f (xn, yn))yn−1 − g(xn, yn)

1 + ε2 f (xn, yn)

Error in derivatives: O(ε2) Error in each step: ε2O(ε2) = O(ε4)Total error: O(ε3)

Higher order ODE

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

II. Make it into two first order ODEs

Define ‘new’ function z(x) = y ′(x)

dy

dx= z ,

dz

dx= −f (x , y)z − g(x , y)

This system can be solved using the Euler method:

yn+1 = yn + εzn

zn+1 = zn − ε[f (xn, yn)zn − g(xn, yn)

]Error in each step: O(ε2)Total error: O(ε)

Higher order ODE

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

II. Make it into two first order ODEs

Define ‘new’ function z(x) = y ′(x)

dy

dx= z ,

dz

dx= −f (x , y)z − g(x , y)

This system can be solved using the Euler method:

yn+1 = yn + εzn

zn+1 = zn − ε[f (xn, yn)zn − g(xn, yn)

]

Error in each step: O(ε2)Total error: O(ε)

Higher order ODE

Generic second order ODE:

d2y

dx2+ f (x , y)

dy

dx+ g(x , y) = 0

II. Make it into two first order ODEs

Define ‘new’ function z(x) = y ′(x)

dy

dx= z ,

dz

dx= −f (x , y)z − g(x , y)

This system can be solved using the Euler method:

yn+1 = yn + εzn

zn+1 = zn − ε[f (xn, yn)zn − g(xn, yn)

]Error in each step: O(ε2)Total error: O(ε)

Molecular dynamics

The classical equations of motion for a collection of interacting particles(eg, molecules or planets) will be

coupled second-order ODEs for positions xi (t) (Lagrange), or

coupled first-order ODEs for positions xi (t) and momenta pi (t)(Hamilton)

Solving such coupled equations numerically is molecular dynamics.

Reducing everything to first-order ODEs corresponds to the ‘Hamiltonian’approach. This has several advantages, first and foremost:

You can use the same methods for all equations

Black-box solvers (ode45 etc) require as ‘input’ a set of first-orderequations with initial values.

You may still need physical or mathematical insight to choose the mostappropriate method!

Molecular dynamics

The classical equations of motion for a collection of interacting particles(eg, molecules or planets) will be

coupled second-order ODEs for positions xi (t) (Lagrange), or

coupled first-order ODEs for positions xi (t) and momenta pi (t)(Hamilton)

Solving such coupled equations numerically is molecular dynamics.

Reducing everything to first-order ODEs corresponds to the ‘Hamiltonian’approach. This has several advantages, first and foremost:

You can use the same methods for all equations

Black-box solvers (ode45 etc) require as ‘input’ a set of first-orderequations with initial values.

You may still need physical or mathematical insight to choose the mostappropriate method!

Molecular dynamics

The classical equations of motion for a collection of interacting particles(eg, molecules or planets) will be

coupled second-order ODEs for positions xi (t) (Lagrange), or

coupled first-order ODEs for positions xi (t) and momenta pi (t)(Hamilton)

Solving such coupled equations numerically is molecular dynamics.

Reducing everything to first-order ODEs corresponds to the ‘Hamiltonian’approach. This has several advantages, first and foremost:

You can use the same methods for all equations

Black-box solvers (ode45 etc) require as ‘input’ a set of first-orderequations with initial values.

You may still need physical or mathematical insight to choose the mostappropriate method!

Summary

Trapezium, midpoint: O(δ2); Simpson: O(δ4)

Trapezium (or midpoint) easier to extend with Romberg integration

Euler method: simple iterative procedure, accurate to O(ε)

Higher-order ODEs can be reduced to coupled first-order ODEs

Summary

Trapezium, midpoint: O(δ2); Simpson: O(δ4)

Trapezium (or midpoint) easier to extend with Romberg integration

Euler method: simple iterative procedure, accurate to O(ε)

Higher-order ODEs can be reduced to coupled first-order ODEs