Post on 13-Sep-2019
Numerical Programming 2 (CSE)
Massimo Fornasier
Fakultat fur MathematikTechnische Universitat Munchen
massimo.fornasier@ma.tum.dehttp://www-m15.ma.tum.de/
Johann Radon Institute (RICAM)Osterreichische Akademie der Wissenschaften
massimo.fornasier@oeaw.ac.athttp://hdsparse.ricam.oeaw.ac.at/
Department of MathematicsTechnische Universitat Munchen
Lecture 9
Why are you here?In this course we shall provides the rudiments (the basics!) of thenumerical solution of (partial) differential equations, i.e, equationsinvolving the derivatives of a (possibly) multivariate functionu : Ω ⊂ Rd → R:
F (x1, . . . , xd , u(x),∂
∂x1u(x), . . . ,
∂
∂xdu(x),
∂2
∂x1∂x1,
∂2
∂x1∂x2, . . . ,
∂2
∂x1∂xd, . . . ) = 0.
Often one of the variables indicates the time, e.g., x1 = t, and theequation governs a phenomena which evolves in time:
∂u
∂t+ F (x , u,Du,D2u, . . . ) = 0.
Differential equations were for thefirst time formulated in the 17th
century by I. Newton (1671) and byG.W. Leibniz (1676).
Just old boring stuff?
Yes, of course!?But also NO, because the history did not stop ...
used by Euler, Maxwell, Boltzmann, Navier, Stokes, Einstein,Prandtl, Schrodinger, Pauli, Dirac,Turing, Black&Scholes ...........
I provide an important modeling tool for the physical sciences,theoretical chemistry, biology, socio-economic sciences,engineering sciences ....
I accessible to numerical simulations
For instance ...Fourier in Theorie analytique de la chaleur (1822) formulated and solvedanalytically the equation governing the heat conduction in homogenous media:
∂u
∂t(t, x)− α
„∂2u
∂x21
(t, x) +∂2u
∂x22
(t, x) +∂2u
∂x23
(t, x)
«= 0.
A similar equation is studied by Fornasier and March in 2007 to recolor ancientItalian frescoes from fragments:
See Restoration of color images by vector valued BV functions and variationalcalculus (M. Fornasier and R. March), SIAM J. Appl. Math., Vol. 68 No. 2,2007, pp. 437-460.
http://www.ricam.oeaw.ac.at/people/page/fornasier/SIAM_Fornasier_
March_067187.pdf
Illustrations of differential equations and some of their(modern) applications
Reference: P. A. Markowich, Applied Partial Differential Equations - A Visual
Approach, Springer, 2006
Slides: http:
//homepage.univie.ac.at/peter.markowich/galleries/vortrag.pdf
I Gas dynamics Boltzmann equationI Fluid/gas dynamics: Navier-Stokes/Euler EquationsI Kinetic modeling of granular flowsI Chemotaxis and formation of biological patternsI Semiconductor modelingI Free boundary problems and interfacesI Reaction-diffusion equationsI Monge-Kantorovich optimal transportationI Wave equationsI Digital image processingI Socio-Economic modeling
From partial differential equations to ordinary differentialequations
As we shall see in the course of this lecture, several partialdifferential equations (PDE) can be reduced, after discretization ofthe space variable, to a system of ordinary differential equations(ODE):
∂u
∂t+ F (t, x , u,Du,D2u, . . . ) = 0⇒ u′h(t) + Fh(u(t)) = 0,
where h is a space discretization parameter.
Hence, the first part of this course is dedicated to the numericalsolution of ODE, while the second part of the course is dedicatedto the solution of some relevant equations of elliptic (e.g., materialelasticity), parabolic (e.g., heat conduction), hyperbolic (e.g., waveevolution) types.
Program of the course in a nutshellODE:
I forward Euler method
I theta method
I Adams methods and more general multi-step methods
I Runge-Kutta methods
I stability and stiffness
I Backward differentiation formulae
I Predictor-corrector methods
PDE:
I finite difference
I numerical solution of the Poisson equation in one dimension
I Poisson equation in two dimensions: five-point method
I finite element method
I spectral method
I finite difference for parabolic equations: the heat equation
I stability issues and analysis by the Fourier method
I finite difference for the advection equation
I Lax-Wendroff method for hyperbolic equations
I hyperbolic systems
References for this course
Besides the slides provided after the lecture online ...Books:
I09 A. Iserles, A First Course in the Numerical Analysis of DifferentialEquations (2nd ed.), Cambridge University Press, 2009.
QSG10 A. Quarteroni, F. Saleri, P. Gervasio, Scientific Computing with Matlaband Octave (3rd ed.), Springer, 2010.
Lecture notes:
F13 M. Fornasier, Numerik der gewohnlichen Differentialgleichungen,http://www-m3.ma.tum.de/foswiki/pub/M3/NumerikDG13/WebHome/
Numerik_2013-08-04.pdf (password required)
QSG10 C. Lasser, Numerical Programming 2 (CSE), synthesis of class lectures,http://www-m3.ma.tum.de/foswiki/pub/M3/Allgemeines/
NumericalCSE12/CSE_SS12.pdf
All you find at the webpage of the course:
http://www-m15.ma.tum.de/Allgemeines/NumericalProgramming
References do not mean ...
that you do not come to the lecture and think to be smart ...
’cause I will NOT let you escape ...
At the end there will be the exam ... and I’ll be waiting for youthere (REMEMBER IT!) ...you better show up! (just a friendly - Italian - invitation not toskip the lectures :-) !)
Let’s start ... why ODE?
As just mentioned, ODE comes often after the space discretizationof PDE.
But ODE have their own dignity and relevance, since, actually,PDE can be derived as the “limit” of sytems of ODE governing theevolution of particles driven by Newton laws:
F = m · a.
Hence, ODE ⇒ PDE ⇒ ODE.
Particle systemsBesides in physics, large particle systems arise in many modernapplications:
Image halftoning via variational
dithering.
Dynamical data analysis: R. palustris
protein-protein interaction network.
Large Facebook “friendship” network
Computational chemistry: molecule
simulation.
Social dynamics
We consider large particle systems ofform:
xi = vi ,
vi =∑N
j=1 H(xj − xi , vj − vi ),
Several“social forces” encoded in theinteraction kernel H:
I Repulsion-attraction
I Alignment
I ...
Possible noise/uncertainty by addingstochastic terms.
Patterns related to different balance of
social forces.
Understanding how superposition of re-iterated binary “socialforces” yields global self-organization.
An example inspired by nature
Mills in nature and in our simulations.
J. A. Carrillo, M. Fornasier, G. Toscani, and F. Vecil, Particle, kinetic,
hydrodynamic models of swarming, within the book “Mathematical modeling
of collective behavior in socio-economic and life-sciences”, Birkhauser (Eds.
Lorenzo Pareschi, Giovanni Naldi, and Giuseppe Toscani), 2010.
http://www.ricam.oeaw.ac.at/people/page/fornasier/bookfinal.pdf
The genearal ODEOur goal is to approximate the solution of the problem
y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)
(In the previous modeling, yi = (xi , vi ) andf (t, y)i = (vi ,
∑Nj=1 H(xj − xi , vj − vi )).)
I f is a map of [t0,∞)× Rd to Rd and the initial conditiony0 ∈ Rd is a given vector
I f is “nice”, obeying, in a given vector norm, the Lipschitzcondition
‖f (t, x)− f (t, y)‖ ≤ λ‖x − y‖, ∀x , y ∈ Rd , t ≥ t0 (2)
Here λ > 0 is a real constant that is independent of thechoice of x and y . In this case we say that f is a Lipschitzcontinuous function.
I Subject to (2), it is possible to prove that the ODE system(1) possesses a unique solution (see, for instance [Theorem3.5, F13] pag. 52).
Just an idea of the proof of existence and uniqueness(Picard 1890, Lindelof 1894)
Actually (1) can be rewritten by integration
y(t) = y0 +
∫ t
t0
f (s, y(s))ds = F (y(t)). (3)
Hence, a solution y(t) is a fixed point trajectory of the equationy = F (y). How can one solve fixed point equations? Well, if Fwere a contraction, i.e., F is a Lipschitz continuous function withLipschitz constant 0 < Λ < 1, then the iteration
yn+1 = F (yn), n ≥ 0 (4)
converges always to the unique fixed point! Indeed, assume thatsuch fixed point y exists then
‖y−yn+1‖∗ = ‖F (y)−F (yn)‖∗ ≤ Λ‖y−yn‖∗ ≤ Λn‖y−y 0‖∗ → 0, n→∞,
because Λ < 1. The tricky part of the proof is to show that thereexists a fixed point and that there exists always a norm ‖ · ‖∗ whichmakes F a contraction as soon as f is Lipschitz continuous.
Did you noticed that ...... what we wrote in (4) is actually an ALGORITHM?!
OMG! Really, an ALGORITHM?!
... is it a bad thing?!Actually, no, that’s what you are here for! Or, perhaps yes ... Ohwell, nevertheless, an ALGORITHM!
For the brave student: if you find out a way to discretize (4) and a way of
properly “scaling” the equation so that the iteration always converges on a
finite set of time point t0 < t1 < · < tn, then let me know! Actually this course
is (IMPLICITLY) all about this!
From Picard-Lindelof back to EulerInstead of solving globally the fixed point equation (3) by amultitude of iterations (4), we may consider the simpler idea ofsolving it locally, step by step, by iterating the approximation
y(t) = y0+
∫ t
t0
f (s, y(s))ds ≈ y(t0)+(t−t0)f (t0, y(t0)), for t ≈ t0.
(5)Given a sequence t0, t1 = t0 + h, t2 = t0 + 2h, ... , where h > 0 isa time step, we denote by yn a numerical estimate of the exactsolution y(tn), n = 0, 1, . . . . Motivated by (5), we choose
y1 = y0 + hf (t0, y0).
If h is small, it should not be that wrong! But then, why not tocontinue, assuming that we did not that bad before, at t2, t3 andso on. In general, we obtain the recursive scheme
yn+1 = yn + hf (tn, yn), (6)
the celebrated Euler method.
Graphic interpretationConsider the Euler method applied to the logistic equation
y ′ = y(1− y), y(0) =1
10,
with step h = 1:
It’s clear that at each step we produce an error, but our goal is not to avoid
any (numerical error)! Our final goal is to have a practical method that
approximates the analytic solution with increasing accuracy (i.e., decreasing
error) the more computational effort we do.
Convergence of a numerical method
Assume that h > 0 is variable and h→ 0. On each grid t0,t1 = t0 + h, t2 = t0 + 2h, ... we associate a different numericalsequence yn = yn,h, n = 0, 1, . . . , bt∗/hc (not necessarily producedby the Euler method!).
A method is said to be convergent if, for every ODE (1) with aLipschitz function f and every t∗ > 0 it is true that
limh→0
maxn=0,1,...,bt∗/hc
‖yn,h − y(tn)‖ = 0,
where bαc ∈ Z is the integer part of α ∈ R.
Convergence means that, for every Lipschitz function,the numerical solution tends to the true solution as thegrid becomes increasingly fine.
Convergence of the Euler method
TheoremThe Euler method (6) is convergent
Proof. (For this proof we assume that the Taylor expansion of f has uniformlybounded coefficients, i.e., f is analytic, implying that y is analytic as well.) Letus consider the error en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.Taylor expansion of y(t)
y(tn+1) = y(tn) + hy ′(tn) +O(h2) = y(tn) + hf (tn, y(tn)) +O(h2).
Subtracting this equation to (6), we obtain
en+1,h = en,h + h[f (tn, yn)− f (tn, y(tn))] +O(h2).
Triangle inequality and (2) imply
‖en+1,h‖ ≤ ‖en,h‖+ h‖f (tn, yn)− f (tn, y(tn))‖+ ch2
≤ (1 + hλ)‖en,h‖+ ch2.
By induction over this estimate we get
‖en,h‖ ≤c
λh[(1 + hλ)n − 1], n = 0, 1, . . .
Convergence of the Euler method continues ...
We notice now that 1 + ξ ≤ eξ for all ξ > 0, hence (1 + hλ) ≤ ehλ and(1 + hλ)n ≤ ehnλ. But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying
(1 + hλ)n ≤ et∗λ
and‖en,h‖ ≤
h c
λ(et∗λ − 1)
i· h. (7)
Sinceh
cλ
(et∗λ − 1)i
is independent of n and h, it follows
limh→0
max0≤nh≤t∗
‖en,h‖ = 0.
The error bound in (7) tells us that actually Euler’s methodconverges with order q = 1 since it decays as O(hq). However,letus stress that the constant
[cλ(et∗λ − 1)
]is by far over-pessimistic
and should not be used for numerical puroposes (it’s justtheoretical!).
Example
Let’s consider
y ′ = −100y , y(0) = 1, t∗ = 1.
The error estimate (7) would give us the following bound on theerror
|en,h| ≤ 2.69× 1045h,
while the exact and numerical solutions are respecivelyy(t) = e−100t and yn = (1− 100h)n, giving
|yn − yn,h| = |(1− 100h)n − e−100nh|,
which is by far smaller than 2.69× 1045h!
Consistency error and orderEuler’s method can be rewritten
yn+1 − yn − hf (tn, yn) = 0.
How well this equation represents the original solution? Let’s plug the originalsolution in this equation
y(tn+1)− y(tn)− hf (tn, y(tn)) = y(tn+1)− y(tn)− hy ′(tn) = O(h2), (8)
by Taylor’s expansion error. This residual error O(h2) is called the consistencyerror. In general, given an arbitrary time-stepping method
yn+1 = Yn(f , h, y0, y1, . . . , yn),
for the ODE (1), we say that it is of consistency order p if
y(tn+1)− Yn(f , h, y(t0), y(t1), . . . , y(tn)) = O(hp+1).
for every analytic f and n = 0, 1, . . . . Hence, by (8) Euler’s method hasconsistency order 1!
Never confuse the order of convergence with the consistency order! They may
well be different! A method can have a positive consistency order but not be
convergent! (see later)
How to implement Euler’s method in MATLAB
Try out Euler’s method on the logistic equation:
f = @(t,x) x.*(1-x); [t,yy]=ode45(f,[0:5],1/10); y(1)=1/10;h=1; for n=1:5, y(n+1)=y(n)+h*y(n)*(1-y(n)); end,plot([0:5],y,’r*-’,[0:5],yy,’b’) h=1/2; for n=1:10,y(n+1)=y(n)+h*y(n)*(1-y(n)), end, hold on,plot([0:0.5:5],y,’g-*’)
How does Euler’s method discretize a PDE?
Given∂u
∂t− F (t, x , u,Du,D2u, . . . ) = 0
one considers
un+1i − un
i
∆t= Fi (tn, x , u,Du,D2u, . . . ).
Trapezoidal rule alias Crank-Nicolson method
Euler’s method approximates the derivative by a constant in[tn, tn+1], namely by its value at tn. Clearly, the “leftist”approximation is not very good (it’s NOT a political comment!Left can be good :-) )and it makes more sense to make the constant approximation ofthe derivative equal to the average of its values at the endpoints:
y(t) = y0+
∫ t
t0
f (s, y(s))ds ≈ y(t0)+1
2(t−t0)[f (t0, y(t0))+f (t, y(t))],
(9)for t ≈ t0. This brings us to consider the trapezoidal rule or theCrank-Nicolson method
yn+1 = yn +h
2[f (tn, yn) + f (tn+1, yn+1)]. (10)
Consistency order of the Crank-Nicolson method
To obtain the order of (10), we substitute the exact solution,
y(tn+1)− y(tn)− h
2[f (tn, y(tn)) + f (tn+1, y(tn+1))]
= [y(tn) + hy ′(tn) +h2
2y ′′(tn) +O(h3)]
− (y(tn) +h
2y ′(tn) + [y ′(tn) + hy ′′(tn) +O(h2)]) = O(h3).
Therefore the trapezoidal rule is of order 2. This is very good, butwe should be careful (as we shall see later) already to concludethat the method is convergent. Actually we need to prove it.
Convergence of the Crank-Nicolson method
TheoremThe Crank-Nicolson method (10) is convergent.
Proof. (For this proof we assume again that f is analytic.) Let us consider theerror en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0. Substracting
y(tn+1) = y(tn) +h
2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).
(see for instance [Section 2.2.2, F13] pag. 21 for this asymptotic error) to (10),we obtain
‖en+1,h‖ ≤ ‖en,h‖+h
2[‖f (tn, yn)− f (tn, y(tn))‖+ ‖f (tn+1, yn+1)− f (tn+, y(tn+1))‖]
+ ch3
≤ ‖en,h‖+hλ
2[‖en,h‖+ ‖en+1,h‖] + ch3.
By assumption that h > 0 gets so small that hλ < 2, we get
‖en+1,h‖ ≤
1 + 1
2hλ
1− 12hλ
!‖en,h‖+
c
1− 12hλ
!h3.
Convergence of the Crank-Nicolson method continues ...
By induction over this estimate we get
‖en,h‖ ≤c
λh2
" 1 + 1
2hλ
1− 12hλ
!n
− 1
#, n = 0, 1, . . .
Since 0 < hλ < 2, it is true that 1 + 1
2hλ
1− 12hλ
!= 1 +
hλ
1− 12hλ
!≤∞X`=0
1
`!
hλ
1− 12hλ
!`= exp
hλ
1− 12hλ
!.
But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying
‖en,h‖ ≤
"c
λexp
t∗λ
1− 12hλ
!#· h2. (11)
It followslimh→0
max0≤nh≤t∗
‖en,h‖ = 0.
The error bound in (11) tells us that actually the methodconverges with order q = 2 since it decays as O(hq).
But not so easy ...Notice that the Crank-Nicolson method
yn+1 = yn +h
2[f (tn, yn) + f (tn+1, yn+1)]︸ ︷︷ ︸
:=Fn(yn+1)
.
does not allow for the explicit/direct computation of yn+1 from yn
as it occurs for (6). Indeed it is a so-called implicit method and itrequires the solution of the nonlinear equation (10) in the knownyn+1. How can we do it? Well, for instance, we observe thatactaully yn+1 = Fn(yn+1) is a fixed point of an equation and thatFn is a contraction for h sufficiently small! Therefore we cancompute
yn+1 = limm→∞
ymn+1,
where the sequence (ymn+1)m is generated (similarly to the
Picard-Lindelof iteration!!) by
ym+1n+1 = Fn(ym
n+1), m ≥ 0.
Newton ... again
The solution of the equation
0 = G (yn+1) = yn+1 − Fn(yn+1)
can also be solved by Newton’ method:
ym+1n+1 = ym
n+1 −∇G (ymn+1)−1 · G (ym
n+1), m = 0, 1, . . .
where ∇G (ymn+1) is the Jacobian matrix of G computed at ym
n+1.Again we can compute
yn+1 = limm→∞
ymn+1,
How to implement the Crank-Nicolson method inMATLAB?
Try this out:
f = @(t,x) -x + 2*exp(-t)*cos(2*t); t(1)=0; x(1)=0;y(1)=0; h=1/2; for n=1:(10/h), y(n+1)=fsolve(@(x)x-y(n)-h/2*(f(t(n),y(n))+f(t(n+1),x)),y(n)); end,semilogy(t,abs(y-xx),’k’,t, h2*ones(size(t)),’r:’)
How does the Crank-Nicolson method discretize a PDE?
Given∂u
∂t− F (t, x , u,Du,D2u, . . . ) = 0
one considers
un+1i − un
i
∆t=
1
2[Fi (tn, x , u,Du,D2u, . . . )+Fi (tn+1, x , u,Du,D2u, . . . )]
Example
Euler’s method and the trapezoidal rule, as applied to y ′ = −y + 2e−tcos2t,
y(0) = 0. The logarithm of the error, ln(|yn − y(tn)|), is displayed for h = 1/2
(solid line), h = 10 (broken line) and h = 50 (broken-and-dotted line).
Logarithm of the error
If ‖e‖ ≈ chp, then ln e ≈ ln c + p ln h. Denoting by e(1) and e(2)
the errors corresponding to step sizes h(1) and h(2) respectively, itfollows that
ln e(2) ≈ ln e(1) + p ln(h(2)/h(1)).
The ratio of consecutive step sizes being five, we expect the errorto decay by (at least) a constant multiple of ln 5 ≈ 1.6094 and2 ln 5 ≈ 3.2189 for Euler and the trapezoidal rule respectively.
Midpoint rule
To derive the the average
y ′(t) ≈ 1
2[f (tn, yn) + f (tn+1, yn+1)]
is a sensible choice. Similar reasoning leads, however, to analternative approximation,
y ′(t) ≈ f (tn +1
2h,
1
2(yn + yn+1)), t ∈ [tn, tn+1]
and to the implicit midpoint rule
yn+1 = yn + hf (tn +1
2h,
1
2(yn + yn+1))
Exercise: prove that midpoint rule is of second order and that itconverges! The implicit rule is a special case of the Runge-Kuttamethod (more to come).
The theta-method
Both Euler’s method and the trapezoidal rule fit the generalpattern
yn+1 = yn + h[θf (tn, yn) + (1− θ)f (tn+1, yn+1)], (12)
for θ = 1 and θ = 1/2 respectively.
Consistency order
Substituting the exact solution y(t) into the formula
y(tn+1)− y(tn)− h[θf (tn, y(tn)) + (1− θ)f (tn+1, y(tn+1))]
= y(tn+1)− y(tn)− h[θy ′(tn) + (1− θ)y ′(tn+1)]
= [y(tn) + hy ′(tn) +1
2h2y ′′(tn) +
1
6h3y ′′′(tn)]
− y(tn)− hθy ′(tn) + (1− θ)[y ′(tn) + hy ′′(tn) +1
2h2y ′′′(tn)]
+O(h4)
= (θ − 1
2)h2y ′′(tn) + (
1
2θ − 1
3)h3y ′′′(tn) +O(h4).
Therefore the method is of order 2 for θ = 1/2 (the trapezoidalrule) and otherwise of order one.
Error estimates and convergence
Subtracting (as usual!) the last expression from
yn+1 − yn + h[θf (tn, yn) + (1− θ)f (tn+1, yn+1)] = 0,
one obtains
en+1 = en + θh[f (tn, y(tn) + en)− f (tn, y(tn))]
+ (1− θ)h[f (tn+1, y(tn+1) + en+1)f (tn+1, y(tn+1))]− 1
12 h3y ′′′(tn) +O(h4), θ = 12 ,
+(θ − 12 )h2y ′′(tn) +O(h3), θ 6= 1
2 .
By using the Lipschitz continuity of f and proceeding as alreadydone so far, one can show that the θ-methods are alwaysconvergent, for all θ ∈ [0, 1]. Exercise!
An important byproduct: implicit Euler’s method
The choice θ = 0 is of great practical relevance. The first-orderimplicit method
yn+1 = yn + hf (tn+1, yn+1), n = 0, 1, . . . (13)
is called the backward Euler’s method (or implicit Euler’s method)and is a favourite algorithm for the solution of stiff ODEs (more tocome).
Adams method: introduction
Let us suppose again that yn is the numerical solution attn = t0 + nh. Assume that
ym = y(tm) + O(hs+1), m = 0, 1, . . . , n + s − 1, (14)
where s ≥ 1 is a given integer. Let us specify now how can weadvance the solution from tn−s+1 to tn+s . First write
y(tn+s) = y(tn+s−1)+
∫ tn+s
tn+s−1
y ′(τ)dτ = y(tn+s−1)+
∫ tn+s
tn+s−1
f (τ, y(τ))dτ .
We shall compute the integral by extrapolation, constructing theinterpolation polynomial of f on the nodes f (t`, y(t`)) for` = n, n + 1, n + s − 1 and then compute the integral of thispolynomial between tn+s−1 and tn+s .
Interpolation polynomial
Explicitly,
p(t) =s−1∑m=0
pm(t)f (tn+m, yn+m),
where
pm(t) =s−1∏
`=0,` 6=m
t − tn+`
tn+m − tn+`=
(−1)s−1−m
m!(s − 1−m)!
s−1∏`=0,` 6=m
(t − tn
h− `)
=(−1)s−1−m
m!(s − 1−m)!
s−1∏`=0,`6=m
(t − t0 − hn
h− `)
and the Lagrange interpolation polynomials (see [Section 1.1, F13]pag. 8).
Construction of the scheme
Exercise: show that (14) implies
p(tm) = f (tm, ym) = y ′(tm)+O(hs+1), m = n, n+1, . . . , n+s−1.
(use the Lipschitz continuity of f ). Hence, p interpolates y ′ up toan error of order at least O(hs). By using the theory of [Theorem1.2, F13] we can deduce that
p(t) = y ′(t) +O(hs) = f (t, y(t)) +O(hs), ∀t ∈ [n + s−1, n + s].
Hence if we substitute p(t) to f (t, y(t)) in∫ tn+s
tn+s−1f (τ, y(τ))dτ
and yn+s−1 to y(tn+s−1), we produce and error of order O(hs+1)and we obtain the scheme
yn+s = yn+s−1 + hn+s−1∑m=n
bmf (tn+m, yn+m), (15)
which is, by construction, of consistency order p = s.
On the coefficients of the s-step Adams-Bashforth method
Notice that
bm = h−1
∫ tn+s
tn+s−1
pm(τ)dτ =(−1)s−1−m
m!(s − 1−m)!
∫ 1
0
s−1∏`=0,`6=m
[s−(`+1)−ξ]dξ
do not depend on n or h and can be used for subsequent iterations.
Examples
For s = 1 this scheme coincides with the Euler’s method, whereass = 2 gives explicitly
yn+2 = yn+1 + h[3
2f (tn+1, yn+1)− 1
2f (tn, yn)],
and s = 3 gives
yn+3 = yn+2 + h[23
12f (tn+2, yn+2)− 4
3f (tn+1, yn+1) +
5
12f (tn, yn)],
ComparisonLogarithm of the errors of Euler, 2-step, and 3-step Adams-Basgforth methodsfor the equation
y ′ = y 2, y(0) = 1.
When h is halved, Euler’s error decreases linearly, the error of 2-step AB decays
quadratically and 3-step AB displays cubic decay.
Order of convergence of multistep methods
Write a general s-step method in the form
s∑m=0
amyn+m = hs∑
m=0
bmf (tn+m, yn+m), n = 0, 1, . . . (16)
where am, bm, m = 0, 1, . . . , s, are given constants, independent ofh, n and the underlying equation. It is conventional to normalizeas = 1. When bs = 0 the method is explicit (as in theAdams-Bashford methods), otherwise again implicit. Adapting thedefition of consistency order to these methods we say that (16) hasorder p if
ψ(t, y) =s∑
m=0
amy(t+mh)−hs∑
m=0
bmf (t+mh, y(t+mh)) = O(hp+1),
for h→ 0, for all sufficiently smooth y and there is one functionwhich do not allow to improve this order.
Characterization of the order
The method (16) can be characterized in terms of the polynomials
ρ(w) :=s∑
m=0
amwm, σ(w) :=m∑
s=0
bmwm.
TheoremThe multistep method is of order p ≥ 1 if and only if there existsc 6= 0 such that
ρ(w)− σ(w) ln w = c(w − 1)p+1 +O(|w − 1|p+2), w → 1.
ProofWe assume that y is analytic and that its radius of convergence exceeds sh.Taylor + exchanging sums:
ψ(t, y) =sX
m=0
am
∞Xk=0
y (k)(t)
k!mkhk − h
sXm=0
bm
∞Xk=0
y (k+1)(t)
k!mkhk
=
sX
m=0
am
!y(t) +
∞Xk=1
1
k!
sX
m=0
mkam − ksX
m=0
mk−1bm
!hky (k)(t).
Thus for having order p it is necessary and sufficient that
sXm=0
am = 0,sX
m=0
mkam = ksX
m=0
mk−1bm, k = 1, . . . , p,
sXm=0
mp+1am 6= (p + 1)sX
m=0
mpbm.
Let w = ez ; then w → 1 corresponds to z → 0. Taylor + exchanging sumsagain gives
ρ(z)− zσ(z) =∞X
k=0
1
k!
sX
m=0
mkam
!zk −
∞Xk=1
1
(k − 1)!
sX
m=0
mk−1bm
!zk .
Proof continues ..
Hence,ρ(z)− zσ(z) = czp+1 +O(zp+2)
for some c 6= 0 iff the conditions written above hold. The result follows by
restoring w = ez (remember that z = ln w = (w − 1) +O(|w − 1|2) for
w → 1).
An application to the 2-step Adams-Bashforth
Recall
yn+2 = yn+1 + h[3
2f (tn+1, yn+1)− 1
2f (tn, yn)].
An explicit computation shows that for ξ = w − 1
ρ(w)−σ(w) ln(w) = (ξ+ξ2)−(1+3
2ξ)(−1
2ξ2+
1
3ξ+. . . ) =
5
12ξ3+O(ξ4),
thus order 2 is validated. Exercise: try the same computation forthe 3-step Adams-Bashforth method.
Yeah! Happiness, joy!
We have a method for constructing consistent methods of arbitraryorder!
Bad news ... this sometimes does not help!
Consider the two-step implicit scheme
yn+2−3yn+1+2yn = h[13
12f (tn+2, yn+2)−5
3f (tn+1, yn+1)− 3
12f (tn, yn)].
Exercise: show that it has consistency order 2.Now consider the ODE y ′ = 0 with y(0) = 1 with unique solutiony ≡ 1. A single step reads
yn+2 − 3yn+1 + 2yn = 0,
whose general solution is given by yn = c1 + c22n, n = 0, 1, ....Suppose that c2 6= 0 and we need y0, y1 to initiate the iteration.The assumption c2 6= 0 is equivalent to y0 6= y1. Hence, in thiscase the method does NOT converge at all: in fact let t > 0 andh→ 0 and n→∞ so that hn→ t. Obviously n→∞ and|yn| → ∞, which is far from y(t) ≡ 1, ooops :-)
One smart student may say ...
... Ok but you have assumed a “stupid” choice of y0 6= y1. Wereyou choosing them equal, you would have ended up on the rightsolution, don’t you? Well, not really ... the motivation is thatalthough y0 = y1 in the first iteration, then after a while, due toroundoff errors yn 6= yn+1 and you are again dead in the water!
Touch to believe ...
Breakdown in the numerical solution of y ′ = −y , y(0) = 1, by anonconvergent numerical scheme, showing how the situationworsens with decreasing step size.
Dahlquist equivalence theoremWe say that a polynomial obeys the root condition if all its zerosreside in the closed complex unit disc and all its zeros of unitmodulus are simple.
Theorem (The Dahlquist equivalence theorem)
Suppose that the error in the starting values y1, y2, . . . ys1 tends tozero as h→ 0. The multistep method (16) is convergent if andonly if it is of order p ≥ 1 and the polynomial ρ obeys the rootcondition.
Touch to believe ... check that roots of
ρ(w) = w 2 − 3w + 2 = (w − 1)(w − 2)
are indeed w = 1, 2 and this explains the non-convergence of thecorresponding method. However, all Adam-Bashforth methods aresafe, because for them
ρ(w) = w s−1(w − 1).
Exercise!
Dahlquist first barrier
The multistep method (16) has 2s + 1 parameters and thanks tothe consistence order theorem we can tune them in order tooptimize the order (i.e., we can choose them to make it thelargest). The outcome, an (implicit) s-step method of order 2s, isunfortunately not convergent for s ≥ 3 (we have already seen thecase s = 3!). In general, it is possible to prove that the maximalorder of a convergent s-step method is at most 2b(s + 2)/2c forimplicit schemes and just s for explicit ones; this is known as theDahlquist first barrier.
Good construction
Choose an arbitrary s-degree polynomial ρ that obeys the rootcondition (how to do that?) and such that ρ(1) = 0 (how to dothat? Why do we need it?). Dividing the order condition by ln(w)
we obtain σ(w) = ρ(w)ln(w) +O(|w − 1|p). Suppose first p = s + 1
and expand the fraction in Taylor series around w = 1 and let σ bethe polynomial coinciding with the Taylor expansion up to orderO(|w − 1|s+1). The outcome is a convergent (!!!), s-step methodof order s + 1. Similarly , to obtain an explicit method of order s,we let σ be an (s − 1)th degree polynomial (to force bs+1 = 0)that matches the series to order O(|w − 1|s).
An example and Adams-Moulton methods
Try, for example, s = 2 and ρ(w) = w 2 − w . The solution isσ(w) = − 1
12 + 23 w + 5
12 w 2. for the implicit case, whereas for theexplicit case, where σ is linear we have p = 2, and so recover,unsurprisingly, the Adams-Bashforth scheme.The choice ρ(w) = w s−1(w − 1) is associated with Adamsmethods. We have already seen the explicit Adams-Bashforthschemes; their implicit counterparts are called Adams-Moultonmethods.
Explicit Runge-Kutta methods
Consider
y(tn+1) = y(tn) +
∫ tn+1
tn
f (s, y(s))ds
= y(tn) + h
∫ 1
0f (tn + hs, y(tn + hs))ds,
and replace the second integral by a (Gaussian) quadrature. Theoutcome might have been the method
yn+1 = yn + hν∑
j=1
bj f (tn + cjh, y(tn + cjh)), n = 0, 1, . . . ,
except that we do not know the value of y at the nodes tn + c1h,tn + c2h, . . . , tn + cνh. We must resort to an (additional)approximation!
Construction
We denote our approximation of y(tn + cjh) by ξj , j = 1, 2, . . . , ν.To start with, we let c1 = 0, since then the approximation isalready provided by the former step of the numerical method,ξ1 = yn. The idea is to construct the ξj ’s updating yn by a linearcombination of f (tn + c`h, ξ`) for ` < j . Specifically, we let
ξ1 = yn
ξ2 = yn + ha21f (tn, ξ1)
ξ3 = yn + ha31f (tn, ξ1) + ha32f (tn + c2h, ξ2)
. . . = . . .
ξν = yn + hν−1∑`=1
aν`f (tn + c`h, ξ`)
yn+1 = yn + hν∑`=1
bj f (tn, ξ`)
Butcher tableau
The matrix A = (aij)i ,j=1,...,ν , with zero entries where cofficientsare not used in the definition of the scheme, is called the RKmatrix, while the vectors b = (b1, . . . bν)T and c = (c1, . . . , cν)T
are the RK-weights and RK nodes respectively. We say that themethod has ν stages. A Runge-Kutta scheme is represented oftenby means of the so-called Butcher tableau
c | A−−− + −−−
| bT(17)
How to construct an explicit Runge-Kutta method?
Let us start assuming that
j−1∑i=1
aji = cj , j = 2, 3, , ν.
This is necessary and sufficient condition for a consistent methodto be invariant under autonomization (see [Lemma 3.19, F13]).The simplest device to derive RK methods (only valid for p ≥ 3!)consists of verifying the order for the scalar autonomous equation
y ′ = f (y), t ≥ t0, y(0) = y0.
How to construct an explicit Runge-Kutta method?
Let us start observing that Let show how to do it for ν = 3 only.We write below f ∼ f (tn) and y ∼ y(tn) unless differentlyspecified.
ξ1 = y
⇒ f (ξ1) = f
ξ2 = y + hc2f
⇒ f (ξ2) = f (y + hc2f ) = f + hc2fy f +1
2h2c2
2 fyy f 2 +O(h3)
ξ3 = y + h(c3 − a32)f (ξ1) + ha32f (ξ2)
= y + hc3f + h2a32c2fy f +O(h3)
⇒ f (ξ3) = f (y + hc3f + h2a32c2fy f ) +O(h3)
= f + hc3fy f + h2(1
2c2
3 fyy f 2 + a32c2f 2y f ) +O(h3).
How to construct an explicit Runge-Kutta method?
Therefore
yn+1 = y + hb1f + hb2(f + hc2fy f +1
2h2c2
2 fyy f 2)
+hb3[f + hc3fy f + h2(1
2c2
3 fyy f 2 + a32c2f 2y f )] +O(h4)
= yn + h(b1 + b2 + b3)f + h2(c2b2 + c3b3)fy f
+h3[1
2(b2c2
2 + b3c33 )fyy f 2 + b3a32c2f 2
y f ] +O(h4).
How to construct an explicit Runge-Kutta method?Since for a high-order consistent method we need
y ′(tn) ≈ f , y ′′(tn) ≈ ffy , y ′′′(tn) ≈ fyy f 2 + f 2y f ,
we have
y(tn+1) ≈ y + hf +1
2h2ffy +
1
6h3(fyy f 2 + f 2
y f ) +O(h4).
and comparing the terms of the exapansion up to the order threewith
yn+1 = yn + h(b1 + b2 + b3)f + h2(c2b2 + c3b3)fy f
+h3[1
2(b2c2
2 + b3c33 )fyy f 2 + b3a32c2f 2
y f ] +O(h4).
we obtain
b1+b2+b3 = 1, b2c2+b3c3 =1
2, b2c2
2 +b3c23 =
1
3, b3a32c2 =
1
6.
(18)
Classical RK methods
Third order three-stage explicit RK method:
0 |12 | 1
21 | −1 2
−−− + −−− −−− −−−| 1
623
16
(19)
and the so-called Nystrom scheme
0 |23 | 2
323 | 0 2
3−−− + −−− −−− −−−
| 14
38
38
(20)
Classical RK methods
The fourth order method is not beyond the capability of the Taylorexpansion technique presented, but it is significantly harder to besolved. The best-known fourth-order four-stage RK method is
0 |12 | 1
212 | 0 1
21 | 0 0 1
−−− + −−− −−− −−− −−−−| 1
613
13
16
(21)
To obtain order 5 we need six stages, and matters becomeconsiderably worse for higher orders.
Implicit RK methods
The idea behind implicit RungeKutta (IRK) methods is to allowthe vector functions ξ1, ξ2, . . . , ξν to depend upon each other in amore general manner. Thus, let us consider the scheme
ξj = yn + hν∑
i=1
aji f (tn + cih, ξi ), j = 1, 2, . . . , ν,
yn+1 = yn + hν∑
i=1
bj f (tn + cjh, ξj).
Here A = (aji )j ,i=1,...,ν is an arbitrary matrix (while before it wasstrictly lower triangular). We impose the convention
ν∑i=1
aji = cj , j = 1, 2, 3, , ν.
Disadvantage and advantages
The progressing of an implicit RK scheme one needs to solve ateach time a system of ν equations in the unknown ξj ’s to be ableto implement a step. This is clearly higher complexity, but IRKpossess important advantages; in particular they may exhibitsuperior stability properties (we shall see that later).
Unique solvability for IRK methods
Let us consider ki ’s so defined
ξj = y + hν∑
i=1
aji f (t + cih, ξi )︸ ︷︷ ︸:=ki
, j = 1, 2, . . . , ν.
Clearly we can define a set of equations for them
ki = f (t + cih, y + hν∑
j=1
aijkj), i = 1, 2, . . . , ν.
TheoremIf f is Lipschitz-continuous then for h > 0 small enoughk = (k1, . . . , kν), and hence ξ = (ξ1, . . . , ξν), are uniquely definedby the above system of nonlinear equations.
Unique solvability for IRK methodsSketch of a proof. Let us assume that f is globaly Lipschitzcontinuous with constant Λ. Set k = (k1, . . . , kν)T ∈ Rν×d and
Fi (k) = f (t + cih, y + hν∑
j=1
aijkj), i = 1, . . . , ν
F (k) = (F1(k), . . . ,Fν(k))T .
Let us consider the norm ‖k‖ = max1≤i≤ν ‖ki‖2. Thenk = (k1, . . . , kν)T of the Runge-Kutta step satisfies the fixed pointequation
k = F (k).
Now it holds
‖F (k)− F (k)‖ ≤ Λh max1≤i≤ν
ν∑j=1
|aij | ‖kj − k j‖2
≤ Λh‖A‖∞‖k − k‖.
For h < 1Λ‖A‖∞ the function F is a contraction with a unique fixed
point.
Larger time stepsOne of the advantages of IRK methods is their stability for largetime steps when dealing with stiff equations. The previous methodof computation is not so efficient, as it requires a barrier to h. Wecan therefore consider the following Newton iteration: Let usdefine zi = ξi − y , i = 1, . . . , ν
G (z) = z − h
∑ν
j=1 a1j f (t + cjh, y + zj)...∑ν
j=1 aνj f (t + cjh, y + zj)
and
z0 = 0
∇G (z`)∆z` = − G (z`), z`+1 = z` + ∆z`.
The drawback of the iteration is as usual that the Jacobi-Matrix∇G has to be evaluated and inverted at each step. A simplesolution is to substitute ∇G (z`) by ∇G (z0), giving the so-calledQuasi-Newton method.
Example of a two stages implicit Runge-Kutta method
Let us consider the method
ξ1 = yn +1
4h[f (tn, ξ1)f (tn +
2
3h, ξ2)]
ξ2 = yn +1
12h[3f (tn, ξ1) + 5f (tn +
2
3h, ξ2)]
yn+1 = yn +1
4h[f (tn, ξ1) + 3f (tn +
2
3h, ξ2)].
In short tableau form
0 | 14 −1
423 | 1
45
12−−− + −−− −−−
| 14
34
Try to prove that this method is of consistency order 3.
How can be construct Runge-Kutta methods? Usecollocation
Define the (unique!) polynomial u ∈ Pνu(tn) = yn
u(tn + cih) = f (tn + cih, u(tn + cih)), i = 1, . . . , ν,(22)
for 0 ≤ c1 < · · · < cν ≤ 1, the so-called relative nodes. Then onedefine the evolution
yn+1 = u(tn + h).
The evolution defined by (22) is called a collocation method
Indeed collocation methods are Runge-Kutta methods!
Let p1, . . . , pν be the Lagrangian polynomial associated to the(interpolation) nodes c1, . . . , cν (see [Section 1.1, ,F13]), so that
pj(ci ) = δij , i , j = 1, . . . , ν.
TheoremThe collocation method on the nodes c = (c1, . . . , cν) is animplicit Runge-Kutta method (b, c ,A) where
aij =
∫ ci
0pj(θ)dθ, i , j = 1, . . . , ν, (23)
and
bi =
∫ 1
0pi (θ)dθ, i = 1, . . . , ν. (24)
Recall how an implicit RK is formulated ...
Define
ki = f (tn + cih, yn + hν∑
j=1
aijkj), i = 1, 2, . . . , ν.
Then
yn+1 = yn + hν∑
i=1
biki .
ProofSet
kj = u(tn + cjh), j = 1, . . . , ν.
The Lagrange interpolation basis on c1, . . . , cν gives
u(tn + θh) =ν∑
j=1
kjpj(θ).
Integration over θ yields
u(tn + cih) = yn + h
∫ ci
0u(tn + θh)dθ
= yn + h
∫ ci
0
ν∑j=1
kjpj(θ)dθ
= yn + hν∑
j=1
[∫ ci
0pj(θ)dθ
]kj
= yn + hν∑
j=1
aijkj .
Proof continues ...
If we sustitute this expression in formula (22)
u(tn + cih)︸ ︷︷ ︸=ki
= f (tn + cih, u(tn + cih)︸ ︷︷ ︸=yn+h
Pνj=1 aijkj
),
we obtain
ki = f (tn + cih, yn + hν∑
j=1
aijkj) (25)
and analogously it follows
yn+1 = u(tn + h) = yn + h
∫ 1
0u(tn + θh)dθ
= yn + hν∑
i=1
biki .
(26)
We obtain so an implicit Runge-Kutta method.
Collocation as numerical quadrature ...
We define
y(tn + h) =
∫ tn+1
tn
f (t)dt, (27)
so thaty(t) = f (t), y(tn) = 0. (28)
The application of the collocation method will give us
y(tn+1) ≈ yn+1 = hν∑
i=1
biki
= hν∑
i=1
bi f (tn + cih),
(29)
over the nodes c1, . . . , cν , i.e., a Newton-Cotes integration formula!
Collocation as numerical quadrature ...
LemmaThe collocation method determined by the coefficients b, c, definesa quadrature
ν∑i=1
biϕ(ci ) ≈∫ 1=tn+h
0=tn
ϕ(x)dx , (30)
which is exact for polynomial of degree ν − 1.
Proof. It holds
θk−1 =νX
j=1
ck−1j pj(θ), k = 1, . . . , ν,
(since pj is a basis for Pν−1). Hence, it follows
νXj=1
bjck−1j =
νXj=1
Z 1
0
ck−1j pj(θ)dθ
=
Z 1
0
θk−1dθ =1
k, k = 1, . . . , ν,
and this means that polynomial of degree at most ν − 1 are exactly integrated.
Construction principles of collocation/IRK methods
Construction principle for ν-stages Runge-Kutta-methods:
I Choose a quadrature formula on nodes c1, . . . , cνI Construct the corresponding collocation methods by means of
bi =
∫ 1
0pi (θ)dθ, aij =
∫ ci
0pj(θ)dθ, i , j = 1, . . . , ν
Examples of collocation/IRK methods.Examples
I Gauss-methods:
I ci are the zeros of orthogonal polynomial w.r.t. the weightw ≡ 1 (Legendre polynomials, see [Sections 2.6.2 and 2.7.2,F13])
I ν-stages with consistency order p = 2νI they are A-stable (proof in the book of A. Iserles p. 63)I Example: the implicit middle point rule
yn+1 = yn + hf
(yn + yn+1
2
)I Radau-methods:
I cν = 1, c1, . . . , cν−1 zeros of the Jacobi-polynomial (see theRemark after [Section 2.6.2, F13])
I They are L-stableI Example: implicit Euler method
yn+1 = yn + hf (yn+1)
Not every implicit Runge Kutta method is a collocationmethod
Let ν = 2 and consider c1 = 0 and c2 = 23 . The corresponding two
stages collocation method build on these nodes has Butchertableau
0 | 0 023 | 1
313
−−− + −−− −−−| 1
434
But this is the unique collocation method associated to the nodesc1 = 0 and c2 = 2
3 . However, before we saw another two stagesimplicit Runge Kutta method with tableau (exercise!)
0 | 14 −1
423 | 1
45
12−−− + −−− −−−
| 14
34
We deduce that not all implicit Runge-Kutta methods can becollocation methods!
Remember the hard computation we did?
TheoremAn explicit RK method (c, b,A) has the consistency order p = 1 iff
1.P
i bi = 1 ,
the consistency order p = 2 iff additionally hold
2.P
i bi ci = 12,
the consistency order p = 3 iff additionally hold
3.P
i bi c2i = 1
3
4.P
i,j bi aijcj = 16,
and the consistency order p = 4, iff additionally hold
5.P
i bi c3i = 1
4
6.P
i,j bi ci aijcj = 18
7.P
i,j bi aijc2j = 1
12
8.P
i,j,k bi aijajkck = 124
Here the summations are between 1 and ν for ν ≥ p.
How can one solve this complicated system of nonlinearequations?!
We observe that the conditions 1,2,3, and 5 are easily recognizedas quadratue formulas
4∑i=1
biϕ(ci ) ≡∫ 1
0ϕ(t)dt,
which are exact for polynomials of degrees at most 3 on theinterval [0, 1]! We recall two important Newton-Cotes formulas,the classical Simpson formula and the so-called Newton- 3
8 -rule (see[Section 2.2.3 and Theorem 2.3, F13]).
Degree Name Rule Error
2 Simpson b−a6 (f0 + 4f1 + f2) − (b−a)5
2880 f (4)(ξ)
3 Newton- 38
b−a8 (f0 + 3f1 + 3f2 + f3) − (b−a)5
6480 f (4)(ξ)
Computing then ...
By the Newton - 38-rule we obtain the nodes c and weights b
c1 = 0, c2 =1
3, c3 =
2
3, c4 = 1
b1 =1
8, b2 =
3
8, b3 =
3
8, b4 =
1
8.
The rest of the conditions allows easily, together with the condition
j−1Xi=1
aji = cj , j = 2, 3, . . . , ν.
to fix all the values of the matrix A = (aij) (see pag. 74-75 in [F13]). Theresulting scheme
0 |13| 1
323| − 1
31
1 | 1 −1 1−−− + −−− −−− −−− −−−−
| 18
38
38
18
(31)
The classical 4 stages RK method revisited
The method (88)
0 |12 | 1
212 | 0 1
21 | 0 0 1
−−− + −−− −−− −−− −−−−| 1
613
13
16
can be computed analogously, but it is based on the Simpsonquadrature formula.
Stiff equations ... finally!
We consider the so-called Dahlquist test equation withcorresponding initial value
y = −λy , x(0) = 1, (32)
where λ > 0. The solution is explicitly given by y(t) = e−λt andmonotonically descresing to 0 for t →∞. When we apply theexplicit Euler method for this problem with time step h = 0.01, weobtain two different results, depending on λ.
What goes so wrong?!Let us consider the Euler method, defined by discrete evolution
yn+1 = yn + hf (yn),
applied to the Dahlquist test equation, to obtain
yn = (1− hλ)ny0.
In principle, we wish to expect that yn → 0 for n→∞ (since thisis the behavior of y(t) for t →∞). For this, it is necessary that
|1− hλ| < 1, and for h, λ > 0
− (1− hλ) < 1, (33)
and this holds iff
h <2
λ. (34)
But this is not satisfied for λ = 210 and h = 0.01! Hence, themethod is numerically unstable.
Implicit is betterLet us consider the implicit Euler method now:
yn+1 = yn + hf (yn+1), n = 0, 1, . . . , h > 0.
Again for the Dahlquist test equation it holds
yn+1 = xn − hλxn+1 or
yn+1 =1
1 + hλyn, n = 0, 1, . . . . (35)
We again require yn → 0 for n→∞ and we obtain the necessarycondition ∣∣∣∣ 1
1 + hλ
∣∣∣∣ < 1 (h, λ > 0)
⇐⇒ 1 < 1 + hλ (36)
⇐⇒ 0 < hλ
This is always satisfied for h, λ > 0! We conclude that the implicitmethod is always qualitatively correct, independently of the timestep!
Stiff problems ...
... are, roughly speaking, those for which explicit methods are“inefficient” for relatively large time steps.
Stability of solutions
Let us define Φt(y0) = y(t), the solution to the ODE
y ′ = f (y), y(0) = y0.
DefinitionA solution Φt(y0) is said
I stable, if for all ε > 0 there exists a δ > 0, such that
Φt(y) ∈ Bε(Φt(y0)) (37)
for all t ≥ 0 and for all y ∈ Bδ(y0).
I asymptotically stable, if additionally there exists a δ0 > 0,such that
limt→∞
∥∥Φt(y)− Φt(y0)∥∥ = 0 (38)
for all y ∈ Bδ0(y0).
I unstable, if it is not stable.
Stability of solutions
DefinitionA point y is a stationary point or a pixed point of the equationy ′ = f (y), if f (y) = 0. We call y asymptotically stable stationarypoint, if there exists δ0 > 0, such that
limt→∞
∥∥Φt(y)− y∥∥ = 0, (39)
for all y ∈ Bδ0(y).
Matrix exponential function
For A ∈ Cd×d , the function etA is defined by
exp(tA) := etA =∞∑
k=0
(tA)k
k!.
This series converges uniformly on intervals [−T ,T ]. The matrixexponential has the following properties
1. exp(tMAM−1) = M exp(tA)M−1, M ∈ GL(d);
2. exp(t(A + B)) = exp(tA) exp(tB), as soon as AB = BA;
3. For B = Block-diag(B1, . . . ,Bk)
exp(tB) = Block-diag(exp(tB1), . . . , exp(tBk)).
The proof is left to the brave students.
Linear equations
Consider the linear equation (here A is a matrix)
y = Ay . (40)
In this caseΦt(y0) = etAy0,
and this implies
Φt(y)− Φt(y0) = Φt(y − y0)
andsupt≥0
∥∥Φt(y)− Φt(y0)∥∥ ≤ ε
for ‖y − y0‖ sufficiently small, iff
supt≥0
∥∥Φt∥∥ = sup
t≥0
∥∥∥etA∥∥∥ <∞. (41)
Linear equations
We collect these observations in the following
LemmaA solution of the linear equation (40) is stable, iff (41) holds. It isasymptotically stable, iff
limt→∞
∥∥Φt∥∥ = 0.
Spectrum of matrices
Definition
I The spectrum σ(A) of a matrix A ∈ Cd×d is the set
σ(A) = λ ∈ C : det(λI − A) = 0
of all eigenvalues of A.
I The index ι(λ) of λ ∈ σ(A) is the dimension of theJordan-block of A associated to λ.
Main stability result
TheoremThe solution of y = Ay for A ∈ Cd×d is stable iff Re(λ) ≤ 0 for alleigenvalues λ of A and all eigenvalues for which Re(λ) = 0 havethe index ι(λ) = 1. The solution is aymptotically stable iffRe(λ) < 0 for all λ ∈ σ(A).
Sketch of the proofSince stability is an affine invariant we can use the 1. property of theexponential matrix, and assume A = J is already in Jordan form, i.e.,(J = MAM−1 and y = Ay implies (My) = J(My)). By property 3., we canadditionally reduce A to a single Jordan-Block J = λI + N, where
N =
0BBB@0 1 0
. . .
10 0
1CCCA| z
k
is a nilpotent matrix, i.e., Nk = 0. Then
‖exp(tA)‖ = |eλt | ‖exp(tN)‖
≤ eRe(λ)t
„1 + t ‖N‖+ · · ·+ tk−1
(k − 1)!‖N‖k−1
«| z
≤Mεeεt
,
for all ε > 0 and associated Mε > 0, from which
‖exp(tA)‖ ≤ Mεe(Re(λ)+ε)t
follows.
Sketch of the proof ... continues
Were Re(λ) < 0, then we could choose ε > 0 so small that Re(λ) + ε < 0 and‚‚‚etA‚‚‚ ≤ Mεe(Re(λ)+ε)t → 0, t →∞.
For Re(λ) = 0, k = 1, then ‖exp(tA)‖ = 1, and this means that the solution isstable. Were Re(λ) = 0 holding and k > 1, it would follow forek = (0, . . . , 0, 1)T ∈ Rk , that
exp(tJ)ek = eλt
„tk−1
(k − 1)!, . . . , t, 1
«T
,
and so ‖exp(tJ)‖ → ∞ for t →∞, and the condition ι(λ) = 1 is for all λ with
Re(λ) = 0 necessary. What happens when Re(λ) > 0 is left as an exercise.
Stablity of discrete evolutionFor one step methods the evolution is rules by a discrete fluxΨh(y) = y + hψ(y , h). For example for the explicit Euler-methodψ(y , h) = f (y) for y = f (y). Let us again consider
y = Ay ,
for which Ψh is also linear. In general:
yn+1 = Myn, n = 0, 1, 2, . . . (42)
DefinitionA linear iteration as (42) is called stable if
supn≥1‖Mn‖ <∞,
and asymptotically stable if
limn→∞
‖Mn‖ = 0. (43)
Stablity of discrete evolution
Assume we have y = Ay with continuous evolution
Φt(y) = exp(tA)y .
For one step methods, one wants Ψh(y) ' exp(hA)y . The issue isfor which h > 0 the discrete approximation is stable. For explicitmethods, in particular for ν-stages Runge-Kutta-methods, we have(exercise!)
Ψh(y) = P(hA)y ,
where P ∈ Pν . For implicit Runge-Kutta-methods (e.g., theimplicit Euler-method), it holds
Ψh(y) = R(hA)y ,
where R(z) = P(z)Q(z) is a rational function with polynomials P,Q
(where we assume that Q(λ) 6= 0 for all λ ∈ σ(A)).
Stablity of discrete evolution
TheoremThe linear iteration, given by M as in (42) is stable iff ρ(M) ≤ 1and all the eigenvalues λ for which |λ| = 1 have index ι(λ) = 1,where
ρ(M) := supλ∈σ(M)
|λ| .
Sketch of the proof. Similarly to the previous theorem: first,reduce the problem to a Jordan-block. Then it is
Jn = λnI + λn−1
(n
1
)N + · · ·+ λn−k+1
(n
k − 1
)Nk−1,
‖Jn‖ ≤ |λ|n(
1 + |λ|−1
(n
1
)‖N‖+ · · ·+ |λ|−(k−1)
(n
k − 1
)‖N‖k−1
)≤ Mθ(θ |λ|)n
for sufficiently large Mθ depending on θ > 1, so that θ |λ| < 1.The rest continues as in the proof of the previous theorem.
Summarizing
Continuous time Discrete timey = Ay yn+1 = exp(hA)yn
exp(hσ(A)) = σ(exp(hA))Reλ < 0 ∀λ ∈ σ(A) ⇐⇒ ρ (exp (hA)) < 1.
Rational functions ...
From the previous theorem, we saw that Ψh is stable iff ρ(Ψh) ≤ 1and ι(µ) = 1 for |µ| = 1, µ ∈ σ(R(hA)). Actually it holds
σ(R(A)) = R(σ(A)) (44)
for all rational functions R such that R(λ) 6=∞ for all λ ∈ σ(A).Hence, ρ(Ψh) = ρ(R(hA)) = maxλ∈σ(A) |R(hλ)| and we get
Ψhstable ⇐⇒
|R(hλ)| ≤ 1, for λ ∈ σ(A) and
ι(µ) = 1, for |µ| = 1.
Stability domains of rational functions
DefinitionThe set
S := SR := z ∈ C : |R(z)| ≤ 1 (45)
is called the stability domain of the function R.
Then we can say that Ψh is stable iff hλ ∈ SR for λ ∈ σ(A) andι(µ) = 1 for |µ| = 1 where µ = R(hλ), λ ∈ σ(A).
Examples of stability domains
For the explicit Euler method of discrete evolution
Ψh(y) = y + hf (y)
= y + hAy
= (I + hA)y = R(hA)y ,
e.g., R(z) = 1 + z . The associated stability domain is
SR = z ∈ C : |R(z)| ≤ 1= |1 + z | ≤ 1= B1(−1).
Examples of stability domains
For the implicit Euler method of discrete evolution
Ψhy = y + hA(Ψh(y))
= y + hAΨh(y),
implying Ψh(y) = (I − hA)−1y = R(hA)y , then R(z) = (1− z)−1.The associated stability domain is
SR = z ∈ C : |R(z)| ≤ 1
=
1
|1− z |≤ 1
= |1− z | ≥ 1
= (B1(1))c ,
the complement of B1(1).
Examples of explicit Runge-Kutta methodsFor consistent explicit Runge-Kutta methods of order s, we havethat
R(z) = 1 + z + · · ·+ 1
p!zs .
The corresponding stability domains (for s = 1, 2, 3, 4) are showedbelow.
Rational functions for Runge-Kutta methods
LemmaThe rational function R has for Runge-Kutta methods always theform
R(z) = 1 + zbT (I − zA)−1e, e = (1, . . . , 1)T . (46)
Moreover it is R(z) = P(z)Q(z) for unique polynomials P,Q ∈ Pν ,
P(0) = 1, Q(0) = 1.
ProofFor the Dahlquist equation y = λy , y(0) = 1, the application of RK gives
Ψh(1) = R(hλ) = 1 + hνX
j=1
bjλξj ,
ξi = 1 + hνX
j=1
aijλξj .
For z = hλ und ξ = (ξ1, . . . , ξν)T ∈ Rν we get
R(z) = 1 + zbT ξ (47)
ξ = e + zAξ, e.g. ξ = (I − zA)−1e. (48)
After using Cramer’s rule
Bx = c =⇒ xi =det(Bi )
det(B), Bi = (a1, . . . , c
i-th column, . . . , aν),
it holds
ξi =Pi (z)
det(I − zA)for Pi ∈ Pν−1, (49)
det(I − zA) ∈ Pν and R(z) = 1 +
Pνj=1 bjzPj
det(I − zA).
Example of an implicit Runge-Kutta methodThe trapezoidal rule
yn+1 = yn +h
2[f (tn, yn) + f (tn+1, yn+1)],
has Butcher scheme
0 | 0 01 | 1
212
−−− + −−− −−−| 1
212
The corresponding rational function is given by
R(z) = 1 + zbT (I − zA)−1e
= 1 + zˆ
12
12
˜„ 1 0− z
21− z
2
«−1 »11
–= 1 + z
ˆ12
12
˜ " 11+ z
21− z
2
#
= 1 +z
2
„1 +
1 + z2
1− z2
«=
1 + z2
1− z2
.
Stability concepts
We write C− = z ∈ C : Re(z) ≤ 0.
DefinitionA rational function R : C→ C such that C− ⊆ SR is calledA-stable.
TheoremA discrete evolution Ψh = R(hA) is (asymptotically) stable for allh > 0 and all (asymptotically) stable A if an only if R is A-stable.
Proof
Let Ψh be stable for all h > 0 and all stable A, this means that itholds |R(hλ)| ≤ 1 for all h > 0 and all λ ∈ C−, in particular|R(λ)| ≤ 1 for all λ ∈ C−. Hence, this imply the A-stability of R.Conversely let R be A-stable, h > 0 und A = λ ∈ C−. SinceC− ⊆ SR , it holds |R(hλ)| ≤ 1.Case 1: Reλ < 0. Then it holds hλ ∈ int S for all h > 0 andtherefore |R(hλ)| < 1.Case 2: Reλ = 0. Since A is stable, it follows ι(λ) = 1, henceι(R(hλ)) = 1. We conclude that Ψh is stable.
Stability concepts ... continues
I A rational function R such that C−α ⊆ SR is calledA(α)-stable, where
C−α = z ∈ C : |arg(−z)| ≤ α .
I An A-stable rational function R such that
limz→∞
R(z) = 0
is called L-stable. Were A asymptotically stable, thenΨh = R(hA)→ 0 for h→∞.
Examples of collocation/IRK methods.Examples
I Gauss-methods:
I ci are the zeros of orthogonal polynomial w.r.t. the weightw ≡ 1 (Legendre polynomials, see [Sections 2.6.2 and 2.7.2,F13])
I ν-stages with consistency order p = 2νI they are A-stableI Example: the implicit middle point rule
yn+1 = yn + hf
(yn + yn+1
2
)I Radau-methods:
I cν = 1, c1, . . . , cν−1 zeros of the Jacobi-polynomial (see theRemark after [Section 2.6.2, F13])
I They are L-stableI Example: implicit Euler method
yn+1 = yn + hf (yn+1)
Stability of multistep methods
Let us consider the application of a general multistep method as in(16), i.e.,
s∑m=0
amyn+m = hs∑
m=0
bmf (tn+m, yn+m), n = 0, 1, . . .
on the Dahlquist test equation, obtaining
s∑m=0
amyn+m = hλs∑
m=0
bmyn+m, n = 0, 1, . . .
or
s∑m=0
(am − hλbm)︸ ︷︷ ︸:=gm
yn+m =s∑
m=0
gmyn+m = 0, n = 0, 1, . . . . (50)
Stability of multistep methods
Defining the characteristic polynomial
η(w) =s∑
m=0
gmwm,
whose zeors are w1, . . . ,wq with multiplicities k1, k2, . . . , kq, where∑qi=1 ki = s, the general solution to (50) is given explicitly by
yn =
q∑i=1
ki−1∑j=0
cijnj
wni , n = 0, 1, . . . , (51)
the s constants cij are uniquely determined by the initial valuesy0, y1, . . . , ys−1.
Stability of multistep methods
LemmaLet us suppose that the zeros (as a function of w) of
η(z ,w) :=s∑
m=0
(am − zλbm)wm, z ∈ C,
are w1(z),w2(z), . . . ,wq(z)(z), while their multiplicities arek1(z), k2(z), . . . , kq(z)(z) respectively. The multistep method (16)is A-stable if and only if
|wi (z)| < 1, i = 1, 2, . . . , q(z)
for every z ∈ C−.
Proof
As for (51), z = λh,
yn =
q(z)∑i=1
ki (z)−1∑j=0
cijnj
wi (z)n, n = 0, 1, . . . ,
the behavior of yn depends on the magnitude of the wi (λh). If allreside inside the complex unit disc, then their power vanishes fasterthan any polynomial in n, hence yn → 0 for n→∞. Hence thecondition is sufficient for the A-stability to hold.However, if |w1(hλ)| ≥ 1, say, then there exist starting values suchthat c1,0 6= 0; therefore it is impossible for yn to tend to zero asn→∞. We deduce that the condition is necessary for A-stability.
Bad news! Stability domains of Adams methods
Backward differentiation formulae
An s-order s-step method is said to be a BDF if σ(w) = βw s forsome β ∈ R \ 0.
LemmaFor a BDF we have
β =
(s∑
m=1
1
m
)−1
and ρ(w) = β
s∑m=1
1
mw s−m(w − 1)m.
ProofBeing the order p = s one has
ρ(w)− βw s ln w = O(|w − 1|)s+1, w → 1.
We substitute v = w−1 and obtain
v sρ(v−1) = β ln v +O(|v − 1|)s+1, v → 1.
Now
ln v = ln(1 + [v − 1]) =sX
m=0
(−1)m−1
m(v − 1)m +O(|v − 1|)s+1
and we deduce that
v sρ(v−1) = βsX
m=0
(−1)m−1
m(v − 1)m,
hence
ρ(w) = βsX
m=1
1
mw s−m(w − 1)m.
To complete the proof, we need only to derive the explicit form of β. It follows
at once by imposing the normalization condition as = 1 on the polynomial ρ.
Examples and convergence
The simplest BDF has been already encountered: when s = 1 werecover the implicit Euler method! The next two BDFs are
s = 2 yn+2 − 43 yn+1 + 1
3 yn = 23 hf (tn+2, yn+2),
s = 3, yn+3 − 1811 yn+2 + 9
11 yn+1 − 211 yn = 6
11 hf (tn+3, yn+3),(52)
Since BDFs are derived by specifying σ, we cannot be sure thatthe polynomial ρ obeys the root condition. In fact, the rootcondition fails for all but a few such methods.
TheoremThe polynomial
ρ(w) = β
s∑m=1
1
mw s−m(w − 1)m
obeys the root condition and the underlying BDF method isconvergent if and only if 1 ≤ s ≤ 6.
Another derivation of BDF
We define the values yj+s byp(tj+k) = f (tj+s , yj+s)
p(tj+i ) = yj+i , i = 0, . . . , s.(53)
Via the Largange polynomials we can define
p(tj+s + θτ) =s∑
i=0
yi+jps−i (θ)
wherepi (−`) = δi`, i , ` = 0, . . . , s.
From (53) one obtains the explicit representation
αsyj+s + · · ·+ α0yj = hf (tj+s , yj+s) (54)
where αi = p′s−i (0), i = 0, . . . , s.
Connection to numerical differentiation
From (53) we have
p(ts) = f (ts , ys) and p(tj) = yj , j = 0, . . . , s,
and from (54)
p(ts) =αsp(ts) + · · ·+ α0p(t0)
h.
Hence, the formula
αsp(ts) + · · ·+ α0p(t0)
h' y(ts) (55)
can be used for the numerical differentiation of a function y ∈ C 1.This explains the name Backward Differentiation Formulae.
Stability domains of BDF methods
Stability domains of BDF methods of orders s = 2, 3, 4, 5, shown at the same
scale. Note that only s = 2 is A-stable.
Theorem (The Dahlquist second barrier)
The highest order of an A-stable a multistep method (16) is 2.
Additional complements: local error estimate and timestep refinement
As we know, for several methods we can hope for a global errorrate of the order of
err = O(hp).
But. how large is practically the constant hidden in the O? Isthere a way to ensure that the global eror keeps under a prescribedrelatively small bound?
Local error estimate and time step refinement
Let us consider now the solution of an ODE by means of aRunge-Kutta method of order p and simultaneously of aRunge-Kutta method of order p ≥ p + 1. We shall assume that yn
and xn are the corresponding numerical solutions and have
yn+1 = y(tn+1) + `hp+1 +O(hp+2),
xn+1 = y(tn+1) +O(hp+2)
The outcome of the subtraction of these equalities is
`hp+1 = yn+1 − xn+1,⇒ κ := ‖yn+1 − xn+1‖.
Once κ is available, we can check whether it reaches a wishedaccuracy threshold, for instance
κ ≤ δh.
If not, then it would better halve the time step h.
Local error estimate and time step refinement
Embedded Runge Kutta methodsA naive application of the above approach requires the doubling, atthe very least, of the expense of calculation, since we need tocompute both yn+1 and xn+1. This is unacceptable since, as a rule,the cost of error control should be marginal in comparison with thecost of the main scheme.However, when an explicit Runge-Kutta method is used for themain scheme it is possible to choose the two methods in such away that the extra expense is small. We shall choose the higherorder method
c | A−−− + −−−
| bT
such that ν ≥ ν,
c =
[cc
], A =
[A 0
A
]where c ∈ Rν−ν and A ∈ Rν−ν×ν , so that A is strictly lowertriangular.
Embedded Runge Kutta methods
In this case the vectors ξ1, . . . , ξν will be the same for both thelower and higher order method and only ξν+1, . . . , ξν−ν needs a bitof additional effort!We say then that the first RK method is embedded in the second.In tableau notation:
c | A−−− + −−−
| bT
−−− + −−−| bT
Embedded Runge Kutta methods
A well-known example of an embedded RK pair is the Fehlbergmethod, with p = 4, p = 5, ν = 5, ν = 6 and tableau
ode23. (Matlab). The Bogacki-Shampine pair embeds a 3rd order3-stage ERK into a 2nd order 4-stage ERK.ode45 (Matlab). The Dormand-Prince pair embeds a 5th order6-stage ERK into a 4th order 7-stage ERK.
Numerical methods for Partial Differential Equations: anintroduction
A first kind of partial differential equations (PDE) are the so-calledelliptic ones. We exemplify them by deriving the so-calledLaplace/Poisson equations from the elastic mechanics. Let usconsider an elestic membrane whose boundary has prescribe shape:
Which shape will the mambrane take? How does its shape dependson the one at the boundary?
Elastic energy
The membrane will certainly take the shape which minimizes itspotential energy. We define Ω ⊂ R2 the domain of the membraneand ∂Ω the boundary of the domain. Let u : Ω→ R the functiondescribing the displacement of the membrane with respect to itsideal position at rest and g : ∂Ω→ R the function describing thegiven shape of the membrane on the boundary. In particular weshall require that u ≡ g on ∂Ω and that the graph of u in factdescribes the surface of the membrane
We also define the potential elastic energy given by
J(u) =
∫Ω
√1 + |∂xu(x , y)|2 + |∂y u(x , y)|2dxdy
The variational problem and its simplificationWe search for a function u : Ω→ R which solves the followingvariational problem:
minu:Ω→R
J(u)
u|∂Ω = g .
To solve this problem, we first proceed with a semplification, underthe assumption that the minimal displacement function is “not toolarge”:
√1 + z = 1 +
1
2z +O(z2), z → 0,
leads us to the minimization of the approximate energy
J(u) =1
2
∫Ω
(|∂xu(x , y)|2 + |∂y u(x , y)|2
)dxdy
=1
2
∫Ω〈∇u,∇u〉dxdy → min!
Here the symbol 〈·, ·〉 represents the scala product in R2 and∇u = (∂xu(x , y), ∂y u(x , y))T is the gradient vector of u.
Calculus of variations
How can we minimize the energy J(u)? We compute the“derivative” of J and we set it to 0.Consider u + αv , α ∈ R, for v ∈ C 1(Ω), v |∂Ω = 0. The minimizer
u of J(u) should solve
0 =d
dαJ(u + αv)|α=0 = lim
α−>0
J(u + αv)− J(u)
α
= limα−>0
1
2α
ZΩ
h“|∂x(u(x , y) + αv(x , y))|2 + |∂y (u(x , y) + αv(x , y))|2
”−“|∂xu(x , y)|2 + |∂y u(x , y)|2
”idxdy
= limα−>0
1
2α
ZΩ
hα2“|∂xv(x , y)|2 + |∂y v(x , y)|2
”+ 2α〈∇u,∇v〉
idxdy
=
ZΩ
〈∇u,∇v〉dxdy .
Gauss divergence theorem
TheoremLet Ω ⊂ Rd be a compact set with a smooth boundary ∂Ω, andν : ∂Ω→ Rd is the external normal vector function, U ⊃ Ω is openon which we define a continuously differentiable vector fieldfunction F : U → Rd . Then it holds∫
Ωdiv Fdx =
∫Ω∇ · Fdx =
∫∂Ω〈F , ν〉ds.
Here we denoted div F (x) = ∇ · F (x) =∑d
j=1 ∂xj Fj(x) thedivergence of F = (F1, . . . ,Fd).
The Laplace equationLet’s go back now to our example and d = 2. We set F = v∇u,then div F = v∆u + 〈∇u,∇v〉, where
∆u = ∂2xxu + ∂2
yy y ,
is the Laplace operator. Hence, by the divergence theorem,∫Ω〈∇u,∇v〉dxdy = −
∫Ω
(v∆u)dxdy +
∫∂Ω
v〈∇u, ν〉s︸ ︷︷ ︸≡0
,
where the last term vanishes because v |∂Ω = 0. Hence∫Ω
(v∆u)dxdy = 0, ∀v ∈ C 1(Ω), v |∂Ω = 0,
or∆u = 0.
the Laplace or potential equation.
Solution on the disk
Let us remind that R2 ' C, z = x + iy = re iϕ. Assume that
w(z) = p(z) + iq(z),
is an holomorphic function(http://en.wikipedia.org/wiki/Holomorphic_function)i.e., it satisfies hte Cauchy-Riemann differential equations
∂xp = ∂y q, ∂y p = −∂xq.
This implies
∆p = ∂yxq − ∂xy q = 0,
∆q = −∂yxp + ∂xy p = 0,
Solution on the diskConsider Ω = B1(0) = z ∈ C : |z | < 1 and
w(x , y) = zk = r ke ikϕ = r k cos(kϕ)| z :=p(x,y)
+i r k sin(kϕ)| z :=q(x,y)
,
k = 0, 1, 2, . . . . This is an holomorphic function and hence
∆p = ∆q = 0.
If we consider the Fourier expansion of the boundary value function
g(x , y) = a0 +∞X
k=1
ak cos(kϕ) + bk sin(kϕ)
then
u(x , y) = a0 +∞X
k=1
r k (ak cos(kϕ) + bk sin(kϕ))
is a function solution of
∆u = 0
u|∂Ω = g .
Poisson equation
In Newtonian gravity and in Electrostatics, there is anotherimportant related equation
−∆u = f ,
called the Poisson equation, where f is a given datum and thesolution is usually additional constrained by suitable boundaryconditions, such as u|∂Ω = g .
Example from Newtonian gravity
In the case of a gravitational field F due to an attracting massiveobject, of density ρ, Gauss’ law for gravity in differential form canbe used to obtain the corresponding Poisson equation for gravity.Gauss’ law for gravity is:
∇ · F = −4πGρ.
Since gravity is a conservative field there has to exists Φ such that
F = ∇Φ.
By substitution we obtain
−∇ · ∇Φ = 4πGρ
or−∆Φ = 4πGρ.
The wave equationLet us start with a simplified model of longitudinal wave andconsider a longitudinal deformation as the one of a set of Nconnected springs subjected to a force:
The deviation of the right edge of the kth spring from its originalposition xk = x1 + kh, is denoted by uk(t). If we assume that allthe springs has the same rigidity/elasticity constantsK = k1 = k2 = k3 = . . . then the balance of forces gives
mu′′k = K (uk+1−uk)−K (uk−uk−1) = K (uk+1−2uk +uk−1). (56)
The wave equation...If we now assume that there is a continuous description limit forthe number N of springs to infinity, we have
uk(t) ≈ u(xk , t).
We re-write (56) as follows
m
h∂ttu(xk , t) = Kh
(u(xk+1, t)− 2u(xk , t) + u(xk−1, t)
h2
),
and for “h→ 0” (or for N →∞) – by rescaling aslo m(h)/h→ 1and K (h)h→ c2 – we obtain the wave equation:
∂ttu = c2∂xxu
which is usually complemented with the initial time conditions
u(x , 0) = f (x) ∂tu(x , 0) = g(x),
In general, the wave equation is given by
∂ttu = c2∆u
The heat equation
Let us consider a one dimensional body represented by an interval: theFourier’s law of heat diffusion says that the rate of flow of heat energy througha surface within the body is proportional to the negative temperature gradientacross the surface,
q = −k∇u (57)
where k is the thermal conductivity and u is the temperature. In onedimension, the gradient is an ordinary spatial derivative, and so Fourier’s law is
q = −k∂u
∂x
In the absence of work done, a change in internal energy per unit volume in thematerial, δQ, is proportional to the change in temperature, δu. That is,
δQ = cpρ δu
where cp is the specific heat capacity and ρ is the mass density of the material.By integration, choosing zero energy at absolute zero temperature, this can berewritten as
Q = cpρu.
The heat equationThe increase in internal energy in a small spatial region of the material[x − δx , x + δx ] over the time period [t − δt, t + δt] is given by
cpρ
Z x+δx
x−δx[u(ξ, t + δt)− u(ξ, t − δt)] dξ = cpρ
Z t+δt
t−δt
Z x+δx
x−δx
∂u
∂τdξ dτ
where the fundamental theorem of calculus was used. If no work is done andthere are neither heat sources nor sinks, the change in internal energy in theinterval [x − δx , x + δx ] is accounted for entirely by the flux of heat across theboundaries. By Fourier’s law (57), this is
k
Z t+δt
t−δt
»∂u
∂x(x + δx , τ)− ∂u
∂x(x − δx , τ)
–dτ = k
Z t+δt
t−δt
Z x+δx
x−δx
∂2u
∂ξ2dξ dτ.
By conservation of energy,Z t+δt
t−δt
Z x+δx
x−δx[cpρ∂τ − k∂ξξu] dξ dτ = 0.
This is true for any rectangle [t − δt, t + δt]× [x − δx , x + δx ], hence theintegrand must vanish identically:
cpρ∂τu − k∂ξξu = 0, or ∂τu =k
cpρpartialξξu,
which is the heat equation, where the coefficient α = kcpρ
is called the thermal
diffusivity.
The heat equation
In its full glory the heat equation reads
∂tu = α∆u,
and it is usually complemented by boundary and initial conditions
u(x , 0) = f (x),∀x ∈ Ω u(·, t)|∂Ω = g(·, t).
Solution of the heat equation in 1DThe following solution technique for the heat equation was proposed by JosephFourier in his treatise Theorie analytique de la chaleur, published in 1822. Letus consider the heat equation for one space variable. The equation isut = αuxx , where u = u(x , t) is a function of two variables x ∈ [0, L] and t ≥.Here we assume the initial condition
u(x , 0) = f (x) ∀x ∈ [0, L]
and the boundary conditions
u(0, t) = 0 = u(L, t) ∀t > 0.
Let us attempt to find a solution of the type
u(x , t) = X (x)T (t).
This solution technique is called separation of variables. Substituting u backinto the heat equation,
T (t)
αT (t)=
X ′′(x)
X (x).
Since the right hand side depends only on x and the left hand side only on t,both sides are equal to some constant value −λ. Thus:
T (t) = −λαT (t)
andX ′′(x) = −λX (x).
Solution of the heat equation in 1D ...Suppose that λ < 0. Then there exist real numbers B,C such that
X (x) = Be√−λ x + Ce−
√−λ x .
From the boundary conditions, we get X (0) = 0 = X (L) and thereforeB = 0 = C which implies u is identically 0. Suppose that λ = 0. Then thereexist real numbers B,C such that X (x) = Bx + C . From the boundaryconditions we conclude in the same manner as in 1 that u is identically 0.Therefore, in order to have nontrivial solutions, it must be the case that λ > 0.Then there exist real numbers A,B,C such that
T (t) = Ae−λαt
andX (x) = B sin(
√λ x) + C cos(
√λ x).
From the boundary condition we get C = 0 and that for some positive integern, √
λ = nπ
L.
This solves the heat equation in the special case that the dependence of u hasthe given special form. In general, the solution is given by
u(x , t) =∞Xn=1
Dn sin“nπx
L
”e− n2π2αt
L2
where Dn = 2L
R L
0f (x) sin
`nπxL
´dx .
Finite Difference Method for Elliptic EquationsThe main idea is to approximate derivatives by means of finitedifferences.Let us start with a simple 1D boundary value problem. LetΩ = (0, 1) and we consider u : Ω→ R solution of the Poissonequation
u′′ = f , on Ω,
u(0) = g0
u(1) = g1.
We construct the numerical approximation to the solution on agrid Ωh := h, 2h, . . . , 1− h and we define Ωh = Ωh ∪ 0, 1 ofgrid size h = 1
n , for some n ∈ N.Exercise: show that
u′′(x) =u(x + h)− 2u(x) + u(x − h)
h2+ h2R, (58)
where |R| = |R(x)| ≤ 112‖u
(4)‖∞, as soon as u ∈ C 4(Ω).
Discrete boundary value problemAccording to the approximation derived from (58), we define thefollowing discrete boundary value problem:
uh(x + h)− 2uh(x) + uh(x − h)
h2= f (x), for all x ∈ Ωh,
uh(0) = g0
uh(1) = g1.
Our hope is that uh ≈ u on Ωh. The discrete linear equations canbe written in matrix form as follows
1
h2
0BBBB@−2 1 0 . . . . . . 01 −2 1 0 . . . 0. . . . . . . . . . . . . . . . . .0 . . . 0 1 −2 10 . . . . . . 0 1 −2
1CCCCA| z
:=Lh
0BBBB@uh(h)
uh(2h). . .
uh(1− 2h)uh(1− h)
1CCCCA| z
:=uh
=
0BBBB@f (h)− 1
h2 g0
f (2h). . .
f (1− 2h)f (1− h)− 1
h2 g1
1CCCCA| z
:=fh
Hence uh can be computed by solving the above defined linearsystem by using methods from numerical linear algebra (Gausselimination, LU/QR decompositions, Gauss/Jacobi methods,gradient and conjugate gradient methods, multigrid methods).
Abstract finite differencesLet us consider a general elliptic problem defined on a openbounded domain Ω ⊂ R2
Lu = f on Ω, u|∂Ω = g , (59)
where L is an elliptic differential operator, for instance L = −∆.We shall consider a discretization of the problem on a grid Ωh ⊂ Ωof grid size h, for instance Ωh = Ω∩ hZ2. We define ∂Ωh ⊂ Ω as afinte set of boundary grid points approximating ∂Ω and being closeto it. We finally write Ωh = Ωh ∪ ∂Ωh.An abstract finite difference method for the solution of the abovewritten equation is given by means of the representation
Lu = Lhu + error ,
whereLhu(x) =
∑x ′∈N(x)
w(x , x ′)u(x ′), x ∈ Ωh,
and N(x) ⊂ Ωh is the set of neigborhood points of x ∈ Ωh,w(x , x ′) = O(h−2).
Discrete boundary value problem
Accordingly we define the discrete boundary value problem
Lhu = fh on Ωh, uh = gh on ∂Ωh, (60)
where fh, gh are approximations to f and g on Ωh and ∂Ωh
respectively, for instance f ≡ fh and g ≡ gh.
Consistency error
Denoted by δh(x) = Lhu(x)− fh(x) the consistency deviation,provided u being the exact solution of the boundary value problem(59), we say that the finite difference method is consistent if
limh→0‖δh‖∞ = 0.
It has the consistency order p if
‖δh‖∞ = O(hp), h→ 0.
Notice that this is different definition with respect to theconcistency error for methods for ODE, since there we requiredthat the consistency error was of order O(hp+1)!