Solution of initial value problems

In the method of lines, time integration is performed using numericalmethods for initial value problems of the form

dudt (t) = f(t, u(t)), t ∈ (tn, tn+1), u(tn) = un

Exact integration∫ tn+1

tndudt (t)dt = un+1 − un =

∫ tn+1

tnf(t, u(t))dt

Evaluation of f(t, u(t)) is impossible for t ∈ (tn, tn+1)

The integral is approximated using numerical quadrature

Multipoint methods

Order barrier: two-level methods are at most second-order accurate

Additional points are needed to construct higher order schemes

Adams methods perform time integration using the points

tn+1, . . . , tn−m, m = 0, 1, . . .

Runge-Kutta methods insert points between tn and tn+1

tn+α ∈ [tn, tn+1], α ∈ [0, 1]

Adams methods

Adams methods are multipoint time integrators which can beinterpreted as Newton-Cotes quadrature rules

A multipoint approximation of p-th order is derived by fitting aLagrange polynomial to the data at p+ 1 time levels

t

n+1

t

present futurepast

f

t

n

t

n�1

t

n�2

Adams methods

Adams-Bashforth methods (explicit)

un+1 = un + f(tn, un)∆t

un+1 = un + 12 [3f(tn, un)− f(tn−1, un−1)]∆t

un+1 = un + 112 [23f(tn, un)− 16f(tn−1, un−1) + 5f(tn−2, un−2)]∆t

Adams-Moulton methods (implicit)

un+1 = un + f(tn+1, un+1)∆t

un+1 = un + 12 [f(tn+1, un+1) + f(tn, un)]∆t

un+1 = un + 112 [5f(tn+1, un+1) + 8f(tn, un)− f(tn−1, un−1)]∆t

Adams methods

Adams-Bashforth method of order p− 1 can be used as predictorfor an Adams-Moulton corrector of order p

Adams methods of any order are easy to derive and implement(as long as the time step ∆t is fixed)

Just one derivative evaluation per time step is required (if solutionvalues calculated at the previous time levels are stored)

Low-order methods are needed to start/restart time integration

Time step is difficult to change (the coefficients are different)

Numerical instabilities and/or spurious oscillations may occur

Runge-Kutta methods

Runge-Kutta methods are predictor-(multi)corrector algorithms

Auxiliary solutions correspond to time levels between tn and tn+1

The second-order Runge-Kutta method employs a forward Eulerpredictor for un+1/2 and a midpoint rule corrector for un+1

un+1/2 = un + 12f(tn, un)∆t

un+1 = un + f(tn+1/2, un+1/2)∆t

Higher order can be achieved using additional corrector steps

Runge-Kutta methods

The classical fourth-order Runge-Kutta method is given by

un+1/2 = un + 12f(tn, un)∆t

un+1/2 = un + 12f(tn+1/2, un+1/2)∆t

un+1 = un + f(tn+1/2, un+1/2)∆t

un+1 = un + 16 [f(tn, un) + 2f(tn+1/2, un+1/2)

+2f(tn+1/2un+1/2) + f(tn+1, un+1)]∆t

Runge-Kutta methods

Self-starting and easy to operate with variable time steps

More stable and accurate than Adams methods of the same order

Higher-order approximations are difficult to derive; a p-stage methodrequires p evaluations of the derivative function per time step

More expensive than Adams methods of comparable order

Time step control

Criteria for the choice of the time step ∆t depend on the type of thetime stepping procedure (explicit or implicit)

In explicit schemes, time steps that are smaller than but sufficientlyclose to the stability bound are usually optimal

In unconditionally stable implicit schemes, the optimal value of ∆tdepends on the magnitude of the time discretization error

Adaptive time step control can be performed using error estimatorsdeveloped in the context of numerical methods for ODEs

Time step control

Let u∆t be a solution calculated using the time step ∆t

u∆t = u+ (∆t)2e(u) +O(∆t)4

um∆t = u+m2(∆t)2e(u) +O(∆t)4

The local truncation error can be estimated as follows:

e(u) ≈ um∆t − u∆t

(∆t)2(m2 − 1)

The time step can be adjusted to satisfy

||u− u∆t|| ≈ TOL

Time step control

The optimal time step ∆t∗ can be determined as follows:

||u− u∆t∗ || ≈(

∆t∗∆t

)2 ||u∆t − um∆t||m2 − 1 = TOL

⇒ (∆t∗)2 = TOL(∆t)2(m2 − 1)||u∆t − um∆t||

Accuracy can be improved using Richardson extrapolation

u = m2u∆t − um∆t

m2 − 1 + O(∆t)4

Time step control

Practical implementation

1 Given the old solution un, calculate un+1m∆t and un+1

∆t

2 Calculate∥∥un+1

∆t − un+1m∆t

∥∥ and the next time step ∆t∗

3 If ∆t∗ is much smaller than ∆t, reset and go back to step 1

4 Accept un+1 = un+1∆t or calculate un+1 = m2un+1

∆t−un+1

m∆t

m2−1

Other error estimators: embedded Runge-Kutta and Adams methods

PID control

1 Monitor the relative changes at the current time step

en = ||un+1 − un||||un+1||

2 If en is too large (en > δ), repeat the time step using

∆t∗ = δ

en∆tn

3 Adjust the time step smoothly to approach the prescribedtolerance TOL for the indicator en

∆tn+1 =(en−1

en

)kP(TOL

en

)kI(

e2n−1

enen−2

)kD

∆tn

Properties of numerical methods

Consistency: local truncation errors are proportional to positivepowers of the time step ∆t and/or mesh size h

Stability: Round-off and iteration iteration errors that are generatedin the numerical solution process are not amplified

Convergence: The global error (measured w.r.t. certain norms)tends to zero in the limit h→ 0 and ∆t→ 0

Stability analysis

Two-level explicit schemes produce linear systems of the form

un+1 = Cun

In practice, the old solution already contains numerical errors

un+1 = Cun

If no new errors are generated at the current step, we obtain

en+1 = Cen, en = un − un

Stability analysis

The discretization is stable if the error does not grow

‖en+1‖ = ‖Cen‖ ≤ ‖en‖

A matrix form consistent with ‖ · ‖ is given by

‖C‖ := supu 6=0

‖Cu‖‖u‖

The error growth can be estimated as follows:

‖en+1‖‖en‖

= ‖Cen‖

‖en‖≤ ‖C‖

Stability analysis

Boundedness of the matrix norm ‖C‖ is sufficient for stability

‖C‖ ≤ 1

If φ(m) is an eigenvector of the matrix C and λ(m) is its associatedeigenvalue then Cφ(m) = λ(m)φ(m). It follows that

|λ(m)| = ‖λ(m)φ(m)‖‖φ(m)‖

= ‖Cφ(m)‖

‖φ(m)‖≤ ‖C‖

The corresponding stability condition for the eigenvalues of C is

|λ(m)| ≤ 1, m = 1, . . . , N

Stability analysis

Convection-diffusion equation∂u

∂t+ v

∂u

∂x= d

∂2u

∂x2

Space discretization (CD)duidt + v

ui+1 − ui−1

2∆x = dui−1 − 2ui + ui+1

(∆x)2

Time discretization (FE)

un+1i − uni

∆t + vuni+1 − uni−1

2∆x = duni−1 − 2uni + uni+1

(∆x)2

Stability analysis

Linear algebraic system un+1 = Cun

un+1i = uni −

ν

2 (uni+1 − uni−1) + δ(uni−1 − 2uni + uni+1)

where ν = v ∆t∆x is the Courant number and δ = d ∆t

(∆x)2

C =

· · ·a b c

a b ca b c· · ·

a = δ + ν

2

b = 1− 2δ

c = δ − ν2

Stability analysis

Eigenvectors

φ(m)j = cos θmj + i sin θmj = eiθmj , j,m = 1, . . . , N

where i is the complex unity and

θm = m πN is the phase angle

low frequen ieshigh frequen ies

m = N m = 0

0�

Cφ(m) = λ(m)φ(m) ⇒ aφ(m)j+1 + bφ

(m)j + cφ

(m)j−1 = λ(m)φ

(m)j

b = 1− (a+ c) ⇒ λ(m) = 1 + a(eiθm − 1) + c(e−iθm − 1)

Stability analysis

Eigenvalues

λ(m) = 1 + 2δ(cos θm − 1) + i2ν sin θm

Stability condition

|λ(m)|2 = [1 + 2δ(cos θm − 1)]2 + 4ν2 sin2 θm ≤ 1

pure convection: δ = 0 ⇒ |λ(m)| ≥ 1 (unstable)

pure diffusion: ν = 0 ⇒ |λ(m)| ≤ 1 for δ ≤ 12

δ = d ∆t(∆x)2 ⇒ ∆t ≤ d (∆x)2

2 (conditionally stable)

Fourier analysis

The eigenvectors φ(m)j = eiθmj form the Fourier basis such that

unj − unj =∑m

anmeiθmj

The von Neumann stability analysis examines the evolution of

anmeiθmj = anm(cos θmj + i sin θmj)

The discretization is stable if the amplitude of the sine and cosinewaves remains bounded for all Fourier modes / phase angles

Fourier analysis

The amplification factor of the m-th mode is defined by

Gm = an+1m

anm= λ(m)

The wave amplitude anm is non-increasing if

|Gm| ≤ 1 ∀m = 1, . . . , N

In general Gm is a complex number. We have

Gm = α+ iβ ⇒ |Gm| =√α2 + β2

1st example

Pure convection equation∂u

∂t+ v

∂u

∂x= 0, v > 0

Space discretization (CD)duidt + v

ui+1 − ui−1

2∆x = 0

Time discretization (FE)

un+1i − uni

∆t + vuni+1 − uni−1

2∆x = 0

1st example

Discrete problem

un+1j = unj + ν

2 (unj+1 − unj−1) = 0, ν = v∆t∆x

Substitution of unj = aneiθj yields an equation for an+1

(an+1 − an)eiθj + ν

2an(eiθ(j+1) − eiθ(j−1)) = 0

⇒ an+1 = an − ν

2an(eiθ − e−iθ)

where eiθ − e−iθ = cos θ + i sin θ − cos θ + i sin θ = 2i sin θ

1st example

Amplification factor

G = an+1

an= 1− iν sin θ

Conclusion: unstable since

|G|2 = 1 + ν2 sin2 θ ≥ 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1x 10

22

2nd example

Pure convection equation∂u

∂t+ v

∂u

∂x= 0, v > 0

Space discretization (BD)duidt + v

ui − ui−1

∆x = 0

Time discretization (FE)

un+1i − uni

∆t + vuni − uni−1

∆x = 0

2nd example

Discrete problem

un+1j = unj + ν(unj − unj−1) = 0, ν = v

∆t∆x

Substitution of unj = aneiθj yields an equation for an+1

(an+1 − an)eiθj + νan(eiθj − eiθ(j−1)) = 0

⇒ an+1 = an − νan(1− e−iθ)

G = an+1

an = 1− ν + νe−iθ = 1− ν + ν(cos θ − i sin θ)

2nd example

Amplification factorRe(G) = 1− ν + ν cos θ, Im(G) = −ν sin θ

Conclusion: stable under theCFL condition

ν = v∆t∆x ≤ 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Initial

Exact

∆ x=10−2

∆ x=10−3

3rd example

Pure convection equation∂u

∂t+ v

∂u

∂x= 0, v > 0

Space discretization (CD)duidt + v

ui+1 − ui−1

2∆x = 0

Time discretization (BE)

un+1i − uni

∆t + vun+1i+1 − u

n+1i−1

2∆x = 0

3rd example

Discrete problem

un+1j = unj + ν

2 (un+1j − un+1

j−1 ) = 0, ν = v∆t∆x

(an+1 − an)eiθj + ν

2an+1(eiθ(j+1) − eiθ(j−1)) = 0

Amplification factorG = 1

1+iν sin θ

Conclusion: stable for all ν since |G|2 = G · G = 11+ν2 sin2 θ ≤ 1

Numerical diffusion

Pure convection equation: BD/FE approximation

∂u

∂t+ v

∂u

∂x= 0 7→ un+1

i − uni∆t + v

uni − uni−1∆x = 0

un+1i−un

i

∆t + vun

i+1−uni−1

2∆x =(v∆x

2) un

i+1−2uni +un

i−1(∆x)2

This corresponds to the 2nd order CD approximation of

∂u

∂t+ v

∂u

∂x=(v∆x

2

)∂2u

∂x2

where v∆x2 is the numerical diffusion coefficient

Example

BD/FE approximation, ε = v∆x2 , ∆t = ∆x/2

∂u

∂t+ v

∂u

∂x= 0, u(x, t) = u0(x− vt), v ≡ 1, T = 0.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Initial

Exact

∆ x=10−2

∆ x=10−3

smooth solution

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Initial

Exact

∆ x=10−2

∆ x=10−3

discontinuous solution

Modified equations

Let uh be a numerical solution of the time-dependent problem

∂u

∂t+ Lu = 0

This approximation represents the exact solution of another PDE

∂u

∂t+ Lu = ε

Such modified equations provide a useful tool for qualitativeanalysis of accuracy and stability of numerical approximations

Derivation of ME

Double Taylor series expansion about (xi, tn) yields a PDE

The difference between this PDE and the original one is the localtruncation error which contains both space and time derivatives

Time derivatives can be expressed in terms of space derivativesusing the new PDE and its derivatives

The resulting representation of the local truncation error is bettersuited for qualitative analysis of accuracy and stability

Interpretation of MG

Modified equation (Warming & Hyett, 1974)

∂u

∂t+ v

∂u

∂x=∞∑p=1

µ2p∂2pu

∂x2p +∞∑p=1

µ2p+1∂2p+1u

∂x2p+1

µ2p 6= 0 ⇒ amplitude error (numerical diffusion)

µ2p+1 6= 0 ⇒ phase error (numerical dispersion)

Stability condition (necessary, not sufficient)

k := min{p ∈ N : µ2p 6= 0}, sign(µ2k) = (−1)k+1

Examples of ME

Pure convection equation∂u∂t + v ∂u∂x = 0, v > 0

Upwind discretizationun+1

i−un

i

∆t + vun

i −uni−1

∆x = 0

Taylor expansion about (xi, tn)

un+1i = uni + ∆t

(∂u∂t

)ni

+ (∆t)2

2

(∂2u∂t2

)ni

+ (∆t)3

6

(∂3u∂t3

)ni

+ . . .

uni−1 = uni −∆x(∂u∂x

)ni

+ (∆x)2

2

(∂2u∂x2

)ni− (∆x)3

6

(∂3u∂x3

)ni

+ . . .

Examples of ME

Local truncation error

ε = −∆t2

(∂2u∂t2

)ni− (∆t)2

6

(∂3u∂t3

)ni

+ v∆x2

(∂2u∂x2

)ni− v(∆x)2

6

(∂3u∂x3

)ni

+ . . .

Modified equation∂u

∂t+ v

∂u

∂x= v∆x

2 (1− ν)∂2u

∂x2 + v(∆x)2

6 (3ν − 2ν2 − 1)∂3u

∂x3 + . . .

Stability condition (CFL)

0 ≤ ν ≤ 1, ν = v∆t∆x

Examples of ME

Central differences∂u

∂t+ v

∂u

∂x= −v

2∆x2

∂2u

∂x2 −v(∆x)2

6 (1 + 2ν2)∂3u

∂x3 + . . .

Finite elements (P1)∂u

∂t+ v

∂u

∂x= −v

2∆x2

∂2u

∂x2 −v(∆x)2

3 ν2 ∂3u

∂x3 + . . .

µ2 = − v2∆x2 < 0 ⇒ unstable since the ME is ill-posed

(no continuous dependence of the solution on the data)

Lax-Wendroff method

Taylor expansion un+1 = un + ∆tunt + (∆t)2

2 untt +O(∆t)3

∂u∂t + v ∂u∂x = 0 ⇒ ut = −vux, utt = (ut)t = −v(ut)x = v2uxx

Time discretization un+1 = un − v∆tunx + (v∆t)2

2 unxx + O(∆t)3

Lax-Wendroff / central differences

un+1i − uni

∆t + vuni+1 − uni−1

2∆x = (v∆t)2

2uni+1 − 2uni + uni−1

(∆x)2

Stabilization via inherent numerical diffusion, 2nd order accuracy

Taylor-Galerkin methods

Taylor expansion un+1 = un + ∆tunt + (∆t)2

2 untt + (∆t)3

6 unttt +O(∆t)4

uttt = (utt)t = un+1tt − untt

∆t +O(∆t) = v2un+1xx − unxx

∆t +O(∆t)

un+1 = un − v∆tunx + (v∆t)2

2 unxx + (v∆t)2

6 (un+1xx − unxx) + O(∆t)4

Mun+1i − uni

∆t + vuni+1 − uni−1

2∆x = (v∆t)2

2uni+1 − 2uni + uni−1

(∆x)2

Mui = ui+1 + 4ui + ui−1

6 − v2∆t6

ui+1 − 2ui + ui−1

(∆x)2

Modified equations

ME for Lax-Wendroff / CD ⇒ ν2 ≤ 1∂u∂t + v ∂u∂x = −v(∆x)2

6 (1− ν2)∂3u∂x3 − v(∆x)3

8 ν(1− ν2)∂4u∂x4 + . . .

ME for Lax-Wendroff / FE ⇒ ν2 ≤ 13

∂u∂t + v ∂u∂x = v(∆x)2

6 ν2 ∂3u∂x3 − v(∆x)3

24 ν(1− 3ν2)∂4u∂x4 + . . .

ME for Taylor-Galerkin FE ⇒ ν2 ≤ 1∂u∂t + v ∂u∂x = − v(∆x)3

24 ν(1− ν2)∂4u∂x4 + v(∆x)4

180 (1− 5ν2 + 4ν4)∂5u∂x5 + . . .

Example

Lax-Wendroff / Taylor-Galerkin ∆x = 10−2, ν = 0.5∂u

∂t+ v

∂u

∂x= 0, u(x, t) = u0(x− vt), v ≡ 1, T = 0.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2Initial

Exact

LW

TG

smooth solution

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2Initial

Exact

LW

TG

discontinuous solution