Process Model Formulation and Solution, 3E4Section F, Ordinary Differential Equations

Instructor: Kevin Dunn dunnkg@mcmaster.ca

Department of Chemical Engineering

Course notes: © Dr. Benoıt Chachuat15 November 2010

1

Why solve ordinary differential equations?

Consider the modelling of an isothermal CSTR, with a single inlet stream.

Assumptions:

I A1: Perfect mixing

I A2: Equal inflow and outflow

I A3: Constant liquid density

I A4: Single first-order reaction

Modelling goals:

I Calculate the concentration CA in thereactor at a given time t ≥ 0

2

Why solve ordinary differential equations?

F in C inA V k

1 L s−1 1 mol L−1 1 L 1 s−1

3

Why solve ordinary differential equations?

Consider the modelling of a CSTR, with a single inlet stream.

Assumptions:

I A1: Perfect mixing

I A2: Outflow F out(t) = α√

V (t)

I A3: Constant liquid density

I A4: Single first-order reaction

Modeling Goals:

I Calculate the volume V and theconcentration CA in the reactor at agiven time t ≥ 0

4

Outline and recommended readings

Motivation

Differential equations: concepts and classification

Numerical solution of ODEsEuler’s methodTaylor series methodsRunge-Kutta methodsHeun’s predictor-corrector method

(Strongly) recommended readings:

I Chapters PT7, 25.1-25.4, and 26.1 in: S. C.Chapra, and R. P. Canale, “NumericalMethods for Engineers”, McGraw Hill, 5th/6thedition

5

Differential equations: concepts and classification

We consider ordinary differential equations (ODE) of the following form:x ′1(t) = f1(t, x1, . . . , xn)

...x ′n(t) = fn(t, x1, . . . , xn)

or x′(t) = f(t, x(t))

I t, independent variable; x, dependent variable(s)

I Same number of dependent variables and equations

Scalar vs. vectorial ODEs:

I Scalar ODE: single variable/equation, x(t) ∈ IR

I Vectorial ODEs: multiple variables/equations, x(t) ∈ IRn

I Examples: Revisit CSTR case studies

Need an equal number of initial conditions as dependent variables!

6

Differential equations: concepts and classification

Geometrical interpretation: Scalar ODEs x ′(t) = 1− 2x(t)

I Infinitely many curves satisfy the differential equation!

I Extra condition needed to uniquely specify the solution: x(0) = x0

7

Differential equations: concepts and classification

Initial Value Problems (IVPs): ←− our focus

I All initial conditions specified at the same time

I Example: {x ′1(t) = 2x1(t)− x2(t); x1(0) = a

x ′2(t) = −x1(t) + 3x2(t); x2(0) = b

Boundary Value Problems (BVPs):I Not all initial conditions specified at the same time

I Two-point boundary value problem: Split boundary conditionsI Also, multi-point boundary value problem

I Example: {x ′1(t) = 2x1(t)− x2(t); x1(0) = a

x ′2(t) = −x1(t) + 3x2(t); x2(1) = b

8

Differential equations: concepts and classification

linear ordinary differential equations:x ′1(t) = a11(t)x1(t) + · · ·+ a1n(t)xn(t) + b1(t)

...x ′n(t) = an1(t)x1(t) + · · ·+ ann(t)xn(t) + bn(t)

or x′(t) = A(t)x(t) + b(t)

I aij and bj may depend on time (differentiable functions in t)

I Analytical solutions are known for linear ODEs

I Solution are guaranteed to exist at all times

Nonlinear ordinary differential equations:x ′1(t) = f1(t, x1(t), . . . , xn(t))

...x ′n(t) = fn(t, x1(t), . . . , xn(t))

or x′(t) = f(t, x(t))

I General, nonlinear functions f1, . . . , fn (differentiable in t, x1, . . . , xn)

I No analytical solution in general!

I Solution may not exist at all time instants

9

Differential equations: concepts and classification

Example: Linear vs. nonlinear ODEs

Which of the following ODEs are linear? Which ones are nonlinear?{x ′1(t) = 2x1(t)− t x2(t)x ′2(t) = −x1(t) + 3x2(t)

{x ′1(t) = 2e−t [1− x2(t)]x ′2(t) = x1(t)[3x2(t)− 1]

Example 2: A “nasty” nonlinear ODE

Consider the scalar ODE: x ′(t) = x2(t),with the initial condition x(0) = x0

I Is x(t) =x0

1− x0ta solution?

I What’s happening for x0 = 0?

I What’s happening when x0 > 0?

10

Differential equations: concepts and classification

Example 3: asymptotic behaviour and stability

Consider the scalar ODE: x ′(t) = 1− x2(t). Observe and discuss thesolutions obtained from various initial conditions at t = 0.

11

Differential equations: concepts and classification

First- and higher-order ODEs:

I 1st-Order ODE: x ′(t) = f (t, x(t))

I nth-Order ODE: x (n)(t) = f (t, x(t), x ′(t), . . . , x (n−1)(t))

The good news!

A nth-order ODE is equivalent to a system of n 1st-order ODEs: how?

I The solution to a nth-order ODE is uniquely specified by exactly ninitial conditions

I e.g., x(0), x ′(0), . . . , xn−1(0)

Workshop:

Reformulate the following 2nd-order ODE as a set of 1st-order ODEs:

x ′′(t) + x(t)[3x ′(t)− 1] = e−t

12

Numerical solution of ODEs

Exact solution to an initial value problem in ODEs:

I x satisfies the initial condition x(t0) = x0

I x satisfies the differential relation x ′(t) = f (t, x(t)) at any t ≥ t0

The numerical solution problem

Given k time instants t1, t2, . . . , tk ,calculate values x1, x2, . . . , xk such that:

x1 ≈ x(t1), x2 ≈ x(t2), . . . , xk ≈ x(tk)

All numerical solution methods for ODEs are iterative!I Given solution estimates x1, x2, . . . , xk at previous points

t1, t2, . . . , tk , calculate an estimate xk+1 at next point tk+1

I One-step methods: estimate xk+1 by using xk onlyI Multi-step methods: estimate xk+1 by using xk , xk−1, . . .

13

One-step solution methods: principles

Main concept:

Given xk ≈ x(tk), calculatexk+1 ≈ x(tk+1) as:

xk+1 = xk + φ h

I φ, slope

I h = tk+1 − tk , stepsize

I The slope φ is used to extrapolate to a new value

I The formula can be applied step-by-step to approximate thesolution:

1. Initialize: k ← 0, x0 = x(t0)2. Repeat: k ← k + 1, tk+1 = tk + h, xk+1 = xk + φ h; Until: tk+1 ≥ tf

I One-step methods differ in the manner the slope is estimated

I Applicable to both scalar and vectorial ODEs

14

Euler’s method: basics (a.k.a. Euler-Cauchy, or Point-Slope)

Main concept:

Use the time derivative as the slopeestimate:

φ = x ′(tk) = f (tk , xk)

Calculate xk+1 ≈ x(tk+1) as:

xk+1 = xk + f (tk , xk) h

(Local) error analysis:

I The error is: ek+1 = x(tk+1)− xk+1 = x(tk+1)− xk − f (tk , xk) h

I If xk = x(tk), then ek+1 = O(h2) why?

15

Euler’s method: application

Use Euler’s method to solve the scalar ODE x ′(t) = − 12x(t) + exp(−t)

for t ∈ [0, 1], with initial condition x(0) = 1 and various stepsizesh = 10−1, 10−2, 10−3 and 10−4

Exact solution: x(t) = 3 exp(− 12 t)− 2 exp(−t)

h x1 e1 xN eN

1e-1 1.05 -5.99e-3 1.10991 -2.61e-2

1e-2 1.005 -6.22e-5 1.08634 -2.51e-3

1e-3 1.0005 -6.25e-7 1.08408 -2.50e-4

1e-4 1.00005 -6.25e-9 1.08386 -2.50e-5

x(1) = 1.08383

Error on x(1) is not O(h2)!

16

Euler’s method: revisited

I Global error analysis: The global error grows after each step!

I The number of steps to go from t0 to tf is N ≈ tf−t0h

etot =N∑

k=1

ek = N × O(h2) = O(h) : why?

17

Euler’s method: coding

Write a function that solves the differential equation x ′(t) = f (t, x(t))for t ∈ [t0, tf ], with initial condition x(t0) = x0, using N steps

18

Euler’s method: application to vectorial ODEs

Use Euler’s method to solve the high-order ODEx ′′(t) + x(t)[3x ′(t)− 1] = exp(−t) for t ∈ [0, 1], with initial conditionsx(0) = 0, x ′(0) = 0, and stepsize h = 10−1

tk x1,k x2,k

0 0 0

0.1000 0 0.1000

0.2000 0.0100 0.1905

0.3000 0.0290 0.2728

0.4000 0.0563 0.3474

0.5000 0.0911 0.4142

0.6000 0.1325 0.4726

0.7000 0.1797 0.5220

0.8000 0.2319 0.5615

0.9000 0.2881 0.5905

1.0000 0.3471 0.6090

19

Taylor series method: principles

Principle: Try to get more accurate estimates with larger/fewerstepsizes!

I Include higher-order terms of the Taylor series expansion of x(t):

x(t + h) ≈ x(t) + x ′(t)h + x ′′(t)h2

2+ · · ·+ x (j)(t)

hj

j!

I k = 1: [Euler’s method] local error O(h2); global error O(h)

x ′(tk) ≈ f (tk , xk),

xk+1 = xk + f (tk , xk)h

I k = 2: local error O(h3); global error O(h2)

x ′′(tk) ≈∂f (tk , xk)

∂t+

∂f (tk , xk)

∂xf (tk , xk),

xk+1 = xk + f (tk , xk)h +

[∂f (tk , xk)

∂t+

∂f (tk , xk)

∂xf (tk , xk)

]h2

2

I k = 3: ...

Problem: Higher-order derivatives become increasingly more complicated

20

Taylor series method: application

Use the Taylor series method (order 2) to solve the scalar ODEx ′(t) = − 1

2x(t) + exp(−t) for t ∈ [0, 1], with initial condition x(0) = 1and stepsize h = 10−1

Exact solution: x(t) = 3 exp(− 12 t)− 2 exp(−t)

I Euler’s method (k = 1):

xk+1 = xk+[−xk

2+ exp(−tk)

]h

I Taylor series method (k = 2):

xk+1 = xk+?

21

Runge-Kutta methods in a nutshell

x ′(t) = f (t, x(t)); x(t0) = x0

Procedure for all Runge-Kutta methodsI Given an estimate xk of the solution at time tk ,

I Calculate the next estimate xk+1 = xk + φ h, with:

φ = a1K1 + a2K2 + · · ·+ aNKN , N: Order of the method

and,

K1 = f (tk , xk)

K2 = f (tk + p1h, xk + q1,1 K1 h)

K3 = f (tk + p2h, xk + q2,1 K1 h + q2,2 K2 h)

...

KN = f (tk + pN−1h, xk + qN−1,1 K1 h + qN−1,2 K2 h + · · ·+ qN−1,N−1 KN−1 h)

22

Runge-Kutta methods in a nutshell

I The Ki ’s are computed recursively

I N function evaluations are required at each time stepI Euler’s method if N = 1

I Parameters: pk , qk,i , ai

Butcher tableau:

p1 q1,1

p2 q2,1 q2,2

......

.... . .

pN−1 qN−1,1 qN−1,2 · · · qN−1,N−1

a1 a2 · · · aN−1 aN

I Consistent Runge-Kutta scheme: pk =∑

i qk,i

Selection of the parameters pk , qk,i , ai

Runge-Kutta of order N to be as accurate as a Taylor Series of order N

23

Second-order Runge-Kutta method: derivation

Formula:

xk+1 = xk +

a1 f (tk , xk)︸ ︷︷ ︸K1

+a2 f (tk + p1h, xk + q1,1f (tk , xk)h)︸ ︷︷ ︸K2

h

I 4 coefficients: p1, q1,1, a1, a2

I Express equivalence conditions with 2nd-order Taylor series method:

a1 + a2 = 1, a2 p1 =1

2, a2 q1,1 =

1

2: why?

I These conditions ensure: Local error of O(h3): global error of O(h2)I Consistency condition trivially satisfied: p1 = q1,1

I 4 parameters ↔ 3 conditions: 1 degree of freedom!

Example: what is the formula for a Runge-Kutta method of order 2 witha1 = 1

2?

24

Second-order Runge-Kutta method: application

Use a second-order Runge-Kutta method to solve the scalar ODEx ′(t) = − 1

2x(t) + exp(−t) for t ∈ [0, 1], with initial condition x(0) = 1and stepsize h = 10−1

Exact solution: x(t) = 3 exp(− 12 t)− 2 exp(−t)

h xk |x(tk ) − xk |0.0 1.0000 0

0.1 1.0440 2.16e-05

0.2 1.0770 3.72e-05

0.3 1.1004 4.77e-05

0.4 1.1155 5.39e-05

0.5 1.1233 5.66e-05

0.6 1.1248 5.63e-05

0.7 1.1208 5.35e-05

0.8 1.1123 4.88e-05

0.9 1.0997 4.25e-05

1.0 1.0838 3.49e-05

25

Fourth-order Runge-Kutta method: derivation

I Exact same idea: Express equivalence conditions with the4th-order Taylor series method!

Classical 4th-order Runge-Kutta formula:

xk+1 = xk +1

6[K1 + 2K2 + 2K3 + K4] h

with: K1 = f (tk , xk)

K2 = f

(tk +

1

2h, xk +

1

2h K1

)K3 = f

(tk +

1

2h, xk +

1

2h K2

)K4 = f (tk + h, xk + h K3)

Butcher tableau:

1

2

1

21

20

1

21 0 0 11

6

1

3

1

3

1

6

I Local error: O(h5): global error: O(h4)

26

Classical Runge-Kutta method: graphical illustration

27

Classical Runge-Kutta method: application

Use the classical Runge-Kutta method to solve the scalar ODEx ′(t) = − 1

2x(t) + exp(−t) for t ∈ [0, 1], with initial condition x(0) = 1and stepsizes h = 0.1 and h = 1

Exact solution: x(t) = 3 exp(− 12 t)− 2 exp(−t)

h xk ek xk ek

0.0 1.0000 0 1.0000 00.1 1.0440 1.10e-080.2 1.0770 2.01e-080.3 1.1004 2.74e-080.4 1.1155 3.33e-080.5 1.1233 3.79e-080.6 1.1248 4.14e-080.7 1.1208 4.40e-080.8 1.1123 4.57e-080.9 1.0997 4.67e-081.0 1.0838 4.71e-08 1.0829 8.89e-04

28

Fourth-Order Runge-Kutta method: vectorial ODEs

Use the classical Runge-Kutta method to compute the solution of thesecond-order ODE x ′′(t) + x(t)[3x ′(t)− 1] = exp(−t) at t = 1, for theinitial conditions x(0) = 0, x ′(0) = 0, and using a single step h = 1.

tk x1,k x2,k

0 0 0

0.5000 0.10765 0.39613

1.0000 0.35958 0.57207

I Exact same procedureas with scalar ODEs!

I Excellent accuracy, evenfor large steps

29

Heun’s predictor-corrector method: basics

Predictor/corrector basics:

I Predictor step:

φ(0) = f (tk , xk)

x(0)k+1 = xk + h φ(0)

I Corrector step:

φ(1) =1

2

[φ(0) + f (tk+1, x

(0)k+1)

]xk+1 = xk + h φ(1)

I Same formula as the 2nd-order Runge-Kutta method withparameters:

a1 = a2 =1

2, p1 = q1,1 = 1 : check!

I Error analysis: local error is O(h3); global error is O(h2)30

Heun’s predictor-corrector method: iterative corrector

Iterative corrector step:

φ(j+1) =1

2

[φ(0) + f (tk+1, x

(j)k+1)

]x

(j+1)k+1 = xk + h φ(j+1)

I May not improve things upfor a large step h

I Iterates may even diverge!

0. Initialization:Set j ← 0; Chooseεtol > 0

1. Predictor step:

φ(0) = f (tk , xk)

x(0)k+1 = xk + h φ(0)

2. Corrector step: repeat:

φ(j+1) =1

2

[φ(0) + f (tk+1, x

(j)k+1)

]x

(j+1)k+1 = xk + h φ(j+1)

UNTIL: |x (j+1)k+1 − x

(j)k+1| < εtol

31

Heun’s predictor-corrector method: application

Use Heun’s predictor-corrector method with tolerance εtol = 0.1 tocompute the solution of the scalar ODE x ′(t) = − 1

2x(t) + 4 exp(0.8t) att = 1, for the initial conditions x(0) = 2 and using a single step h = 1.

Exact solution: x(t) =40

13exp(0.8t)− 14

13exp(−0.5t)

j φ(j) x(j)1 |x (j)

1 − x(j−1)1 | e1 [%]

0 3.00000 5.00000

1 4.70108 6.70108 1.701 8.18%

2 4.27581 6.27581 0.425 1.31%

3 4.38213 6.38213 0.106 3.03%

4 4.35555 6.35555 0.027 2.60%

x(1) = 6.19463

32

Numerical solution of ODEs: final wordsAdaptive step-size control:

I Give the ability to vary the stepsize over the integration horizonI Typically based on the local truncation error

I E.g., adaptive Runge-Kutta methods

Stability:

When the effects of local errors do not accumulate catastrophically

I Unrelated to accuracy: A stable method can be very inaccurate!I Determined by: 1) differential equations, 2) solution method, 3)

stepsize h

Stiffness:When some terms in the solution vary much more rapidly than others

I Special care needed in the numerical integration to guaranteestability

I Usual solution methods (Runge-Kutta, etc.) often impractical!I Implicit solution techniques and multistep methods needed

33