PETE 301 PETROLEUM ENGINEERING

NUMERICAL METHODS LECTURE NOTES AND

STUDY GUIDE

Peter P. Valko

August 18, 2004

Contents

I Introduction 40.1 Course Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60.3 Course structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

II Tools 8

1 Introduction to Excel and VBA 91.1 Starting Excel, saving a file . . . . . . . . . . . . . . . . . . . . . . . . . . 91.2 Workbook, worksheet, views, printing . . . . . . . . . . . . . . . . . . . . 9

1.2.1 Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3 Spreadsheet basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.4 Graphing with Excel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.5 Numerical methods inside Excel . . . . . . . . . . . . . . . . . . . . . . . . 101.6 Database Management with Excel . . . . . . . . . . . . . . . . . . . . . . 111.7 Excel and the WWW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.8 Keyboard macros and what they cant do . . . . . . . . . . . . . . . . . . 111.9 VBA, variable, sub, function, module . . . . . . . . . . . . . . . . . . . . . 111.10 Structured Programming: Subs, Decisions and Loops . . . . . . . . . . . . 17

2 Mathematica 21

III Engineering models and numerical methods 23

3 Mathematical Modeling and Engineering Problem Solving 243.1 Conservation Laws and Engineering . . . . . . . . . . . . . . . . . . . . . 243.2 Accuracy and Precision, Approximations and Round-Off Errors . . . . . . 243.3 Error propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.4 Existence and uniqueness, Convergence criterion . . . . . . . . . . . . . . 263.5 Order of the method, rate of convergence, stability, sensitivity . . . . . . . 273.6 Classification of problems and methods . . . . . . . . . . . . . . . . . . . . 27

4 Methods related to single variable problems 294.1 Roots of equations and extrema of functions . . . . . . . . . . . . . . . . . 29

4.1.1 Bisection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.1.2 False position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1

4.1.3 Newtons method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.1.4 Optimization: Golden ratio search for a minimum . . . . . . . . . 32

4.2 Fixed-point iteration as a general framework for iterative processes . . . . 324.3 Representing, manipulating functions . . . . . . . . . . . . . . . . . . . . . 33

4.3.1 Numerical Differentiation of functions that can be evaluated every-where . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.3.2 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . 344.3.3 Simpsons 1/3 rule . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.4 Interpolation, Smoothing, Curve fit, Least squares . . . . . . . . . . . . . 364.4.1 Least Squares and its Numerical Aspects . . . . . . . . . . . . . . 37

4.5 Numerical solution of ordinary differential equations . . . . . . . . . . . . 394.5.1 Basic methods for solving ODE . . . . . . . . . . . . . . . . . . . . 39

5 Methods related to multi-variable problems 415.1 Matrices, vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.2 System of linear equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.3 LU Decomposition and Backsubstitution . . . . . . . . . . . . . . . . . . . 465.4 System of nonlinear equations, Newton-Raphson method . . . . . . . . . . 505.5 Multivariable extrema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.6 Solution of system of ordinary differential equations: RK4 . . . . . . . . . 545.7 Partial differential equations and reservoir simulation . . . . . . . . . . . . 55

5.7.1 Reservoir Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . 59

6 Integral Transforms, Spectral methods, inversion of the Laplace trans-form 616.1 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616.2 Laplace transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

6.2.1 Numerical inversion of the Laplace transform . . . . . . . . . . . . 61

IV Appendix 64

7 Study guide 657.1 Be able to ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

8 Assignments, test problems 688.1 Error propagation: borehole volume . . . . . . . . . . . . . . . . . . . . . 688.2 Error propagation: one-gallon cube . . . . . . . . . . . . . . . . . . . . . . 688.3 Error propagation: barrel . . . . . . . . . . . . . . . . . . . . . . . . . . . 688.4 Error propagation: hydrocarbon volume . . . . . . . . . . . . . . . . . . . 688.5 Root finding; Newton iteration: z factor . . . . . . . . . . . . . . . . . . . 698.6 Root finding: solution of the PR equation of state . . . . . . . . . . . . . 698.7 Root finding: heat balance . . . . . . . . . . . . . . . . . . . . . . . . . . . 708.8 Root finding: friction factor . . . . . . . . . . . . . . . . . . . . . . . . . . 708.9 Root finding: Well deliverability . . . . . . . . . . . . . . . . . . . . . . . 718.10 Root finding: isoterm flash . . . . . . . . . . . . . . . . . . . . . . . . . . 718.11 Optimization: Maximizing Net Present Value . . . . . . . . . . . . . . . . 728.12 Integration of a function: PV . . . . . . . . . . . . . . . . . . . . . . . . . 738.13 Numerical integration of discrete data: pore volume . . . . . . . . . . . . 73

2

8.14 Straight-line: formation volume factor model 1 . . . . . . . . . . . . . . . 748.15 Straight-line: formation volume factor model 2 . . . . . . . . . . . . . . . 748.16 Straight-line: gas in place . . . . . . . . . . . . . . . . . . . . . . . . . . . 758.17 Straight-line: Flow-After-Flow Test (IPR) . . . . . . . . . . . . . . . . . . 758.18 Nonlinear least squares: oil viscosity as a function of pressure and temper-

ature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768.19 ODE: Production rate decline from a differential equation . . . . . . . . . 768.20 ODE: pressure versus depth in a shut-in well . . . . . . . . . . . . . . . . 778.21 Meaning of matrices and vectors: chemical component balance . . . . . . 778.22 System of linear equations: mixing . . . . . . . . . . . . . . . . . . . . . . 778.23 System of linear equations with many right-hand-sides: log interpretation 788.24 Nonlinear System of Equations: Chemical Equilibrium of Methan Combus-

tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 798.25 Units: Darcys law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

9 Tables 81

3

Part I

Introduction

4

0.1 Course Objectives

Learn the underlying principles behind some commonly used numerical methods. Understand the route from arithmetic and calculus to numerical methods.

Understand the basic concepts including uniqueness, approximation, error, con-vergence, and stability.

Learn how to program a numerical method using modern programming concepts Problem solving steps

Structured programming

Use of subroutines, modules

Learn how to make use of Visual Basic for Applications.

Learn how to use an integrated program development system providing debug-ging facilities.

Recognize the main features of a mathematical problem arising in a technical con-text.

Single- or multi-variable

Linear or non-linear

Involves algebraic, differential or integral equations

Direct or iterative technique should be applied

Explicit or implicit scheme is appropriate

Develop skills to formulate a problem starting from physical principles. Start with conservation principles

Include material properties

Add problem specific conditions

Learn how to use various resources in solving a complex problem

Use of software systems; Subroutine libraries, modules

Internet resources

Improve ability to critically analyze numerical results Improve documentation skills

5

0.2 Resources

The concepts and methods of technical computing are summarized in the excellent bookof Chapra and Canale: Numerical Methods for Engineers (CC). In order to make efficientuse of our time, we request you to read the material before the lecture. Any similar bookcan be used, however.

The Visual Basic for Applications (Excel) program development environment will beavailable in the lab. Whenever you are asked to solve a problem by computer, we requestan Excel Workbook with Visual Basic source code and results. Excel and VBA for Excelhas a vast literature.

We will give you the opportunity to experiment with many of the numerical methodsusing a Living Textbook, developed particularly for this course. To make full use ofthe Living Textbook you will need a software system called Mathematica. This course,however, does not require you to learn how to program in Mathematica. In fact, to runany portion of the Living Textbook you will need minimum knowledge which can beacquired in 5 minutes.

Many of the specific petroleum engineering problems also will be provided in the formof Living Textbook. You may experiment with them and may use the results for checkingyour Visual Basic for Applications (VBA) program results.

Basic References

1) S.C. Chapra and R. P. Canale : Numerical Methods for Engineers, McGraw Hill,any edition, any year (or any similar title)

2) Either of the following (or any similar title)a) Larsen: Engineering with Excel, Prentice Hall, 2002b) Liengme: A guide to Microsoft Excel 2002 for scientists and engineers, Butterworth

Heinemann, 2002c) Bloch : Excel for Engineers and Scientists, 2002

3) All of your Petroleum Engineering Books

Special thanks go to L. Piper, B. Maggard, R. Archer, R. Wattenbarger .

6

0.3 Course structure

1. Engineering problem solving tools and programming

(a) Engineering problem solving approaches, software tools

(b) Excel

(c) VBA

(d) Structured and Modular Programming

2. Basics of numerical methods

(a) Taylor polynomial, Numerical errors, Error propagation

(b) Iteration, Convergence, Order, Stability, Sensitivity

(c) Classification of problems and methods

3. Methods related to single variable problems

(a) Roots of equations and extrema of functions Bisection; False position; Newton;golden ratio search

(b) Numerical differentiation and integration Finite difference; Trapezoid rule;Simpsons rule

(c) Interpolation, Smoothing, Curve fit, Least squares

(d) Numerical solution of ordinary differential equations

4. Methods related to multi-variable problems

(a) Matrices, vectors

(b) System of linear equations

(c) Gauss, Gauss-Jordan, LU decomposition Special cases, Iterative methods

(d) Multivariable nonlinear equations: Newton-Raphson

(e) Multivariable extrema: Nelder-Mead simplex, Fully nonlinear least squares

5. Partial differential equations and reservoir simulation

(a) Classification and Numerical Solution of PDEs

(b) Transient solution of the diffusivity equation

(c) Reservoir simulation

6. Transformation methods

(a) Spectral methods

(b) Laplace transform inversion

7

Part II

Tools

8

Chapter 1

Introduction to Excel and VBA

1.1 Starting Excel, saving a file

To create a new workbook: On the File menu, click New, and then click Blank Workbookon the New Workbook task pane.

To save a workbook:On the File menu, click Save As.In the File name box, type a new name for the workbook.In the Save as type list, click a file format (if you want other than .xls)(In general, it is useful to click the right mouse button. Most of the time it will offer amenu item you really want.)

1.2 Workbook, worksheet, views, printing

1.2.1 Review

A Microsoft Excel workbook is a file that contains one or more worksheets, which youcan use to organize various kinds of related information. You can enter and edit data onseveral worksheets simultaneously and perform calculations based on data from more thanone worksheet. When you create a chart, you can place the chart on the same worksheetas its related data or on a separate chart sheet.

To view and scroll independently in different parts of a worksheet, you can split aworksheet horizontally and vertically into separate panes. Splitting a worksheet intopanes allows you to view different parts of the same worksheet side by side and is useful,for example, when you want to paste data between different areas of a large worksheet.

To see how it will look like in print: On the View menu, click Page Break Preview.Before you do any work, make sure that you macros are allowed (security level is low)

and you can save an excel spreadsheet to your own disk.

1.3 Spreadsheet basics

The difference between relative and absolute references is the key concept to understand:

9

Relative references A relative cell reference in a formula, such as A1, is based on therelative position of the cell that contains the formula and the cell the reference refers to. Ifthe position of the cell that contains the formula changes, the reference is changed. If youcopy the formula across rows or down columns, the reference automatically adjusts. Bydefault, new formulas use relative references. For example, if you copy a relative referencein cell B2 to cell B3, it automatically adjusts from =A1 to =A2.

Absolute references An absolute cell reference in a formula, such as $A$1, always referto a cell in a specific location. If the position of the cell that contains the formula changes,the absolute reference remains the same. If you copy the formula across rows or downcolumns, the absolute reference does not adjust. By default, new formulas use relativereferences, and you need to switch them to absolute references. For example, if you copya absolute reference in cell B2 to cell B3, it stays the same in both cells $A$1.

1.4 Graphing with Excel

Import a text file Click the cell where you want to put the data from the text file. Toensure that the external data does not replace existing data, make sure that the worksheethas no data below or to the right of the cell you click. On the Data menu, point to ImportExternal Data, and then click Import Data. In the Files of type box, click Text Files.Check out also: On the Data menu: Text to Columns.

Creating an XY (Scatter) Plot You can create a chart on its own sheet or as anembedded object on a worksheet. You can also publish a chart on a Web page. To createa chart, you must first enter the data for the chart on the worksheet.

Arrange your data in columns, with x values in the first column and corresponding yvalues and/or bubble size values in adjacent columns.

Then select that data and use the Chart Wizard to step through the process of choosingthe chart type XY and the various chart options, or use the Chart toolbar to create abasic chart that you can format later.

Best-fit equations may be calculated for chart data via the Add Trendline dialog.Trendline equations are displayed by checking the correct box under the Options tab

Errors bars can be added to a chart by left clicking on a data series, and accessing theFormat Data Series dialog box

Editing an Existing Graph Check out.

Creating Graphs with Multiple Curves Check out.

More graphs types Check out.

1.5 Numerical methods inside Excel

Goal Seek The Goal Seek dialog box can be used to solve for roots of a single equation.An initial guess is needed. Goal Seek performs calculations which improve the solutionaccuracy.

10

Solver The Solver can be used to solve both linear and nonlinear optimization problems,i.e. maximize or minimize an objective function subject to known constraints.

The Solver Parameters dialog box refers to the following: Cell for the objective functionvalue (which is to be maximized or minimized) Cells for the the values of the indepen-dent (decision) variables If the values of the independent variables are subject to theconstraints, then cells constrained.

1.6 Database Management with Excel

Check out Search, Filter and Sort in the Data menu.

1.7 Excel and the WWW

Excel files may be converted to HTML (web) documents. The HTML Wizard guides youthrough the process.

1.8 Keyboard macros and what they cant do

If you perform a task repeatedly in Microsoft Excel, you can automate the task with amacro. A macro is a series of commands and functions that are stored in a MicrosoftVisual Basic module and can be run whenever you need to perform the task.

For example, if you often enter long text strings in cells, you can create a macro toformat those cells so that the text wraps.

Recording keyboard macros: When you record a macro, Excel stores informationabout each step you take as you perform a series of commands. You then run the macroto repeat, or play back, the commands. If you make a mistake when you record themacro, corrections you make are also recorded. Visual Basic stores each macro in a newmodule attached to a workbook.

Making a macro easy to run: You can run a macro by choosing it from a list in theMacro dialog box. To make a macro run whenever you click a particular button or pressa particular key combination, you can assign the macro to a toolbar button, a keyboardshortcut, or a graphic object on a worksheet.

Keyboard macros are to replay your keystrokes. If you want to program your ownalgorithms, you need VBA programming.

1.9 VBA, variable, sub, function, module

Visual Basic for Applications The programming language of this course is Visual Basicfor Applications (VBA), which is included with the Excel spreadsheet application. Thiscombination of programming language and spreadsheet has many powerful features, whichmake it an excellent tool for problem solving.

1. A modern, structured programming language, with many intrinsic procedures

(a) Mathematics

(b) Data type conversion

11

(c) String manipulation

(d) File system

2. Ability to control all of Excels features through the Microsoft Excel Objects, in-cluding

(a) Data management

(b) Graphics

(c) Solver (for optimization and root finding)

3. Ability to add custom functions which can be used like built-in Excel functions

4. Capabilities allowing the development of event driven, user friendly programs througha variety of interface tools

(a) Dialog and input boxes

(b) Custom user forms with spinners,checkboxes, etc.

(c) Custom menus and toolbars

5. The ability to extend the language through object oriented programming with theuse of Class Modules

Example: VBA011 program (After Hahn): If a stone is thrown vertically upwardwith an initial speed u, its vertical displacement s after a time t has elapsed is given bya simple formula. In this formula, g is the acceleration due to gravity. Air resistance hasbeen ignored. We would like to compute the value of s, given u and t. Note that we arenot concerned here with how to derive the formula, rather we want to compute its value.The logical preparation of this program is as follows:

1. Place the values of g, u and t into variables which can be used for calculations

2. Compute the value of s according to the formula.

3. Output the value of s which is the result

4. Stop

This plan may seem trivial to you, and a waste of time writing down. Yet you wouldbe surprised how many beginners, preferring to rush straight to the computer, try toprogram step 2 before step 1.

It is well worth developing the mental discipline of planning your program first-if penand paper turns you off why not use your word processor? You could even enter the planas comment lines in the program.

Option Explicit

Const g As Double = 9.81 gravitational acceleration, m/s^2

Sub VBA011() vertical motion under gravity

Dim s As Double vertical displacement, m

Dim t As Double time, s

Dim u As Double initial velocity, m/s

12

u = 30

t = 6

s = u * t - (g / 2) * t ^ 2

MsgBox "s = " & CStr(s) &" meters." s is converted to a string

End Sub

To make this program run do the following:

1. From a new Excel workbook, use the Tools, Macros, Visual Basic Editor menu choiceto start the VBA integrated development environment (includes editor, compiler,help and debugger).

2. In Project window of VBA, right click on your new Excel workbook (usuallyVBAProject (Book1)) and insert a new module.

3. Cut and paste the text of VBA011, shown in the above text box into the newlycreated module. Note that the Option Explicit is not part of the program, but isa module option. Note the colors of comments (green), language elements (blue),and names (black).

4. Left click to place the cursor anywhere inside the subroutine, and use the menu tochoose Run, Run Sub/User Form.

5. If you have questions on the language elements, shown in blue, place the cursor onthe word and hit the F1 key.

Some remarks:

1. VBA performs automatic syntax checking and formatting of code.

2. The single quote character () denotes the beginning of a comment.

3. In PETE 301 every module should require explicit variable typing by using OptionExplicit module option.

4. The strange way of declaring g makes it a named constant, (its value cannot bechanged). The fact that it is placed outside the subroutine makes it valid in thewhole module (scope: module).

5. There is no input. All needed data are hard wired into this simple subroutine.(Later we will learn how to read from the worksheets.)

6. Output is done using a message box. Later we learn how to put results into theworksheets.

7. This module contains only one subroutine (macro). In fact many of them can beplaced into one module.

Built-in (or intrinsic) functionsVisual Basic has many built-in functions. You will often need the math functions:

Abs, Atn, Cos, Exp, Fix, Int, Rnd, Sgn, Sin, Sqr, Tan.Log is natural-log. If you wish ten-based logarithm, use Log(x)/Log(10).

13

Example: Sin functionReturns a Double specifying the sine of an angle. Syntax: Sin(number)The required number argument is a Double or any valid numeric expression that expressesan angle in radians.Remarks: The Sin function takes an angle and returns the ratio of two sides of a righttriangle. The ratio is the length of the side opposite the angle divided by the length of thehypotenuse. The result lies in the range -1 to 1. To convert degrees to radians, multiplydegrees by pi/180. To convert radians to degrees, multiply radians by 180/pi.

Those functions are useful, but they are not customizable. Even if you find a Visual Basicfunction that is this close to what you need, you cant worm your way into the innardsof Visual Basic to change the way it works. You can, however, create a function of yourown.

A great number of other functions are also available. For instance, both Excel and VisualBasic have functions that return a random number between 0 and 1. The Excel functionis named Rand(), and the Visual Basic function is named Rnd(). You can use the Excelfunction in a worksheet cell and (with some tricks) also in your VBA macro, but you canuse the Visual Basic function only in a macro.

Data Types The following table shows the supported data types

Byte 1 byte 0 to 255Boolean 2 bytes True or FalseInteger 2 bytes -32,768 to 32,767

Long(long integer) 4 bytes -2,147,483,648 to 2,147,483,647Single (single floating-point) 4 bytes -3.402823E38 to 3.402823E38Double (double floating-point) 8 bytes -1.E308 to 1.8E308

String (variable-length) 10+length 0 to appr 2 billionVariant (with numbers) 16 bytes see range of a Double

Arrays of any data type require 20 bytes of memory plus 4 bytes for each array dimensionplus the number of bytes occupied by the data itself.

Dim Statement: Declares variables and allocates storage space.

Dim NumberOfEmployees As Integer

Dim x As Double, y as Double

When Option Explicit appears in a module, (as it should in PETE301) you must ex-plicitly declare all variables using the Dim, Private, Public, ReDim, or Static statements.If you attempt to use an undeclared variable name, an error occurs at compile time.

Note:

Dim a,b As Integer

will declare b as an Integer, but a will be the default (Variant). You have to write

Dim a As Integer, b As Integer

14

if you wish to declare both as integer.

Numerics VBA makes a clear distinction between integer and real. Integers are used forcounting things. Real numbers include a fractional part. Real numbers stored as integersare rounded to the nearest integer.

Dim n As Integer, m As Double

n = 4/3 !result is 1

n = 5/3 !result is 2

n = 7/2 !result is 2

n = 5/2 !result is 4

m = 5/2 !result is 2.5

You should be aware of rules regarding priorities. For instance a frequent error is towrite

z = p*v/R*T

instead of the correct:

z = p*v/(R*T)

Strings To convert any other value to a string, use the Cstr intrinsic function.

Objects,input and outputThe structure With/End With and the operator point can be understood if you

are familiar with the concept of object.ThisWorkbook is a hierarchical object. It has several Workseets (a whole collection

of them). One of them is called Sheet2. The following example shows how to read anumber into variable a from cell A4 of Sheet2:

With ThisWorkbook.Worksheets("Sheet2")

a =.Cells(1, 4)

End With

This will output date and time into cells A4 and A5 of Sheet2:

Sub VBA014()

Dim d As String, t As String

d = Date() Intrinsic function

t = Time() Intrinsic function

Display the result

With ThisWorkbook.Worksheets("Sheet2")

.Cells(1, 4)= d

.Cells(1, 5)= t

End With

End Sub

(Objects have properties and methods associated with them. For instance, when youare formatting a cell, you change some of its properties using some methods available.)

Advanced topics: Variable scope, lifetime, visibility The following key words areimportant: Static, Public, Private.

The scope of a variable is from where it can be reached/altered. Variables declaredin a procedure can be used only within the procedure (local variable for the procedure).

15

Variables declared on the module level can be used in the entire modul in any of theprocedures within the module (global variable for the module).

Static variables keep their value between two occasions of entering into their scope. Inother words, their lifetime is not limited to one call. In general, a procedure-level variablelooses its value between two calls, unless it is declared Static.

Both module-level variables and all procedures are Public, by default, meaning thatthey are visible even from other modules (or even other applications.) You can makethem Private, and then they are visible only within the module.

We will often use module level variables (globals) to simplify engineering problemsolving, when some data are needed in various parts of the program.

Advanced topics: Argument passing by Value or by Reference All arguments arepassed to procedures by reference, unless you specify otherwise. This is efficient becauseall arguments passed by reference take the same amount of time to pass and the sameamount of space (4 bytes) within a procedure regardless of the arguments data type.

(You can pass an argument by value if you include the ByVal keyword in the pro-cedures declaration. Arguments passed by value consume from 2 16 bytes within theprocedure, depending on the arguments data type. Larger data types take slightly longerto pass by value than smaller ones.)

Errors, Debugging and Testing Compilation errors are errors in syntax and construc-tion, like spelling mistakes, that are picked up by the compiler during compilation, theprocess whereby your program is translated into machine code. They are the most fre-quent type of error. The compiler prints messages, which may or may not be helpful,when it encounters such an error. Generally, there are three sorts of compiler errors:

1. Ordinary errors-the compiler will attempt to continue compilation after one or moreof these errors has occurred, e.g. missing End If statement

2. Fatal errors-the compiler will not attempt further compilation after detecting a fatalerror, e.g. Program too complicated-too many strings

3. Warnings-these are not strictly errors, but are intended to inform you that youhave done something unusual which might cause problems later, e.g. Expressionin If construct is constant or variable has not been assigned a value but used, orvariable has not been declared (if implicit typing is used)

Some typical errors:

g = 9,81 comma instead of decimal point

x = 1 + (2 * 3 unpaired parentheses

.Cell(2,1)=5 Wrong function name: Cell instead of Cells

If (a=b) Missing then

x=3

Else

x=4

End If

If a program compiles successfully, it will run. Errors occurring at this stage are calledrun-time errors, and are invariably fatal, i.e. the program crashes. An error message,such as

1. Floating point division by zero

16

2. Overflow

3. Negative argument in the function sqr()

is generated.Overflow is quite common. It occurs, for example, when an attempt is made to

compute a real expression which is too large, or when you divide by a previously notassigned variable, which might be almost zero, because it contains some trash.

Negative argument to Sqr or non-positive argument to Log is often the result of aprevious logical error.

At times a program will give numerical answers to a problem which appear inexplicablydifferent from what we know to be the correct mathematical solution. This can be dueto rounding error, which results from the finite precision available on the computer. , e.g.two or four bytes per variable, instead of an infinite number.

Run the following program :

Option Explicit

Sub VBA022()

Dim sumx As Double

sumx = 0.1

Do

sumx = sumx + 0.001

Debug.Print sumx

If (sumx = 0.2) Then Exit Do

Loop

End Sub

Watch the Immediate Window! You will find that you need to crash the program tostop, e.g. with ctrl-break. The reason is: x never has the value 0.2 exactly, because ofrounding error. In fact, x misses the value of 0.2 by about 109.Now try this:

If (abs(sumx - 0.2) < 1e-6) Then Exit Do

Programming is not about avoiding errors from the beginning. In fact that wouldbe almost impossible. Real programming is about localizing your errors by debugging andcorrecting them. Check your work 100+1 times. Use test problems. Print out intermediateresults. Investigate limiting cases.

Message Boxes, Custom Dialogue Boxes

1.10 Structured Programming: Subs, Decisions andLoops

Using a Subroutine Subroutines are an important component of top down programdesign. Subroutines allow the programmer to build and test program procedures thatperform a single, well defined task. This can make complex programs easier to design,test, and debug.

All executable code must be contained in a procedure. Procedures cant be nestedwithin other procedures.

17

Example Let in your Excel workbook, Sheet3 contain the following: Cells A1 and B1should contain the X and Y coordinates of a point. Cells A2 and B2 should contain theX and Y coordinates of a second point.This program will read the coordinates of the two points and calculate the slope andintercept of the unique straight line defined by the two points, and output the results in amessage box. These two subroutines should be contained in a brand new module withinyour VBA project. Because the My Line subroutine has arguments, it cannot be executeddirectly, like the previously discussed subroutines, by using the VBA Run menu.

Option Explicit

Private Intercept As Double

Sub VBA015()

Dim X1 As Double, X2 As Double, Y1 As Double, Y2 As Double

Dim Slope As Double

Get the two points from the worksheet, "Sheet3":

With ThisWorkbook.Worksheets("Sheet3")

X1 = .Cells(1, 1)

Y1 = .Cells(1, 2)

X2 = .Cells(2, 1)

Y2 = .Cells(2, 2)

End With

Calculate the Slope and Intercept

Call My_Line(X1, Y1, X2, Y2, Slope)

Display the result

With ThisWorkbook.Worksheets("Sheet3")

.Cells(3, 1)=" Slope:"

.Cells(3, 2)= Slope

.Cells(4, 1)= "Intercept:"

.Cells(4, 2)= Intercept

End With

End Sub

A subroutine used inside:

Sub My_Line(x As Double,y As Double,xx As Double,yy As Double,m As Double)

m = (yy - y) / (xx - x)

Intercept = y - m * x

End Sub

Note: Watch for scope of the various variables! Intercept is a module level variable, butSlope is not. Intercept cannot be seen from other modules.

The If/Then/Else Decision StructureExample (after Hahn) Most banks offer differential interest rates-more for the rich, less

for the poor. (The Else part may be missing and you may create more complex structurewith Else If.)

If balance < 3000 Then

apr = 0.03

Else

apr = 0.06

End If

18

Loops: the heart of programming math

Countable: Do the block for i = 1, then i = 2 , etc.:

For i = 1 To 10

"block..."

Next i

Count down: Do it for j = 100, then j = 90 etc.:

For i = 100 To 50 Step -10

block

Next i

Forever: Loop forever unless If stops it:

Do

If "logical_expression..." Then Exit Do

"block..."

Loop

Condition controlled: Do while logical expression is true:

While "logical_expression..."

"block..."

Wend

Select Case structure makes your code readable Use the Select Case statementas an alternative to using ElseIf in If...Then...Else statements when comparing one expres-sion to several different values. While If...Then...Else statements can evaluate a differentexpression for each ElseIf statement, the Select Case statement evaluates an expressiononly once, at the top of the control structure.

Select Case performance

Case 1

bonus = salary * 0.1

Case 2, 3

bonus = salary * 0.09

Case Is > 3

bonus = 100

Case Else

bonus = 0

End Select

Arrays Arrays are declared the same way as other variables, using the Dim, Static,Private, or Public statements. The difference between scalar variables (those that arentarrays) and array variables is that you generally must specify the size of the array. Anarray whose size is specified is a fixed-size array. An array whose size can be changedwhile a program is running is a dynamic array.

Whether an array is indexed from 0 or 1 depends on the setting of the Option Basestatement. If Option Base 1 is not specified, all array indexes begin at zero.

Declaring a fixed array In the following line of code, a fixed-size array is declared asan Integer array having 10 rows and 10 columns:

19

Option Base 1

Dim MyArray(10, 10) As Integer

The first argument represents the rows; the second argument represents the columns.As with any other variable declaration, unless you specify a data type for the array,

the data type of the elements in a declared array is Variant. To write code accordingto PETE301 conventions, explicitly declare your arrays to be of a data type other thanVariant.

Arrays can also be dimensioned on-the fly, using the ReDim command. It is usedfor dynamic allocation of arrays on the procedure level.

A useful related function is UBound(). It returns the largest available subscript forthe indicated dimension of an array.

20

Chapter 2

Mathematica

Many of the concepts, methods and applications of this course are now programmedas Living Textbook. Using the software Mathematica any section (notebook) of theLiving Textbook can be read on the screen and/or run as a program. The particularexamples can be modified and rerun providing a unique possibility for experimenting withnumerical methods and petroleum engineering applications.

Mathematica is a complete environment for calculating and communicating in scienceand technology, delivering computing power to over one million people around the world.Breakthrough new features such as an innovative typesetting system that can do mathnow make Mathematica even easier to use. Known for delivering quick, accurate numericand symbolic solutions, Mathematica is ideal for creating interactive scientific papers,technical reports, presentations, and courseware that include text, active formulas, two-and three-dimensional graphics, and customizable buttons and palettes. A growing libraryof application packages provides specialized capabilities in areas such as engineering, fi-nance, statistics, data analysis, optics, astronomy, and fuzzy logic. The associated website is at http://www.wolfram.com.

While it is very useful and therefore strongly recommended that you should becomefamiliar with the basics of the fully integrated environment for technical computing, Math-ematica, this course is not intended to teach programming in Mathematica. In fact, tomake use of the Living Textbook you need to know only a very few things:

The basic concept is the notebook. A Mathematica notebook is something between aword processor-type document and a Fortran-type program. When a notebook is runningthe results of the computations are appearing immediately after the input lines.

To start a Mathematica notebook in Windows: Double click on the notebook.To see details: A Mathematica notebook consists of a hierarchical structure of cells.

You can think of the main cells as Chapters. Every Chapter contains sections, everysection contains subsections, etc. At start usually you will see only the main cell names,that is the Chapter titles. Then you can open or close any Chapter by clicking the littletriangle at the left.

To run the whole notebook: Select Kernel, Evaluate notebook from the menu. If thesystem asks a question about evaluating initialization cells, respond clicking yes. Tosee what is happening use the mouse and the scroll bar or the PgUp and PgDn keys.

To make a small change and rerun an example: Go to the item you want to changewith the mouse cursor. Use the Del or Backspace keys and type what you want. Once

21

you are ready, select Kernel, Evaluate Cell . (For those interested in shortcuts: you canuse the Enter key on the numeric keypad, as well.)

Mathematica notebooks may contain a large amount of graphics. Since printing graph-ics may overload the system and especially the printer, we ask you not to print the note-books, unless it is necessary. Before printing make sure you deleted unnecessary parts.

Living Textbook ContentLTS021 Taylor Series Expansion, Remainder Term and Truncation ErrorLTS031 Root-FindingLTS032 More about iterationsLTS033 Extrema of single variable functionLTS041 Numerical Integration of functionsLTS061 Ordinary Differential Equations Basic ConceptsLTS071 Numerical Solution of the ODE Initial Value ProblemLTS072 Reservoir Material-BalanceLTS081 Vectors, Matrices, Linear system of EquationsLTS091 Direct Solution of Systems of Linear EquationLTS092 Systems of Linear Equations: Gauss-Siedel Iterative MethodLTS101 Solution of System of Non-Linear EquationsLTS102 Minimization of a Multi-Variable FunctionLTS121 Oil-Water 2D IMPESLTS131 Spectral Methods: Discrete Fourier Transform

22

Part III

Engineering models andnumerical methods

23

Chapter 3

Mathematical Modeling andEngineering Problem Solving

3.1 Conservation Laws and Engineering

3.2 Accuracy and Precision, Approximations and Round-Off Errors

Precision is related to repeatability or to the number of digits obtained. Accuracy isrelated to how near we are to the true value. Obviously, accuracy is more importantthan precision.

In analysis of the numerical methods we often start with the Taylor series expansion:

f(x+ h) = f(x) + h f (x)/1! + h2 f (x)/2! + . . .

Several terms on the right hand side will enter the formula of the method. The firstterm left out is the leading error term. The additional terms are usually neglected in theanalysis.

Truncation error: difference between true result (for actual input) and the approximateresult produced by given algorithm using exact arithmetic. in general, it is estimated bythe leading error term.

Rounding error: difference between result produced by given algorithm using exact arith-metic and result produced by same algorithm using limited precision arithmetic.

Computational error is sum of truncation error and rounding error, but one of these usu-ally dominates.

Since usually input data are in error and we do a lot of steps, the most important issueis: How does the error propagate?

Measures of the error in approximating a number with true value x by an approxima-tion x:

24

True Error, Et = x x

Relative Error, xxx

Percentage Relative Error, xxx 100.

Of course, in general, the true value will not be known so these error measurementscannot be carried out rigorously. However, in many cases upper bounds for the valuesof these errors can be found which give us enough information to put the result intoperspective.

3.3 Error propagation

A new perspective: The result of a computation is a multi-variable function of the inputvalues.

For simplicity, consider two variables. The first order Taylor approximation:

f(x+x, y +y) = f(x, y) +f

xx+

f

yy + . . .

Rearranging and introducing f = f(x + x, y + y) f(x, y) we obtain the basicrelatiuon of error calculus:

f = fxx+ f

yy + . . .

In this error calculus x means always a positive quantity, it represents the maximumpossible value of the actual error.

Derive what are the two partial derivatives of

f = x+ yf = x yf = x yf = x/y

From there we obtain, that for summation/substraction the absolute errors are added.For multiplication and division the relative errors are added.

Example: Find the error in a laboratory determination of the gas z-factor, given that:

p = 3245 psi p = 3 psi (meaning: 3 psi)v = 1.977 ft3 v = 0.003 ft3

n = 1 lbmole n = 0.001 lbmoleT = 739.67 oR T = 0.03 oR

Differentiating z = pv/(nRT ) we obtain

25

z = vnRT p+ pnRT v + pvn2RT n+ pvnRT 2 T

A simpler form isz = z

(pp +

vv +

nn +

TT

)= 2.8 103

(Does the result have any dimension? Why did we not use more decimal figures? Whydo we use absolute value? Why do we not use absolute value in the p term? Does itmatter that T is in the denominator?

(Note: lb-mole is a rather obsolete unit: it is 453.592 mol, or sometimes the massor weight of it; R = 10.73 psi ft3/(lb mole oR ) is the universal gas constant in theEnglish or field system of units)

Error propagation in a numerical methodWhat happens with the error inherited?The stability of a computational algorithm : errors (originated from truncation and round-off) are not amplified from step to step but are rather kept under control

Ill-conditioned problem A problem is ill-conditioned if results are very sensitive toinput data (regardless of the method used)

3.4 Existence and uniqueness, Convergence criterion

Review the following concepts:

Existence and uniqueness of solutionGraphical versus numerical solutionAnalytical versus numerical method to get the results numericallyDirect vs iterative (explicit vs implicit)ConvergenceMathematically a series converges to a limit if the difference of the n-th term and the limitcan be made smaller than any selected positive epsilon by selecting a large enoughn.

Typically in many numerical problems, a sequence of approximate values will be ob-tained which, hopefully, converge towards the desired solution:

x0, x1, x2, x3, x4, ......

If they appear to be converging, then a measure something like the percentage relativeerror is used:

a =xi+1 xixi+1

100

(If, however, the x can be near-zero, an absolute error criterion is better. Why?)If this error approximation a drops below some user-specified tolerance, then the

process is stopped. (The results should be checked if possible, in some other independentway for the required accuracy.)

26

3.5 Order of the method, rate of convergence, stabil-ity, sensitivity

Since most methods are derived from some Taylor expansion considerations, the theoret-ical order of a method is related to the truncation of the Taylor series. Examples will begiven in the numerical differentiation section. Broadly speaking: the order of the methodis the exponent of the variable step-size in the leading error term.

Total error: truncation and round-off

Error propagation in a numerical method is as important as the magnitude of the error.

Practical rate of convergence is often determined by numerical experimentation. Forthis purpose we need simple examples with known exact solutions. If we plot estimatesof errors on a log-log plot, we will see that the error is decreasing with step-size, and therelation appears to be a straight line.

If, for instance, the for every log cycle of the step-size the error changes two log cycles,the rate is quadratic.

Often instead of the effect of step-size we are more interested in the effect of numberof iterations.

Trade-Off Regarding step-size The point is, that smaller step-size provides largeraccuracy, but after a while the increased number of calculations bring more round-offerrors. Therefore there exists always an optimum.

Trade-Off regarding complexity is very similar. It is better to use a higher ordermethod, then a lower order method, up to a certain point. Too much complexity, however,may backfire the same way as to many small steps.

An algorithm is stable if errors (originated from truncation and round-off) are notamplified from step to step but are rather kept under control. The opposite is unstable.

A problem is well-conditioned, if results are not very sensitive to input data (at leastif an appropriate method is used). The opposite is called ill-conditioned.

3.6 Classification of problems and methods

Single variable, multi variable (our main classification) Other can be linear vs. nonlinear,direct vs. iterative, explicit vs. implicit.

Single variable methods: Root of a single equation Minimum of function Manipula-tion of functions (differentiation, integration) functions given in form of expressionor algorithm discrete data point (smoothing, curve fitting + diff and int) OrdinaryDifferential Equation (ODE)

27

Multi variable problems: Linear Algebra, Matrices, vectors Systems of linear equa-tions direct, special, iterative Nonlinear System of nonlinear equations Minimum ofmulti variable function (general and least squares) System of ODE Partial Differen-tial Equations Reservoir simulation

Other: Transformation methods

28

Chapter 4

Methods related to singlevariable problems

4.1 Roots of equations and extrema of functions

Single and multiple roots, bracketing versus open methodsNonlinear equation may have multiple root, where both function and derivative are zero.

4.1.1 Bisection

In the bisection method first the user estimates, approximately, the position of the rootproviding a lower- and upper-bound bracketing the root. If the interval [a, b] is a bracket,then f(a)f(b) 0.

A first estimate of the root is then computed as the mid-point between the two bounds.

x = (a+ b)/2

We can improve on this estimate by determining which side of x the root lies and thenmoving the bracket accordingly. To do this, we have to overwrite either a or b by thecurrent mid-point, x, depending on which previous function value has the same sign asf(x). At this point we have a new bracket and can continue with the next iteration.

Each iteration halves (bisects) the bracket (and therefore halves the maximum possibleerror) hence the term bisection. We can calculate a-priori the necessary number of stepsto reach a certain fraction of the initial bracket.

4.1.2 False position

The search starts with an interval [a, b]. It is assumed that the function changes sign inthe interval, f(a)f(b) 0 as in the case of the bisection method.

Writing the equation of the line passing through the two points (a, f(a)) and (b, f(b))

y f(b) = f(b) f(a)b a (x b)

29

and requiring that y should be zero, we obtain for the next approximation of the root:

c = b f(b)f(b)f(a)

ba

It is easily seen, that c is inside the interval [a; b]. Now we calculate f(c) and overwriteeither a or b with c, depending on which function value has the same sign as f(c). Afterone step, again we have a bracket, but it is smaller than the initial one.

(Note:

c = a f(a)f(a)f(b)

ab

is the same.)

During the process the size of the bracket does not tend to zero. The criterion to stopthe iteration is the following: the deviation of the new location c from any of the oldlocations a or b should be less than a specified tolerance.

4.1.3 Newtons method

The method (also called Newton-Raphson method) may be used to solve a general equa-tion f(x) = 0, provided the left-hand-side can be differentiated analytically.

If an initial guess is known, we can calculate the function and its derivative at x0.Writing the equation of the tangent line:

y f(x0) = f (x0)(x0 x)

Substituting y = 0 we find that the line crosses the x axis at

x1 = x0 f(x0)f (x0)

This is the iteration formula of the Newtons method. The obtained x1 can be thenrenamed to x0 and hence we can continue the iterations.

Example. Suppose thatf(x) = x3 + x 3

Thenf (x) = 3x2 + 1

The following program starts the iteration with x = 2. It uses two internal functions:F for f(x) and DF for its derivative: f (x). It stops either when the absolute value of thestep is less than 1E-6, or after 20 iterations. Note that there are two conditions that willstop the While loop: either convergence, or the completion of 20 iterations. Otherwisethe program could run indefinitely.

30

Option Explicit

Globals Group 1: Coefficients

Dim c3 As Double, c2 As Double, c1 As Double, c0 As Double

Globals Group 2: Iteration control parameters

Dim MaxIts As Integer, Eps As Double, x0 As Double

left-hand-side function

Function F(x As Double) As Double

F = c3 * x ^ 3 + c2 * x ^ 2 + c1 * x + c0

End Function

derivative

Function DF(x As Double) As Double

DF = 3 * c3 * x ^ 2 + 2 * c2 * x + c1

End Function

Main sub

Sub Newton(x As Double, fx As Double, Converged As Boolean)

Dim Its As Integer, d As Double

Initialize

Its = 0

Converged = False

x = x0

Loop

While (Not Converged And Its < MaxIts)

d = -F(x) / DF(x)

x = x + d

Its = Its + 1

Converged = Abs(d) < Eps

Wend

Clean-up

fx = F(x)

End Sub

Driver program

Sub VBA041()

Values Group 1: Coefficients

c3 = 1: c2 = 0: c1 = 1: c0 = -3

Values Group 2: Iteration control

MaxIts = 20: Eps = 0.000001: x0 = 2

Declaration of Group 3: Arguments

Dim x As Double, fx As Double, Converged As Boolean

Call Newton(x, fx, Converged)

With Worksheets(1)

.Cells(1, 1) = "x"

.Cells(1, 2) = x

.Cells(2, 1) = "fx"

.Cells(2, 2) = fx

End With

If Converged Then

MsgBox ("Convergence criterion satisfied ")

Else

MsgBox ("Convergence criterion NOT satisfied ")

End If

End Sub

To solve another problem you do not have to rewrite the whole program. It is enoughto change the functions F and DF, and some of the input.

31

Failures in the Newton Method: Though it has quadratic convergence, there arelikely to be problems in some cases.

One problem can occur if f (xi) is very large in which case the graph y = f (x) iscurving very rapidly.

A second problem occurs if f (xi) is very small which can happen if the function isalmost parallel to the x axis near the root, or if there is a second root very close by (inwhich case f (x) = 0 somewhere between the roots), or if the graph of y = f (x) justtouches the x - axis at the root but does not cross it. (Multiple root.)

It is often observed, that the method converges from a near enough starting point,but diverges if the starting point is selected without care.

4.1.4 Optimization: Golden ratio search for a minimum

For minimizing a function of one variable, we need a bracket for the solution analogousto sign change for nonlinear equation.

A real-valued function f() is unimodal on interval [a; b] if there is unique location x0such that f is strictly decreasing for x < x0 (that is on the left), and strictly increasingfor x > x0 (that is on the right).

Suppose f is unimodal on [a; b], and let x1 and x2 be two points within interval, withx1 < x2 . Evaluating and comparing f(x1) and f(x2), we can discard either [x2; b] or[a;x1], with minimum secured to lie in the remaining subinterval.

To repeat the iteration, we need to carry out only textbfone new function evaluation.To reduce the length of interval by a fixed fraction at each iteration, each new pair of points

must have the same relationship with respect to the new interval that the previous pair hadwith respect to previous interval. To accomplish this, choose the relative positions of thetwo points to satisfy the golden ratio: = (

(5) 1)/2 0.618. Whichever subinterval

is retained, its length will be 0.618 relative to previous interval, and one interior pointretained will be at position either at 0.618 or 0.382.

4.2 Fixed-point iteration as a general framework foriterative processes

Many iterative methods for solving nonlinear equations can be viewed as

xk+1 g(xk)

where g(x) is constructed from the f(x) function. (If you understand the above formalism,you can easily create the g-function for example, for the Newton iteration: g(x) =x f(x)/f (x).)

The scheme is called fixed-point iteration or direct iteration. There are four basiccases:

monotone convergence oscillatory convergence

32

monotone divergence oscillatory divergenceAlmost all iterative processes manifest one of the above behavior types, even if the

derivation of the method has nothing to do with direct iteration.

4.3 Representing, manipulating functions

4.3.1 Numerical Differentiation of functions that can be evalu-ated everywhere

When an analytical solution to the first derivative of a given function is difficult or in-convenient then a numerical method can be used to provide an approximate, thoughoften highly accurate, computation. Many methods exist, with increasing complexitythey perform, in general, computations to a higher accuracy. A basic understanding ofthe meaning of Truncation Error and Round-off Error is crucial.Intuitive error analysis of numerical differentiation

Take the function f(x) = sin(x) Calculate the derivative at x = pi /3 using the simplestforward divided difference Use step size: 1E-5 , 1E-10 , 1E-15 , 1E-20 , . . . (until yourcalculator stops giving reasonable result )(Use radian switch if you need.)Forward Difference Approximation (first derivative)

The Forward-difference approximation method for the numerical differentiation of a func-tion can be derived by considering differentiation from first principles or by consideringTaylors expansion.The differentiation from first principles derivation is intuitive:

f (x) = limh0

f(x+ h) f(x)h

As a computer cannot divide by zero the computed (finite) version of this expression is:

f (x) f(x+ h) f(x)h

This is the Forward-difference approximation for the numerical derivative of a function,it has the most basic form for a numerical derivative and is the least accurate.The derivation through Taylors expansion provides more information on the method:

f(x+ h) = f(x) +h

1! f (x) + h

2! f (x) + . . .

Rearrange and represent all remaining terms in one remainder term

f (x) =f(x+ h) f(x)

h+h

2 f (z)

We do not know z and hence neglect the term and hence obtain the forward differenceapproximation previously derived. With this derivation, however, we see clearly, that theerror is approximately (h/2) f (z) , i.e. proportional to h. This method is of first order.

33

A plot of the error on a log-log scale would result in a 45o straight line. To minimizethe error choose a small value of h, but h should not be too small as round-off errors inthe machine arithmetic increase as h decreases.

Backward difference approximation (of the first derivative) is very similar:

f (x) =f(x) f(x h)

h+h

2 f (z)

Again, we neglect the term containing z, Therefore

f (x) f(x) f(x h)h

and the method is of first order.The central-difference approximation of the first derivative gives reasonably accurate

results and is often implemented:

f (x) f(x+ h) f(x h)2h

Interestingly, it does not make use of the function value at the base point at all. It isa second order method and the log-log plot of the error (on an example with known exactresult) would give a straight line with slope two.

The central-difference approximation of the second derivative is:

f (x) f(x h) 2f(x) + f(x+ h)h2

This is a second order method.

4.3.2 Numerical Integration

Review geometric meaning Review physical (engineering) meaning

Trapezoid rule

The aim is to calculate, numerically, the integral of a function f(x) from a to b. Thebasic idea is to evaluate the function at specific locations between the limits, summingthe function evaluations will give an approximation to the integral (area under the curve).

There are many formulae for numerical integration (also known as quadrature), withincreasing complexity these formulae provide a greater accuracy. The Trapezoidal Ruleis the simplest.

Consider integrating a known function f(x) over the interval [a; b]. The TrapezoidalRule gives the following expression for the exact integral, I:

I =h

2 (f(a) + f(b)) h

3

12 f (z)

where h is the interval [a; b] , f (z) is the second derivative of the function evaluatedat some unknown point z, between a and b. The value of f (z) is generally not known(z is unknown though bounded between a and b) and so this term is omitted from the

34

solution when we perform the numerical calculation. This leads, in general, to an error(a truncation error) in the numerical evaluation of the integral.

The accuracy of the calculated result can be increased by multiple application. Wewill cut the interval [a, b] into n panels:

I = h(12 f(a) + f(x1) + f(x2) + ...+ f(xn1) + 12 f(b)

)where

x0 = ax1 = a+ hx2 = a+ 2 hxn1 = a+ (n 1) hxn = bh = (b a)/n

4.3.3 Simpsons 1/3 rule

The simple Simpsons rule uses a parabolic approximation to the curve.The multiple application of Simpsons rule is usually the first choice of engineers, becauseof its reliability and simplicity:

I h3(f(a) + 4f(x1) + 2f(x2) + ...+ 4f(xn1) + f(b))

where n is even and h = (b a)/n.

General idea of Gauss quadratures A quadrature rule is weighted sum of finite num-ber of sample values of the integrand function. (Both the trapezoid and Simpsons rulesare in this general class.) Closed formulas: Containing left and right points Open for-

mulas: NOT containing left and right points. Advantage of open formulas e.g. improperintegrals.

Richardson extrapolation (applied to numerical integration) An error estimatecan be formulated by making use of the fact that the error has a known order.

The plan is the following:

Do calculation with h and with h/2; Express remainder term as a power of h; The difference between the two results can be used to improve our final approxima-tion; Try to extrapolate to infinitely small h.

Example: repeated Simpsons rule has a leading error term of 4-th order. Assume wedid the calculations with h and with h/2. Therefore we have two computed values: I1

35

and I2. However, what we really want to know is I, that is still unknown.

I1 = I + Ch4

I2 = I + C h4

24

In the above equations C is also unknown. To eliminate it, multiply by 24 and 1:

24I2 = 24I + Ch4

(I1 = I + Ch4)Then subtract

24I2 I1 = (24 1)IFinally rearrange and obtain estimate of the unknown I:

I 16I2 I115

Note that we do not really need C and we have not calculated it.

4.4 Interpolation, Smoothing, Curve fit, Least squares

Review

Purposes for Interpolation:Plotting smooth curve through discrete data pointsReading between lines of table, etc.If the data is noise corrupted (is of finite accuracy) we rather use approximation (smooth-ing) instead of strict interpolation.

Interpolating by simple function families

Polynomials Piecewise polynomials (splines) Trigonometric functions (will be considered among spectral methods) Exponential functions Rational functionsAn interpolating polynomial is one which goes exactly through a set of data points.

In general a set of n + 1 data points can be interpolated by a polynomial of degree n.Such interpolation leaves no degree of freedom, the interpolating polynomial is unique.Unfortunately, it is only useful for fairly small sets of data points. An interpolatingpolynomial of large degree is required for a large set of data points, and such high degreepolynomials tend to have many oscillations and will approximate the data set very badly.

Cubic spline is piecewise 3rd order polynomial that is twice differentiable. It is usedextensively to interpolate a large number of data points. Rational function is a fraction:

both the numerator and the denominator are polynomials.

36

Differentiation and integration of noise-corrupted data

Differentiation is more sensitive to error in input data: You might want to use quitelarge step size (why?)

Integration is smoothing out: Step-size is the smaller the better in almost allcases.

Approximation by least-squares: The meaning of the criterion Sum of Squaresas a measure of how good the fit is.

Other requirements might include:

Smoothness, Least number of parameters, Extrapolating power, etc.

Two basic cases:

Model is based on first principles Model is just a convenient vehicle

4.4.1 Least Squares and its Numerical Aspects

When we want smooth approximation, we need some degrees of freedom. For instance,to fitting a straight line (2 parameters) to ten points leaves us with 8 degrees of freedom.There are infinitely many straight lines and we need a criterion to select a unique one.The least squares criterion is the most popular one. It minimizes the sum of squareddeviations between observations and values calculated from the model. The LSQ straightline fit is also called linear regression line.

A word of caution: Least squares methods are very adversely affected by bad data.For example, if a data set is basically linear but one faulty data point is a long way outsideof the linear trend of the rest of the data, then this one point can drastically change theregression line. This happens because we square the deviation of the data point fromthe line. Hence one point with a extra large deviation contributes a huge amount to theobjective function. The regression line will therefore be moved a long way towards theone deviant point and away from the main linear trend in order to minimize the deviationof this faulty point. This is a big factor in many engineering applications and specialmethods are used to try to detect faulty data points and eliminate them. (Or to use otherpossible measures of the goodness of fit such as sum of absolute deviations.)

Straight line fit - linear regression Suppose that some the data points should fit astraight line, y = mx + b, if it were not for experimental and measurement errors. Theproblem is to find the straight line characterized by m and b which best fits the datapoints in the least squares sense.Model:

y = mx+ b

37

Observations:(x1, y1) (x2, y2) . . . (xn, yn)

The derivation of formulae for the slope and intercept should be familiar by now. Thefinal formulae are

m =n (n

i=1 xiyi) (n

i=1 xi) (n

i=1 yi)

n (n

i=1 x2i ) (

ni=1 xi)

2

b =(n

i=1 yi)m (n

i=1 xi)n

Useful subroutines for straight-line fit:

Option Explicit

Option Base 1

Sub StraightLineFit(x() As Double, y() As Double, m As Double, b As Double)

Dim n As Integer

n = UBound(x)

m = (n * scalprod(x, y) - sum(x) * sum(y)) / (n * scalprod(x, x) - sum(x) ^ 2)

b = (sum(y) - m * sum(x)) / n

End Sub

Function sum(x() As Double) As Double

Dim n As Integer, i As Integer

n = UBound(x)

sum=0

For i=1 to n

sum=sum + x(i)

Next

End Function

Function scalprod(x() As Double, y() As Double) As Double

Dim n As Integer, i As Integer

n = UBound(x)

scalprod=0

For i=1 to n

scalprod=scalprod + x(i)*y(i)

Next

End Function

Transformations to straight line form Models of physical phenomena are in fewcases in straight-line form. Many 2-parameter nonlinear models can be, however, easilytransformed into straight-line form. In the log-log paper does the same for a power-lawmodel.

For other nonlinear models you should be able to figure out the transformation andmake the correspondence between the straight line parameters (slope and intercept) andthe real model parameters. We will see many examples.

Excel services: trendline option on graphs (with the Show Trendline Option); slopeand intercept spreadsheet functions; linear regression in data analysis package.

38

4.5 Numerical solution of ordinary differential equa-tions

Meaning Ordinary differential equation (ODE): all derivatives are with respect to singleindependent variable, often representing time.

Equations with higher derivatives can be transformed into a system of first orderequations. The ODE f = f(t, y) is non-autonomous. The ODE f = f(y) is autonomous.

Solution embeds into the directional field. The ODE f = f(t, y) does not by itself

determine a unique solution. To single out a particular solution, one must specify a valuey0 of the solution function at some point t0. This is called initial value. Order of ODE

determined by highest-order of derivative involved. Higher order ODEs may be narrowed

down also by using boundary values.

4.5.1 Basic methods for solving ODE

Euler Eulers method uses the forward difference approximation for dydxand can be sum-marized by the marching formula:

xi = xi1 + hyi = yi1 + h f [xi1, yi1]

RK-4 This method for solving y = f (x, ) , starting at (x0, y0) is one of the most popularones because it is quite accurate and stable. The formulae are:

xi = xi1 + hyi = yi1 + h6 (k1 + 2k2 + 2k3 + k4)

wherek1 = f [xi1, yi1]k2 = f [xi1 + h2 , yi1 +

h2 k1]

k3 = f [xi1 + h2 , yi1 +h2 k2]

k4 = f [xi1 + h, yi1 + h k3]Note that the ki values are various approximations to the slope of the unknown function.

Heun Heuns method improves on Eulers method by a better choice of the slope of theline followed from the point (xn, yn) to the next point (xn+1, yn+1). Instead of choosingthis slope as dy(xn)dx = f (xn, yn) as in the Euler method, choose it as the mean betweenthis slope and the slope at the destination point reached by the Euler method.

Simple Heunxi = xi1 + hy0i = yi1 + h f [xi1, yi1]y1i = yi1 +

h2 (f [xi1, yi1] + f [xi, y0i ]

)39

Iterated Heun (simplest predictor-corrector)

yki = yi1 +h2 (f [xi1, yi1] + f [xi, yk1i ]

)continue until convergence.

This is a predictor - corrector (implicit, iterative) method. Needs a stopping criterionfor inner iteration. It is a one-step, second order method and is stable. In practice we usehigher order variants.

Single-multi step, predictor-corrector, explicit-implicit So far weve consideredself-starting (one step) methods. They use only yi. Multiple-step methods use morethan one previously calculated value: that is to use yi and yi1 etc. The traditionalpredictor-corrector solution methods use one equation to predict the value of yn+1 usingthe previously computed yn, and other previously computed yi values. This is followedby a second equation which is used to correct the value given by the first equation. Theyprovide some improvement especially for stiff problems, but need a jump-starter (forinstance RK4)!

In choosing step size for advancing numerical solution of ODE, we want to take largesteps to reduce computational cost, but must also take into account both stability andaccuracy.

Adaptive: step size is adapted automatically from error estimate.

A method is called implicit, if some kind of iteration is involved. Predictor-correctormethods are implicit.

A stiff ODE corresponds to physical process whose components have disparate timescales or whose time scale is small compared to interval over the interval which it is studiedon.

40

Chapter 5

Methods related tomulti-variable problems

5.1 Matrices, vectors

Whats an Array? Whats a Matrix? Whats the Difference? An array is a collection ofobjects, each occupying a position relative to the other objects in the collection. An arrayof antennas, for example, usually refers to a group of antennas laid-out in either a straightline, or in a plane. The objects of the array can be real, physical objects like antennas orspark plugs, or intangible objects like numbers. When an engineer talks about a matrix,she is generally talking about a numerical array. There is the further implication that thelaws of Matrix Algebra dictate how an array can be combined with and/or operated onby other numerical arrays. Although the matrices encountered by engineering studentsare usually one-dimensional (numbers positioned in a straight line) or two-dimensional(numbers positioned in a plane), they can be higher-dimensional, as well.

A subroutine to get the scalar product of two vectors

Function scalprod(x() As Double, y() As Double) As Double

Dim n As Integer, i As Integer

n = UBound(x)

scalprod = 0

For i = 1 To n

scalprod = scalprod + x(i) * y(i)

Next

End Function

41

A subroutine to multiply two (compatible) matrices

Sub matmult(a() As Double, b() As Double, c() As Double)

multiplies matrices a anb b to get c

Dim row As Integer, cola As Integer, rowb As Integer, col As Integer

Dim i As Integer, j As Integer, k As Integer

Dim s As Double

row = UBound(a, 1)

cola = UBound(a, 2)

rowb = UBound(b, 1)

col = UBound(b, 2)

If Not (cola = rowb) Then

MsgBox ("matprod: a and b are not compatible")

Stop

End If

If (UBound(c, 1) < row Or UBound(c, 2) < col) Then

MsgBox ("matprod: c is not large enough to contain the result")

Stop

End If

For i = 1 To row

For j = 1 To col

s = 0

For k = 1 To cola

s = s + a(i, k) * b(k, j)

Next k

c(i, j) = s

Next j

Next i

End Sub

A subroutine to multiply matrix A by a (column) vector b:Sub matvecprod(a() As Double, b() As Double, c() As Double)

Dim row As Integer, col As Integer

Dim i As Integer, j As Integer

Dim s As Double

row = UBound(a, 1)

col = UBound(a, 2)

If ( row < Ubound(b) OR row < Ubound(c) ) Then

MsgBox ("matvecprod: vector b or c is not compatible with matrix a")

Stop

End If

For i = 1 To row

s = 0

For j = 1 To col

s = s + a(i, j) * b(j)

Next j

c(i) = s

Next i

End Sub

Matrix is lower triangular if all entries above main diagonal are zero: aij = 0 for i < j.Matrix is upper triangular if all entries below main diagonal are zero: aij = 0 for i > j.

42

Review linear dependence and rank

5.2 System of linear equations

Given m n matrix A and m-vector b, and unknown n-vector x satisfying Ax = b.System of equations asks: Can b be expressed as linear combination of columns of A? Ifso, coefficients of linear combination given by components of solution vector x.

Example 1 Consider the system

2x 3y = 84x 5y = 16

The determinant is non-zero. Two lines crossing at one point.

Example 2 Consider the system

2x 3y = 84x 6y = 16

Thus the determinant is 2 6 3 4 = 0. And indeed the second equation is 2 timesthe first. Geometrically this corresponds to two coincident lines. In terms of rank: thecoefficient matrix has rank 1, the augmented matrix has rank 1.

Example 3 Consider the system

2x 3y = 84x 6y = 17

Thus the determinant is 2 6 3 4 = 0. There is no solution. Geometrically thiscorresponds to two parallel lines never crossing. In terms of rank: the coefficient matrixhas rank 1, the augmented matrix has rank 2.

Existence and Uniqueness Statement with Rank The rank of a matrix tells us howmany independent rows (or how many independent columns) the coefficient matrix has it may be less than the total number of equations.

If rank of the coefficient matrix is less than the number of equations, two cases canhappen:

Consistent (if rank of coeff matrix = rank of augmented matrix): Infinite numberof solutions

Non-Consistent (if rank of coeff matrix < rank of augmented matrix): No solutionWe can we express the solution with the inverse of the coefficient matrix, as

x = A1b

but it is usually not a good idea to get the solution via the inverse. The reason for thisis simple: to calculate the inverse of the matrix A is more computational work than tosolve the original system of equations.

43

Naive Gauss elimination A system of linear equations can be represented in matrixforms of various types: e.g.

x1 + 2x2 3x3 = 52x1 3x2 + x3 = 3x1 + x2 2x3 = 0

becomes 1 2 32 3 11 1 2

x1x2x3

= 53

0

To solve the system a method (algorithm) calledGaussian Elimination is used, starting

with the augmented matrix containing the coefficient matrix on the left with one extracolumn containing the right hand side values (a similar method is used to find A1).The augmented matrix is changed by operations which do not change the solutions of theassociated system of equations. The allowed changes are the three elementary rowoperations(a) Interchanging two rows(b) multiplying or dividing a row by a non-zero constant(c) Replacing a row by the sum of that row and a multiple of another row (adding anegative multiple of a row e.g. adding (2) row 2 to row 1) is the same as subtractingthe positive multiple (subtracting 2 row 2 from row 1 so that this row operation alsoincludes subtraction).

The objective is to change the coefficient matrix to a special form of matrix calledupper triangular in which all entries in the bottom left corner below the diagonal arezero. Sometimes the goal is to create an echelon form which is upper triangular in whichthe first non-zero in each row is a 1.

The equations represented by this upper triangular form have exactly the same solu-tions but are much easier to solve.

e.g. Solving the example above (a label like R2 = R2 + (2)R1 means replace row2 by the sum of row 2 plus 2 times row 1). At each stage one entry is a special entrycalled the pivot. A pivot, shown in bold face in the next example, is the value in the pivotrow which is used to modify the other rows and is in the column where zeros are beingcreated.

1 2 3 5

2 3 1 31 1 2 0

R2 = R2 + (2)R1R3 = R3 +R1

1 2 3 50 7 7 7

0 3 5 5

R3 = R3 +

(37

)R2

1 2 3 50 7 7 70 0 2 2

This has created an upper triangular matrix (the first three columns). To further

simplify this, divide the second row by 7 and the third row by 2, creating an echelon

44

form. The corresponding equations can easily be solved by starting with the last equationwhich gives the x3 value, then finding x2 from the next equation up and so on (backsubstitution): 1 2 3 50 1 1 1

0 0 1 1

x1 + 2x2 3x3 = 5x2 x3 = 1x3 = 1

x1 = 5 + 3x3 2x2 = 2

x2 = 1 + x3 = 0x3 = 1

Solve thirdSolve secondSolve first

This version is called naive elimination, because we assumed that we can always divideby the diagonal element in the next row (e.g. 7 in iteration No two.)

Naive Gauss-Jordan elimination In this version of the elimination we reduce thesystem to an equivalent system with the identity matrix, and hence the solution can beread off from the last column.

Option Explicit

Option Base 1

Sub GaussJordan(A() As Double) Augmented matrix

Dim PivElt As Double, TarElt As Double

Dim n As Integer number of equations

Dim PivRow As Integer, TarRow As Integer pivot row; target row

Dim i As Integer, j As Integer

n = UBound(A, 1)

For PivRow = 1 To n process every row

PivElt = A(PivRow, PivRow) choose pivot element

If PivElt = 0 Then

MsgBox ("Zero pivot element encountered")

Stop

End If

For j = 1 To n + 1

A(PivRow, j) = A(PivRow, j) / PivElt divide whole row

Next

For TarRow = 1 To n now replace all other rows

If Not (TarRow = PivRow) Then

TarElt = A(TarRow, PivRow)

For j = 1 To n + 1

A(TarRow, j) = A(TarRow, j) - A(PivRow, j) * TarElt

Next j

End If

Next TarRow

Next PivRow

End Sub

Answer the questions: Where is the coefficient matrix at input? Where is the right

hand side?

At output: Where is the solution? What happened to the coefficient matrix?

For both the naive Gauss and Gauss-Jordan methods, in some cases the diagonal entryis simply zero and so it is not an option to use it. In general, solving linear systems ofequations, it is necessary to use different pivot choices than the diagonal values used inthe first example above.

45

5.3 LU Decomposition and Backsubstitution

An advanced form of the Gauss elimination is called LU decomposition-substitution. Theprocedure is divided into two parts: the first one operates on the coefficient matrix anddoes the computationally demanding half of the job; the second one (called substitution)uses the results of the first part to obtain a solution to a given right-hand side.

Note that the decomposition destroys the original coefficient matrix (overwrites it bythe L and U matrix elements.)

An additional vector (Indx) is used to remember the row changes found necessaryduring partial pivoting. The Indx vector then enters the substitution routine. (Notethat it is declared private but global for the modul containing the LUDecompose andLUSubstitute routines.)

If in spite of partial pivoting the decomposition can not be done, then the matrix issingular and the procedure stops with a message. (In other words, partial pivoting isenough, full pivoting does usually not improve the method.)

The solution comes out from the substitution routine as x.

Option Explicit

Option Base 1

Sub VBA092() LU decomposition-substitution driver

Dim n As Integer

With Worksheets("Sheet1")

.Cells(1, 1) = "Number of equations, n"

End With

With Worksheets("Sheet1")

n = .Cells(1, 2)

End With Dim i As Integer, j As Integer

ReDim A(n, n) As Double, b(n) As Double, x(n) As Double

ReDim Indx(n) As Integer

With Worksheets("Sheet1")

For i = 1 To n

For j = 1 To n

A(i, j) = .Cells(i + 1, j)

Next j

b(i) = .Cells(i + 1, n + 1)

Next i

End With

Call LUDecompose(A, Indx)

Call LUSubstitute(A, Indx, b, x)

With Worksheets("Sheet1")

.Cells(1, n + 3) = "Solution"

For i = 1 To n

.Cells(i + 1, n + 3) = x(i)

Next i

End With End Sub

It is a good practice to put the LUDecomposition and LUSubstitute into a separatemodule.

46

Option Explicit

Option Base 1

Sub LUDecompose(A() As Double, Indx() As Integer)

input A(n,n)

output A(n,n) and indx(n)

Const Tiny As Double = 1E-16

Dim n As Integer

n = UBound(A, 1)

Dim i As Integer, j As Integer, k As Integer, m As Integer

Dim dum As Double

For k = 1 To n - 1

m = k

For i = k + 1 To n

If (Abs(A(i, k)) > Abs(A(m, k))) Then m = i

Next i

Indx(k) = m

dum = A(m, k)

If m > k Then

A(m, k) = A(k, k)

A(k, k) = dum

End If

If Abs(dum) > Tiny Then

dum = 1 / dum

Else

MsgBox ("Singular coefficient matrix")

Stop

End If

For i = k + 1 To n

A(i, k) = -A(i, k) * dum

Next i

For j = k + 1 To n

dum = A(m, j)

A(m, j) = A(k, j)

A(k, j) = dum

For i = k + 1 To n

A(i, j) = A(i, j) + A(i, k) * dum

Next i

Next j

Next k

If Abs(A(n, n)) < Tiny Then

MsgBox ("Singular coefficient matrix")

Stop

End If

End Sub

The LUDecomposition overwrites the coefficient matrix by the decomposed (LU) form.You have to be careful not to call it more than once.

Notice that the back-substitution can be called as many times as many right-hand-sides we have (as far as the coefficient matrix of the system is the same.)

47

Sub LUSubstitute(A() As Double, Indx() As Integer, b() As Double, x() As Double)

input A(n,n) containing the LU decomposition

input Indx(n) index vector

input b(n) right hand side

output x(n) solution

Dim n As Integer n =UBound(A, 1)

Dim i As Integer, j As Integer, k As Integer

Dim dum As Double

For i = 1 To n

x(i) = b(i)

Next i

For k = 1 To n - 1

i = Indx(k)

dum = x(i)

x(i) = x(k)

x(k) = dum

For j = k + 1 To n

x(j) = x(j) + A(j, k) * dum

Next j

Next k

For k = n To 1 Step -1

x(k) = x(k) / A(k, k)

For j = 1 To k - 1

x(j) = x(j) - A(j, k) * x(k)

Next j

Next k

End Sub

Sparse systems Tridiagonal: Thomas algorithm

Sparse methods can be applied to all kinds matrices of special form and even to ageneral sparse matrix of no particular type, but the methods are quite complicated, ingeneral.

A special type of matrix called tridiagonal in which the only non zeros are on themain diagonal and immediately below it and above it (i.e. on the diagonals immediatelybelow and above the main diagonal) a very effective algorithm is due to Thomas.

e.g. Given the tridiagonal matrix shown store the three diagonals in a three column arrayas shown:

A =

2 1 0 0 01 2 1 0 00 2 4 0 00 0 1 3 20 0 0 2 4

a : 0 1 2 1 2b : 2 2 4 3 4c : 1 1 0 2 0

The storage saving in the above example is only from 25 down to 15. However in largermatrices the reduction is very significant. For an nn matrix with n2 entries the storagewill be reduced to 3n. Hence if n = 1000 for a tridiagonal matrix, then the full storageis 1, 000, 000 and the reduced storage is 3000. The algorithm not only saves storage, butit enormously reduces the number of computations. This is due to the fact, that we skipmultiplications and additions (because we know that the result will be zero.)

The Thomas algorithm is extensively used in reservoir simulation:

48

Option Explicit

Option Base 1

Sub Thomas(a() As Double, b() As Double, c() As Double d() As Double, x() As Double)

Tridiagonal system of equations:

a is subdiagonal, b is diagonal, c is super diagonal

a(1) and c(n) are not used

d is the right hand side, x is the result

Dim n As Integer, i As Integer

n = UBound(b)

ReDim w(n) As Double, g(n) As Double

w(1) = b(1)

g(1) = d(1) / w(1)

For i = 2 To n

w(i) = b(i) - a(i) * c(i - 1) / w(i - 1)

g(i) = (d(i) - a(i) * g(i - 1)) / w(i)

Next i

x(n) = g(n)

For i = n - 1 To 1 Step -1

x(i) = g(i) - c(i) * x(i + 1) / w(i)

Next i

End Sub

Note that matrix operations and solution of linear systems are also provided by excelas spreadsheet functions, macros.

Direct vs IterativeGauss elimination, Gauss-Jordan elimination, LU-decomposition/substitutionare DIRECT methods. In direct methods we can predict the number of multiplicationsand additions if the number of equations (n) is known.

Iterative methods are generalization of the direct iteration we learned for one variablex = g(x) equations. Iterative methods are used for very large but sparse systems (reservoirsimulation). You can not predict the number of operations for an indirect method andyou need a starting guess and a convergence criterion. The iterative method we are goingto learn is the Gauss-Seidel method.

The method is illustrated for a general 3 3 system. The strategy is to express x1from the first row, x2 from the second row, etc. Then start with a guess and calculate anew x1 from the first equation. Then with the best available approximants calculate x2and so forth. In other words, as soon as a variable is updated in one equation its newvalue is used on the right hand side of the next equation:

a11x1 + a12x2 + a13x3 = b1a21x1 + a22x2 + a23x3 = b2a31x1 + a32x2 + a33x3 = b3

a11x1 = b1 a12x2 a13x3a22x2 = b2 a21x1 a23x3a33x3 = b3 a31x1 a32x2

a11x

k+11 = b1 a12xk2 a13xk3

a22xk+12 = b2 a21xk+11 a23xk3

a33xk+13 = b3 a31xk+11 a32xk+12

It works fine on diagonally dominant matrices.Diagonally dominant matrices: the absolute value of the diagonal element is greater

than the sum of the absolute values of all the other elements in a given row. Identitymatrix is diagonally dominant

49

5.4 System of nonlinear equations, Newton-Raphsonmethod

The method may be used to solve a general equation f(x) = 0, provided the left-hand-sidecan be differentiated analytically. Note that bold-face denotes matrix and vector. Wehave n equations to solve and n components of the unknown to find.

If an initial guess is known, and at that location we can calculate the function f(x0)and its partial derivatives collected into the Jacobian matrix J(x0). The first element inthe first row of the Jacobian matrix will contain the partial derivative of f1 with respectto x1, the second element in the first row will contain the partial derivative of f1 withrespect to x2, and so on.

Writing the equation of the tangent plane:

f(x0) + J(x0)(x x0) = 0

Substituting y = 0 we find that the line crosses the x axis at x1 where

(x1 x0) =x = [J(x0)]1 f(x0)

This is the iteration formula of the Newtons method. The obtained x1 can be thenrenamed to x0 and hence we can continue the iterations.

In practice we do not use the matrix inverse, we just solve

J(x0)x = f(x0)

for x using LU Decomposition.Cautious Approach: We calculate the x and multiply it by a less than one if we

encounter convergence problems. Alpha can be adjusted during the iteration process.Questions: How many subroutines does the user have to provide for a Newton-Raphson

method? Two subroutines, one to evaluate f and the other for the Jacobian matrix, J.N/R