Welcome to - TumCivil.com Engfanatic Club : … Methods for Civil Engineers Welcome to Lecture 1 –...

Numerical Methods for Civil EngineersNumerical Methods for Civil Engineers

Welcome to

Lecture 1 – MATLAB 1

�� ..��..�� S U R A N A R E E INSTITUTE OF ENGINEERING

UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING

TextbookTextbookApplied Numerical MethodsApplied Numerical MethodsWith MATLAB for EngineersWith MATLAB for Engineersand Scientistsand Scientists

STEVEN C. CHAPRASTEVEN C. CHAPRA

McGraw-Hill International Edition

ReferencesNumerical Methods For Engineers with Personal Computer Applications,

(Third Edition) by Chapra, S.C. and R.P. Canale, McGraw-Hill, 1998

Numerical Methods with MATLAB : Implementations and Applications,

Gerald W. Recktenwald, Prentice-Hall, 2000

The Matlab 7 Handbook , Mathwork Inc.

KEEP THESE BOOKS! They are excellent career references

(at least for a while)

An Introduction to Numerical Methods : A MATLAB Approach,

Abdelwahab Kharab and Ronald B. Guenther, Chapman & Hall/CRC, 2002

Numerical Methods using MATLAB,

John H. Mathews and Kurtis D. Fink, Prentice-Hall, 2004

Topics Covered

• Introduction to Matlab

• Approximations and Errors

• Roots of Equations

• Linear Systems

• Curve Fitting

• Interpolation

• Numerical Integration

• Ordinary Differential Equations

• Optimization

Conduct of Course

Assignments 20 %

Midterm Exam 40 %

Final Exam 40 %

Grading Policy

100 - 90

Final Score Grade

A89 - 85 B+84 - 80 B79 - 75 C+74 - 70 C69 - 65 D+64 - 60 D59 - 0 F

��ก��

- �� ก��! ��"��! #��" �$ %"�&

- "'��ก'�� !��!(��)

- "'��#��&� ก��! � *+��!� �+, � I �� F

MATLABThe Language of Technical Computing

The latest version Matlab 7.3 R2006b

www.mathworks.com

ATLAB �)-��)�.ก��ก��/��0��

0 ,��",�1 ก��."�� .�+2�3��)�+" �$ 2�4

�4��"��"2��+.��#� '�"/�#��&ก��/��0

�� "(�)5,�ก��.&&�)6� �/��#��)-�ก��'��

��+�� MATLAB ��ก��"/��7� "��ก�� $8� #��

"��/��0,��7��,��ก��

M

MATLAB = MAT rix LAB oratory

- Math and computation

- Algorithm development

- Modeling, Simulation, and Prototyping

- Many toolboxes for solving problems:

Control System Toolbox

Signal Processing Toolbox

System Identification Toolbox

Neural Network Toolbox

Statistics Toolbox

Optimization Toolbox

Partial Diff. Equation Toolbox

Symbolic Math Toolbox

- Data analysis, exploration, and visualization

- Scientific and Engineering Graphics

http://www.math.utah.edu/lab/ms/matlab/matlab.html

http://www.mathworks.com/

http://www.math.mtu.edu/~msgocken/intro/intro.html

MATLAB Educational Sites

http://www.eece.maine.edu/mm/matweb.html

� The MATLAB System

� Basic Arithmetic

� Built-in Functions

� Built-in Variables

� Matrices & Magic Squares

MATLAB 1

The MATLAB System

The MATLAB system consists of five main parts:

Development Environment: set of tools and facilities that help you use MATLAB functions and files. Many of these tools are graphical user interfaces. It includes the MATLAB desktop and Command Window, a command history, an editor and dedugger, and browsers for viewing help, the workspace, files, and the search path.

The MALAB Mathematical Function Library: a vast collection of computational algorithms.

The MATLAB Language: This is a high-level matrix/array language with control flow statements, functions, data structures, input/output, and objected-oriented programming features.

Graphics: MATLAB has extensive facilities 2-D and 3-D data visualization, animation, and presentation graphics.

The MATLAB Application Program Interface (API): allows you to write C and Fortran programs that interact with MATLAB.

Getting Started

MATLAB Desktop

Getting Started

Command Window

Command prompt >> DEMO

Getting Started

Editor/Debugger

Basic ArithmeticBasic Arithmetic

Last-line editing

Calculator functions work as you'd expect:

>>(1+4)*3

ans =

15

+ and - are addition, / is division, * is multiplication, ^ is an exponent.

>> 5*5*5

>> 5^(-2.5)

>> 3*(23+14.7-4/6)/3.5

Up Arrow

>> 2 + 6 - 4

ans =

4

The value in ans can be recalled:

>> ans/2

ans = 2

Assign value to a variable:

>> a = 5

>> b = 6

>> c = b/a

>> a=2;

>> A=3;

>> 2*a

>> 2*A

Upper & Lower Case

Several commands in the same line:

>>x=2;y=6+x,x=y+7

… For too long command:

>> Num_Apples = 10;

>> Num_Orange = 25;

>> Num_Pears = 12;

>> Num_Fruit=Num_Apples+Num_Orange...+ Num_Pears

sin(x), cos(x), tan(x),

sqrt(x), log(x), log10(x),

asin(x), acos(x), atan(x)

Built-in Functions

>> sin(pi/4)

ans =

0.7071

>> pi

ans =

3.1416

The output of each command can be suppress by using semicolon ;

>> x = 5;

>> y = sqrt(59);

>> z = log(y) + x^0.25

z =

3.5341

The commas allow more than one command on a line:

>> a = 5; b = sin(a), c = sinh(a)

b =

-0.9589

c =

74.2099

MATLAB Variables is created whenever it appears on the left-hand of “ = “

>> t = 5;

>> t = t + 2

t =

7

Format

>> pi

>> format long

>> pi

>> format short

>> format bank

>> format short e

>> format long e

>> format compact

>> format loose

>> format

Any variable appearing on the right-hand side of“ = “ must already be defined.

>> x = 2*z

??? Undefined function or variable ‘z’

Use long variable names is better to remember

and understandable for others

>> radius = 5.2;

>> area = pi*radius^2;

>> who

>> whos

>> clear

>> help log

>> lookfor cosineOn-line Help:

Built-in VariablesUse by MATLAB, Should not be assigned to other values

Variable Meaning

ans value of an expression when not assigned to variable

eps floating-point precision

i, j unit imaginary numbers, i = j =

pi π = 3.14159265 . . .

realmax largest positive floating-point number

realmin smallest positive floating-point number

Inf ∞, a number larger than realmax , result of 1/0

NaN not a number (0/0)

1−

>> x = 0; >> 5/x>> x/x

Matrices and Magic SquaresMatrices and Magic Squares

MATLAB allows you to work with entire matrices quickly and easily.

In MATLAB, a matrix is a rectangular array of numbers.

scalars = 1-by-1 matrices

vectors = one row or column matrices

Renaissance engraving Melencolia I by the German artist and amateurmathematician Albrecht Dürer.

The matrices operations in MATLAB are designed to be as natural as possible.

it is usually best to think of everything as a matrix.

"��,� ก�1

��ก�� ก��ก� � �� :• ��ก�� !� "ก�#��$ �� %�& � �� ,

• ��$�'�� ; ��$��

• �(�%��$��%��$��)#�*ก�%�� [ ]

ก��$��ก'+�� MATLAB �,-�$%��.�:• �� ก��$-��

• �,��$��ก'+!ก-/�+�$ �"�0�� ก

• ��$��ก'+��$/1�ก+��0��

• ��$��ก'+��$ก��ก�� M-file

� �2��2+�$ ��$��%�$��,�� Command Window

>> A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]

MATLAB !��ก'+�� $-�:

A =

16 3 2 13

5 10 11 8

9 6 7 12

4 15 14 1

��ก'+!� "ก#��3ก��%��!,� ��ก��ก�� MATLAB workspace

�� $� ��ก'+�� & A

ก��"�� workspace ��$ �"� �-� �"�#$�!��$�,�� who %�& whos :

>> whos

Name Size Bytes Class Attributes

A 4x4 128 double

Workspace BrowserWorkspace Browser

The MATLAB workspace consists of the set of variables (named arrays) built up during a MATLAB session and stored in memory.

You add variables to the workspace by using functions, running M-files, and loading saved workspaces.

To view the workspace and information about each variable,

use the Workspace browser, or use the functions who and whos.

sum, transpose, and sum, transpose, and diagdiag

>> sum(A)

ans = 34 34 34 34

A =

16 3 2 13

5 10 11 8

9 6 7 12

4 15 14 1

34 34 34 34

>> A’

ans = 16 5 9 4 3 10 6 15 2 11 7 14 13 8 12 1

>> sum(A’)’

>> diag(A)

>> sum(diag(A))

SubscriptsSubscripts

��ก�� i �� j � ��ก A !� "กก,%�� A(i,j)

�� A(4,2) �& �� 4 �� 2

>> A(4,2)

ans =

15

A =

16 3 2 135 10 11 89 6 7 124 15 14 1

�� $�$ �ก��,��45�� 4 � � A:

>> A(1,4)+ A(2,4)+ A(3,4)+ A(4,4)

ans =

34

The Colon OperatorThe Colon Operator

�� : �& �� +��,��6�ก��%�3�� MATLAB

>> 100:-7:50

>> 1:10

ans =

1 2 3 4 5 6 7 8 9 10

��,�� !��$��ก�� +� �� !,��)�!ก 1 3� 10

$�$ �ก��%�� &��-��ก�#%�3�� %$ก,%��%��

ans =

100 93 86 79 72 65 58 51

SubmatrixSubmatrix

>> A(1,:)

>> A(:,2)

>> A(3:4,1:2)

�� $ : �2&� ก,%��#�� ก'+ �� A(1:k,j)

�& ��ก�� 1 3� k � �� j � ��ก'+ A

>> sum(A(1:4,4)

>> sum(A(:,end)

$�$ �ก��,��45�� 4 �,��$�&

%�& !��$��.��ก��$�,�� end %�� 3�� %�& � ��7��$�

>> [A B]

>> size(ans)

>> [A ; B]

JuxtapositionJuxtaposition

>> B = [9 8 2 5 ; 4 5 6 7 ; 2 1 3 4]

Row vector >> A = [ 2 3 5 7 11]

Column vector >> A = [ 2; 3; 5; 7; 11]

Transposition >> At = A’

Row and Column VectorsRow and Column Vectors

>> A / B

>> A ./ B

>> A .^ 2

>> odd = 1:2:11

>> even = 2:2:12

>> natural = 1:6

>> angle = 0:pi/10:pi;

>> sin(angle)

>> A = [ 3 5 7 9 11 ]

>> A(3)

>> length(A)

>> clear(A)

>> B = [ 2 4 6 8 10 ]

>> A + B

>> A - B

>> A * B

>> A .* B

Arrays OperationsArrays Operations

Generate matrices using builtGenerate matrices using built --in functionsin functions

>> A = zeros(4) >> A = zeros(3,4)

>> A = ones(4)

>> A = eye(4)

>> A = magic(4)

>> S = A + B

>> D = A - B

>> A*B

>> C = [ 10 11; 12 13; 14 15];

>> A*C

>> A^2

>> L = log10(A)

>> [m , n] = size(A)

>> det(A)

>> inv(A)

>> v = [1 2 3];

>> A = diag(v)

>> B = diag([1 2 1 2])

>> w = diag(B)

Elementary Matrix OperationsElementary Matrix Operations

linspace function creates row vectors with equally spaced elements.

>> u = linspace(0.0,0.25,5)

>> v = linspace(0,9,4)’

>> x = linspace(0,pi/6,6*pi);

>> s = sin(x);

>> c = cos(x);

>> t = tan(x);

>> [x’ s’ c’ t’]

LinspaceLinspace

Example 2.1 Transportation route analysisExample 2.1 Transportation route analysis

The following table gives data for the distance travel along five truck routes and thecorresponding time required to traveled each route. Use the data to compute the average speed required to drive each route. Find the route that has the highest average speed.

7.510.19.18.210.3Time (hrs)

370530490440560Distance (miles)

54321

Solution:>>d = [560, 440, 490, 530, 370]

>>t = [10.3, 8.2, 9.1, 10.1, 7.5]

>>speed = d./t

speed =54.3689 53.6585 53.8462 52.4752 49.3333

>>[highest_speed, route] = max(speed)

highest_speed =54.3689

route =1

Lecture 2 - MATLAB 2


Mongkol JIRAVACHARADET

S U R A N A R E E INSTITUTE OF ENGINEERING


�� Save & Load from External FilesSave & Load from External Files

�� GRAPHICSGRAPHICS

- Creating a plot

- Plotting Tools

- Plot Graph from Data

Save Data to External FileSave Data to External File

>> clear

>> x = linspace(0,2*pi); y = cos(x); z = sin(x);

>> save xyz

Load Data from External FileLoad Data from External File

>> whos

>> load xyz

>> whos

��ก�ก� �� xyz.matxyz.matxyz.matxyz.mat �� binary filebinary filebinary filebinary file �� Current DirectoryCurrent DirectoryCurrent DirectoryCurrent Directory

>> clear

>> XY = load(‘xyvals.txt’)

>> x = XY(:,1)

>> y = XY(:,2)

>> clear

>> x = 0:5; y=5*x;

>> XY = [x’ y’];

>> save xyvals.txt XY -ascii

Save Data to Plain Text FileSave Data to Plain Text File

Load Data from Plain Text FileLoad Data from Plain Text File

Command Description

Input/Output CommandsInput/Output Commands

User

Input

MATLABMATLAB

Output - Screen

- File

disp (A) Displays the contents, but not the name, of theArray A.

disp (‘text’) Displays the text string enclosed within single quotes.

fprintf Control the screen’s output display format.

x = input(‘text’) Displays the text in quotes, waits for user input fromthe keyboard, and stores the value in x.

x = input(‘text’,’s’) Displays the text in quotes, waits for user input fromthe keyboard, and stores the input as a string in x.

INPUT AND OUTPUTINPUT AND OUTPUT

Prompting for User Input

>> x = input(‘Enter a value for x’);

By default, the input function returns a numerical value.

To obtain a string input, a second parameter, ‘s’ , must be provided.

>> yourname = input(‘Enter your name ‘,‘s’);

function s = inputAbuse% inputAbuse Use input messages to compute sum of 3 variables

x = input(‘Enter the first variable to be added ‘);y = input(‘Enter the second variable to be added ‘);z = input(‘Enter the third variable to be added ‘);s = x+y+z;

Text OutputText Output

>> disp(‘My favorite color is red’)

Since disp requires only one argument, the message and variable

must be combined into a single string.

>> yourName = input(‘enter your name ‘,’s’);

>> disp([‘Your name is ‘,yourName])

The disp Function

>> Speed = 63;

>> disp(‘The vehicle’s predicted speed is:’)

>> disp(Speed)

G R A P H I C SG R A P H I C S

Creating a PlotCreating a Plot

ก��ก�� MATLAB �� plot � �� y ��ก�� plot(y)

��ก�� y �� !�� y

��ก�"��ก�� plot(x,y) MATLAB ��ก�� ก��ก�� x �#��ก��$��ก�� y

>> x = 0:pi/100:2*pi;

>> y = sin(x);

>> plot(x,y)

�� ก��ก��!�� sine(x) �� x ��%�� 0 �&� 2ππππ �"�'$�'��!��#��#��"��

Graph ComponentsGraph Components

MATLAB ��ก��"�� Figure 1 ��(�!��#��

>> xlabel('x = 0 to 2\pi')

>> ylabel('Sine of x')

>> title('Plot of the Sine Function','FontSize',12)

��)��#��'$��ก�)� �� !��ก�)

ก�"�� ก��*# �#� ��ก�$� :

>> axis([0 2*pi -1.2 1.2])

>> grid on

Plotting ToolsPlotting Tools

��,��,� ��ก��"�� Figure �� ก��ก�"��(�#�ก-.��"��/�� ก��

ก�� ก� ��,��,� ��"�� :

>> plottools

ก��,��,� ��ก��ก��( View "�,��#$ก�01� Show Plot Tools

)�(#)��"�� Figure

��ก��2�ก3��#$ก�01� Hide Plot Tools ��(��$�ก��

� �#$ก�#,�ก��ก�� ก�� Property Editor ��#�� "�ก�"��#�ก-.� !��ก��

� �#$ก�#,�ก��"��ก��#�� /�� ก��

� �#$ก��',%� �� ก�"��ก�$�� x �#� y

��ก�� !��ก"��ก�� !��ก"�

>> legend('sin(x)','sin(x-.25)','sin(x-.5)')

>> clear

>> x = 0:pi/100:2*pi;

>> y = sin(x);

>> y2 = sin(x-.25);

>> y3 = sin(x-.5);

>> plot(x,y,x,y2,x,y3)

� �� legend ��)0��ก�� :

>> axis([min(x) max(x) min(y) max(y)]

Controlling the AxesControlling the Axes

>> hold on

>> plot([min(x) max(x)],[0,0])

>> hold off

>> grid

Plot Graph from DataPlot Graph from Data

��ก��ก!��(# �� (� ��ก�� x �#� y �� !��ก��

>> x = [0 1 2 3 4 5]

>> y = [0 6 7 5 8 9]

>> plot(x,y)

"�,��!��!��(#��ก/�#�5��ก:

>> clear

>> load(‘xy.data’)

>> xy

>> x = xy(:,1)

>> y = xy(:,2)

>> plot(x,y)

marker

Import from EXCELImport from EXCEL

� ��/�#�!��(# xls �� EXCEL

>> clear

>> xlsread(‘xy.xls’)

>> x = xy(:,1)

>> y = xy(:,2)

>> plot(x,y)

Lecture 3 - MATLAB 3

Numerical Methods for Numerical Methods for CivilCivil EngineersEngineers




Programming with MATLABProgramming with MATLAB

�� Working with MWorking with M --FilesFiles

�� Script MScript M --FilesFiles

�� Function MFunction M --FilesFiles

�� Flow ControlFlow Control

Working with MWorking with M --FilesFiles

OverviewOverview

�� MATLAB �� ก��ก�� !"�#��$��%$&�$�'��#��'��'

��(��ก��$�� text �%$&)�� !��ก"$ .m ��ก'�� M-file

�!��(��%$&! �� filename.m #�'�� filename ก�*+��,�#��ก��ก�!��ก�� -��+'��-�! ��.-�#��/��$�'�� MATLAB

��!�0�# � scripts .1��,�!"�#��)��

�$+ functions .1�� ก�!�*+��#�� input

�$+�� #�� output

Types of MTypes of M --FilesFiles

Open a BuiltOpen a Built --in Editorin Editor

��'��*+�� M-file .1��,��%$& text ��!� editor �!�� Notepad � � Wordpad

ก�� (��!� built-in editor ��ก� MATLAB *+�+�'กก'��ก

ก��ก�!� built-in editor ��$ �ก��/ File > New > Blank M-file

� �#$0ก�"2� New M-File

Script MScript M --FileFile

�� M-file ��!��#� �� % ��0�� *+��#'��)0 ��ก��!��

$�� M-file ��-�! ��'�� prod_abc.m ��/��$��

A Script to Plot FunctionsA Script to Plot Functions

trigplot.m

t = linspace(0,2*pi);

y1 = sin(t);

y2 = cos(t);

y3 = y1.*y2;

plot(t,y1,’-’,t,y2,’.’,t,y3,’—’);

>> trigplot

Replace the plot statement with

plot(t,y1,’-’,t,y2,’:’,t,y3,’—’);

axis([0 2*pi -1.5 1.5])

legend(‘sin(t)’,’cos(t)’,’sin(t)*cos(t)’);

>> close all; trigplot

Use TEX notation to display “ θθθθ”

legend(‘sin(\theta)’,’cos(\theta)’,

’sin(\theta)*cos(\theta)’);

xlabel(‘\theta (radius)’,’FontName’,’Times’,

’FontSize’,14)

>> close all; trigplot

>> x = linspace(0,2*pi);>> y1 = sin(x);>> y2 = cos(x);>> y3 = y1.*y2;>> plot(x,y1,'-',x,y2,':',x,y3,'--');>> axis([0 2*pi -1.5 1.5])>> legend('sin(\theta)','cos(\theta)','sin(\theta)*cos (\theta)')>> xlabel('\theta (radius)','FontName','Times','Fon tSize',14)>> title('Plot of simple trigonometric functions',.. .

'FontName','Times','FontSize',12)

FunctionFunction MM--FilesFiles

Command Window>>

input parameters

function

output parameters

function [outputParameterList] = functionName (inputPramaterList)

��ก*�ก Built-in function ��ก� MATLAB �$�' ��(��%3�ก&!��1-��!�� .1��*+��(�� #��0��&��#��'4�$�'��ก$� 5$$��)&��

Function Definition LineFunction Definition Line

��ก�� M-file �� ก MATLAB '��,�%3�ก&!�� 0��'�#�'�� function

�� functionName *+��,�! ��'ก� M-file �� # � functionName.m

pyt.m

function h = pyt(a, b)

h = sqrt(a.^2 + b.^2);

>> a = 7.5

>> b = 3.342

>> c = pyt(a,b)

c =

8.2109

Input argument

function name

output argument

Creating a Function MCreating a Function M --FileFile

1 �� M-file ��!� text editor :

2 ��ก�!� M-file *�ก��#�� *�ก M-file � ��

Creating PCreating P--Code Code FilesFiles

You can convert average.m into a pseudocode called P- code file.

>> pcode average

Text file:can be viewedby any editor

Pseudocode:can’t be viewedby any editor

- Faster for large program

- Use to hide algorithm

FLOW CONTROLFLOW CONTROL

MATLAB has several flow control constructs:

•• ifif

•• switch & caseswitch & case

•• forfor

•• whilewhile

•• continuecontinue

•• breakbreak

if Conditional Control

if condition

expression

elseif condition

expression

else

expression

end

Condition:

Equal A == B

Not equal A ~= B

Greater A > B

Smaller A < B

Greater or equal A >= B

Smaller or equal A <= B

AND &

OR |

The if statement evaluates a logical expression and executes a group of statements when the expression is true. The optional elseif and else keywords provide for the execution of alternate groups of statements. An end keyword, which matches the if, terminates the last group of statements.

if a < 0disp(‘a is negative’);

end

Example 1:

if a < 0, disp(‘a is negative’); end

Example 2:

if x >= yc = x^2 - y;

elseif y/x > 2.0c = log(y/x);

elsec = x + y;

end

Example 3:

switch expression

case value1

block of statements

case value2

block of statements

.

.

.otherwise

block of statements

end

switch sign(x)case -1

disp(‘x is negative’);case 0

disp(‘x is exactly zero’);case 1

disp(‘x is positive’);otherwise

disp(‘sign test fail’);end

switch & caseThe switch statement executes groups of statements based on the value of a variable or expression. The keywords case and otherwise delineate the groups. Only the first matching case is executed. There must always be an end to match the switch.

>> P = zeros(5, 5);

>> for k = 1:5for l = 1:5

P(k, l) = pyt(k, l);end

end

>> P

forThe for loop repeats a group of statements a fixed,

predetermined number of times.

A matching end delineates the statements.

while condition

expressions

end

Break Loops break

>> x=1;>> while 1+x > 1

x = x/2;end

>> x

Return Loops return

return

break

LOOP

whileThe while loop repeats a group of statements an indefinite number of times under control of a logical condition.

A matching end delineates the statements.

Here is a complete program, illustrating while, if, else, and end, that uses interval bisection to find a zero of a polynomial.

a = 0; fa = -Inf;b = 3; fb = Inf;while b-a > eps*b

x = (a+b)/2;fx = x^3-2*x-5;if sign(fx) == sign(fa)

a = x; fa = fx;else

b = x; fb = fx;end

endx

The result is a root of the polynomial x3 - 2 x - 5 , namely

x =2.09455148154233

a

fa

fb

bx





Lecture 4 : Approximations & Errors

� Significant Figures

� Error Definitions

� Round-off Errors

� Truncation Errors

� Taylor Series

• For many engineering problems, we cannot obtain

analytical solutions.

• Numerical methods yield approximate results that are close to the exact analytical solution.

• How confident we are in our approximate result?

• The question is “how much error is present in our calculation and is it tolerable?”

Approximation & Approximation & ErrorError

• Modeling Error (�� )

• Round-off Error (��กก��)

• Truncation Error (��กก��)

• Data Error (��ก ��)

• Human Error (��ก�� !")

Sources of Sources of ErrorError

Composite surfaces andterrain modellingMixed approximation methods

Significant FiguresSignificant Figures

Car speed = ?

49 km/h48.8 km/h48.9 km/h48.86 km/h

2 digits3 digits3 digits4 digits

ZERO is NO significant: 184500, 0.1845, 0.01845, 0.00001845 are 4 digits

Significant FiguresSignificant Figures

� Number of significant figures indicates precision.

� Significant digits of a number are those that can be used with confidence, e.g., the number of certain digits plus one estimated digit.

53,800 How many significant figures?

5.38 x 104 3

5.380 x 104 4

5.3800 x 104 5

� Zeros are sometimes used to locate the decimal point not significant figures.

0.00001753 4

0.0001753 4

0.001753 4

Use of Significant Figures in Numerical MethodsUse of Significant Figures in Numerical Methods

1) Numerical method yield approximation results. So we have

to specify how confident in our approximated results.

2) Specific quantities such as π , e , or cannot be expressed

exactly by a limited number of digits.

7

π = 3.141592653589793238462643 . . .

But computers can contain only . . .

>> format short ; pi = 3.1416 (5 digits)

>> format long ; pi = 3.14159265358979 (15 digits)

Try also exp(1), sqrt(7)

* Omission of the remaining significant figures is called round-off error.

Accuracy and PrecisionAccuracy and Precision

Increasing accuracy

Incr

easi

ng p

reci

sion

Computes valueclose to true value

Computed value close to each other

A C C U R A C Y

P R E C I S I O N

B I

A S

UNCERTAINTY

• Accuracy. How close is a computed or

measured value to the true value

• Precision (or reproducibility). How close is a

computed or measured value to previously

computed or measured values.

• Inaccuracy (or bias). A systematic deviation

from the actual value.

• Imprecision (or uncertainty). Magnitude of

scatter.

Error DefinitionsError Definitions

True value = Approximation + Error

True Error, Et = True value – Approximation

��

True fractional relative error = True Error

True value

×××× 100%True Error

True valueTrue percent relative error, εεεεt =

�� ก�� !"�ก#$%�&��'�()��

εa =current approximation - previous approximation

current approximation100%

Approximation Approximation ErrorError

��ก��$�� %ก&�� (��ก+&��)�� (��)

�%��%-��.�)�� ก�� &��

×××× 100%Approximate Error

ApproximationPercent Approximate error, εεεεa =

��ก��ก��"/��'�(ก�� 0- ��(1��%-� ��&��2.-��(1

εs = (0.5×102-n)%

��3��ก ��2.-�� (��%4&�� Prespecified tolerance, εs

�� |εa| < εs �� )4�%-��ก)��.��5��6��(�� n

EXAMPLE 4.1 Error Estimates for Iterative Methods

= + + + + +⋯2 3

12! 3! !

nx x x x

e xn

Approximate e0.5 by using the series

true value: >> format long ; exp(0.5)

>> ans =

1.648721 . . .

1st estimate: ex = 1

2nd estimate: ex = 1 + x e0.5 = 1 + 0.5 = 1.5

true error: ε −= × =

1.648721 1.5100% 9.02%

1.648721t

approximate error: ε −= × =

1.5 1100% 33.3%

1.5a

Terms Results εεεεt (%) εεεεa (%)

123456

11.51.6251.6458333331.6484375001.648697917

39.39.021.440.1750.01720.00142

33.37.691.270.1580.0158

After six terms are included, the approximate error falls below εa = 0.05%,and the computation is terminated.

>> Term = 1:6

>> et = [39.3 9.02 1.44 0.175 0.0172 0.00142]

>> ea = [33.3 7.69 1.27 0.158 0.0158]

>> plot(Term,et,'o-‘,Term(2:6),ea,'+-')

ROUND-OFF

ROUNDROUND--OFFOFF ERRORSERRORS

>> format long e

>> 2.6 + 0.2

>> ans + 0.2

>> ans + 0.2

Creation of erroneous digit

� Computers retain only a fixed number of significant figures

� All computations in MATLAB are done in double precision

FORMAT LONG E : Floating point format with 15 digits.

True value π = 3.141592653589793238462643 . . .

MATLAB pi = 3.141592653589793e+000

% display all of the significant digits

Computer Representation of NumbersComputer Representation of Numbers

Numbers on the computers are represented with a binary (base 2) system.

1 0 1 0 1 1 0 1

27 26 25 24 23 22 21 20

1 × 1 = 1

173

0 × 2 = 01 × 4 = 41 × 8 = 80 × 16 = 01 × 32 = 320 × 64 = 01 × 128 = 128

We use the decimal (base 10) system

Bits Bits && BytesBytes

BIT = Binary digit (0 or 1)

= 0 = 1

BYTE = Group of 8 bits

= 0001 1011

= 27

MAX BYTE NUMBER = 1111 1111 = 27+26+25+24+23+22+21+20 = 255

Integer RepresentationInteger Representation

Computer uses 2 bytes = 16 bits & 1st bit for sign

1 0 0 0 0 0 1 0 1 1 0 1 0 0 1 1

Sign Number

Integer range: [ -32768 to 32767 ]

Upper limit = 214 + 213 + . . . + 22 + 21 + 20

= 215 - 1 = 32,767

Zero = 0000 0000 0000 0000

- Zero = 1000 0000 0000 0000

Redundant

Mantissa

Signed exponent

37.5 = 0.3750 × 102

-812.5 = - 0.8125 × 103

0.005781 = 0.5781 × 10-2

FloatingFloating --point Numberspoint Numbers

Stored in binary equivalent of scientific notation

123.456 = 123.456 × 100 = 1.23456 × 102 = 123456 × 10-3

MATLAB uses the IEEE floating-point standard

Sign

Signed exponent Mantissa

How computer stored floating-point number

Sign

Single Precision 32-bitSign1 bit

Signed exponent8 bit

Mantissa23 bit

Range: +1.18 × 10-38 to +3.40 × 1038

−3.40 × 1038 to −1.18 × 10-38

Double Precision 64-bit

Sign1 bit

Signed exponent11 bit

Mantissa53 bit

Range: +2.23 × 10-308 to +1.80 × 10308

−1.80 × 10308 to −2.23 × 10-308

Single & Double PrecisionSingle & Double Precision

MATLAB Discrete ApproximationMATLAB Discrete Approximation

underflow

underflow usable rangeusable range overflowoverflow

-10+308 -10-308 10-308 103080

denormal

- realmax - realmin realmin realmax

>> format long e

>> 10*realmax

>> realmin/10

>> realmin/1e16

denormal (fewer significant digits)

Effect of Order of OperationsEffect of Order of Operations

0.99 + 0.0044 + 0.0042 = 0.9986

3-Digits: (0.99 + 0.0044) + 0.0042 = 0.998

0.99 + (0.0044 + 0.0042) = 0.999

x approximate x* to t significant digits when

*

5 10 tx x

x−−

< ×

Example: 4 30.998 0.99866.012 10 5 10

0.998− −−

= × < ×

= 3 significant digits

TRUNCATION ERRORSTRUNCATION ERRORS

Example: Derivative of velocity 1

1

( ) ( )i i

i i

v t v tdv v

dt t t t+

+

−∆≅ =

∆ −

dv

dt

v

t

∆∆

Time

Vel

ocity Continuous

ti ti+1

Discrete

Truncation error results from using an approximation in place ofan exact mathematical procedure.

Bungee Jumper ProblemBungee Jumper Problem

rate of change of velocity with respect to time,

2vm

cg

dt

dv d−=

where v = vertical velocity (m/s), t = time (s),

g = gravity acceleration ( ≅ 9.81 m/s2)

cd = drag coefficient (kg/m)

m = jumper’s mass (kg)

Analytical solution by solving differential equation,

= t

m

gc

c

gmtv d

d

tanh)(xx

xx

ee

eex −

−

+−

=)tanh(

Example: Compute velocity of a free fall bungee jumper with a mass of 70 kg.Use a drag coefficient of 0.25 kg/m.

)1872.0tanh(41.5270

)25.0(81.9tanh

25.0

)70(81.9)( tttv =

=

>> t=0:2:12;>> v=52.41*tanh(0.1872*t);>> [t’ v’]

0 02.0000 18.75414.0000 33.25066.0000 42.38298.0000 47.4160

10.0000 49.987412.0000 51.2501

ans =

>> plot(t,v)0 2 4 6 8 10 12

0

10

20

30

40

50

60

Time, s

Vel

ocity

, m/s

Analytical Solution for the bungee jumper problem

Terminal velocity

EulerEuler ’’s method:s method:

Numerical Solution to the differential equationNumerical Solution to the differential equation

tti+1ti

v(ti)

v(ti+1)

True slopedv/dt

Approx. slope∆v

∆t

ii

ii

tttvtv

tv

−−

=∆∆

+

+

1

1 )()(

ii

ii

tttvtv

tv

dtdv

−−

=∆∆

≅+

+

1

1 )()(

Rate of change of velocity can be approximated by

Substitute into dv/dt = g – (cd/m)v2 to give

2

1

1 )()()(

id

ii

ii tvmc

gtt

tvtv−=

−−

+

+

Rearrange equation to yield

)()()()( 12

1 iiid

ii tttvmc

gtvtv −

−+= ++

tdtdv

tvtv iii ∆+=+ )()( 1 New value = old value x step size

Example 2: Numerical Solution to the Bungee Jumper Problem

Perform the same computation as previous example but use

the Euler’s method. Employ a step size ∆t = 2 sec.

@ start ti = 0 s, ti+1 = 2 s, v(0) = 0 m/s:

)()()()( 12

1 iiid

ii tttvmc

gtvtv −

−+= ++

m/s 62.192)0(7025.0

81.90)2( 2 =×

−+=v

Next step ti = 2 s, ti+1 = 4 s, v(2) = 19.62 m/s:

m/s 49.362)62.19(7025.0

81.962.19)4( 2 =×

−+=v

>> 0 [Enter]>> ans+(9.81-(0.25/70)*ans^2)*2 [Enter][Up Arrow] [Enter] many time . . .

MATLAB:

t(s) v(m/s)0 02 19.624 36.496 46.608 50.7110 51.9612 52.30

>> hold on>> v2 = [0 19.62 36.49 46.6 50.71 51.96 52.3]>> plot(t,v2,'+-')

0 2 4 6 8 10 120

10

20

30

40

50

60

Time, s

Vel

ocity

, m/s

Analytical Solution for the bungee jumper problem

Terminal velocity

Analytical solution

Approx. solution

T A Y L O R S E R I E ST A Y L O R S E R I E S

• It is common practice to use a finite number of

terms of the series to approximate a function.

• Taylor series is a representation of a function as an

infinite sum of terms calculated from the values of its

derivatives at a single point.

• It is named after the English mathematician Brook Taylor .

• If the series is centered at zero, the series is also

called a Maclaurin series , named after the Scottish

mathematician Colin Maclaurin .

x

f(x)


a

f(a)f(x)

x

The Taylor series of a function f(x)

that is indefinitely differentiable in a

neighbourhood of a, is the power

series

DefinitionDefinition

⋯+−′′′

+−′′

+−′+= 32 )ax(!3

)a(f)ax(

!2)a(f

)ax)(a(f)a(f)x(f

where n! = the factorial of n

f(n)(a) = the n-th derivative of f at point a

( 1)1( )

( 1)!

nn

n

fR h

n

ξ++=

+Remainder:

nin

iiiii h

nxf

hxf

hxf

hxfxfxf!

)(!3

)(!2

)()()()(

)(32

1 ++′′′

+′′

+′+≅+ ⋯

nni

nii

iii Rhn

xfh

xfh

xfhxfxfxf +++

′′′+

′′+′+=+ !

)(!3

)(!2

)()()()(

)(32

1 ⋯

A remainder term, Rn is included to account for all terms from n+1 to ∞.

TTayloraylor SSerieseries ((nnthth--order approximation)order approximation)

Let’s change a to x i and x to x i+1 and define the step size as h = (x i+1- x i),

the series becomes:

The complete Taylor series is given by

Note that, an equal sign replaces the approximate sign.

Where ξξξξ is not known but lies somewhere between xi and xi+1

xi ξξξξ xi+1

Second

order

f(xi+1) ≅ f(xi) + f’(xi)h + f’’(xi)h2/2!

h

f(xi+1) ≅ f(xi) + f’(xi)h

First order

Zero-order approx .f(xi+1) ≅ f(xi)

True

xi+1

f(xi+1): want to know


Predict function value of one point in term of function value and its derivatives at other points

xi

f(xi)

Known

x

f(x)

Slope @ xi

Example: Approximation of function cos( x) by Taylor Series Expansion

Use Taylor series expansions with n = 0 to 6 to approximate f(x) = cos x at xi+1 = π /3

on the basis of the value of f(x) and its derivatives at xi = π /4.

Step size: h = π /3 - π /4 = π /12 True value: cos(π /3) = 0.5

Zero-order approximation: f(π /3) ≅ cos(π /4) = 0.707106781

which represents a percent relative error of

%4.41%1005.0

707106781.05.0=×

−=tε

First-order approximation: f’(x) = -sin(x)

f(π /3) ≅ cos(π /4) – sin(π /4)(π /12) = 0.521986659

which has εt = 4.40%

Solution:

Second-order approximation: f’’(x) = -cos(x)

which has εt = 0.449%

497754491.0122

)4/cos(124

sin4

cos3

2

=

−

−

≅

ππππππ

f

Order n

0

1

2

3

4

5

6

Approximate

0.7071067810.521986659

0.497754491

0.499869147

0.500007551

0.500000304

0.499999988

εεεεt(%)

41.44.40

0.449

2.62 x 10-2

1.51 x 10-3

6.08 x 10-5

2.44 x 10-6

To get more accurate approximation, we can use higher order (add more terms)

and/or reduce the step size h.

MaclaurinMaclaurin SSerieseries

��7� special case 2� Taylor Series �� ก �� a = 0

Taylor Series :

⋯+−′′′

+−′′

+−′+= 32 )ax(!3

)a(f)ax(

!2)a(f

)ax)(a(f)a(f)x(f

Taylor Series with a = 0 :

⋯+′′′

+′′

++= 32 x!3

)a(fx

!2)a(f

x)0('f)0(f)x(f

Using this formula Taylor series of functions can often be rather

easily computed.

TTayloraylor Series of a Function: Series of a Function: exp(xexp(x ))

� Compute several derivatives of the given function.

� Evaluate these derivatives at x = 0.

,e)x(f x= ,e)x(f x=′ ,e)x(f x=′′ …, x)n( e)x(f =

,1e)0(f 0 == ,1)0(f =′ ,1)0(f =′′ …, 1)0(f )n( =

⋯+′′′

+′′

++= 32 x!3

)a(fx

!2)a(f

x)0('f)0(f)x(f

� Substitute into the series,

2 3

( ) 1 ...2! 3! !

nx x x x

f x e xn

= = + + + + +

MATLAB Example: MATLAB Example: TTayloraylor Series of Series of exp(xexp(x))

>> clear

>> x = -3:0.1:3;

>> y = exp(x);

>> plot(x,y)

>> y0 = ones(size(x));

>> plot(x,y,x,y0)

>> y1 = y0 + x;

>> plot(x,y,x,y1)

>> y2 = y1 + x.^2/2;

>> plot(x,y,x,y2)

>> y3 = y2 + x.^3/6;

>> plot(x,y,x,y3)

�� y4, y5, y6 ... Until then ?

MATLAB Example2: Taylor Series of exp(x) Symbolic Toolbox

>> clear

>> syms x;

>> y = taylor(exp(x),5,x,0)

% declare symbolic x

% expand ex about x = 0 to order 5.

y =

x^4/24 + x^3/6 + x^2/2 + x + 1

Compare at x = 2 with the exact value.

>> approx = subs(y,x,2);

>> exact = exp(2);>> et = (1-approx/exact)*100

et =

5.2653

Plot graph with ezplot :

>> ezplot(y)

>> hold on>> ezplot(exp(x))>> hold off

Which line is an approx.red or blue ?

TaylorTaylor Approximation ErrorApproximation Error

How accurate is the Taylor series approximation?

The n terms of the approximation are the first n terms of the exact expansion:

!2x

x1e2

x ++= ⋯++!3

x3

approximate truncation error

So the function f(x) can be written as the Taylor series approximation plus

an error (truncation) term:

f(x) =Pn(x) + Rn

( 1)1( )

( 1)!

nn

n

fR h

n

ξ++=

+Remainder:

3 5 2 1

sin( ) ( 1)3! 5! (2 1)!

nnx x x

x xn

−

= − + − − −−

⋯

2 4 2

cos( ) 1 ( 1)2! 4! (2 )!

nnx x x

xn

= − + − + −⋯

Taylor Series Expansions for Some Common FunctionsTaylor Series Expansions for Some Common Functions

for all x

for all x

2 3

12! 3! !

nx x x x

e xn

= + + + + +⋯ for all x

2 3 4

ln(1 ) ( 1)2! 3! 4! !

nnx x x x

x xn

+ = − + − − − −⋯ -1 1x≤ ≤

3 5 2

arctan( ) ( 1)3! 5! (2 1)!

nnx x x

x xn

= − + − − −−

⋯

2 3( 1) ( 1)( 2)(1 ) 1

2! 3!p p p p p p

x px x x− − −

+ = + + + +⋯ for 1x <

-1 1x≤ ≤

( 1)1 1( )

( )( 1)!

nn n

n

fR h O h

n

ξ++ += =

+

BigBig --O NotationO Notation

f(x) =Pn(x) + O(hn+1)

Big-O is used to describe the error term in an approximation.

For example:

)x(O!2

xx1e 3

2x +++= 0xas →

expresses the fact that the error, the difference

++−

!2x

x1e2

x

is smaller in absolute value than some constant time |x3|

when x is close enough to 0.

Example : Estimate the error of the approximation

From MATLAB example 2, let’s estimate the error in the interval [-1, 1]

[-1,1]

!4x

!3x

!2x

x1e432

x ++++≈

)x(O!4

x!3

x!2

xx1e 5

432x +++++=

The error done when approximating exp(x) by a Taylor series of degree

5 is bounded by the absolute value of the first term left out.

%8333.0100!5

1100

!5

x 5

t =×≤×≤ε for -1 ≤ x ≤ 1

Example: Taylor series approximation of function sin(x)

Plot the first n = 1 to 6 terms of the Taylor series of ⋯−+−=1206

)sin(53 xx

xx

>> x = -2*pi:pi/30:2*pi;>> y = sin(x);plot(x,y)>> axis([-2*pi 2*pi -1.5 1.5])>> hold on>> y1 = x; plot(x,y1)>> y2 = x-x.^3/6; plot(x,y2)>> y3 = x-x.^3/6+x.^5/120; plot(x,y3)>> legend('sin(x)','n = 1','n = 2','n = 3')

-6 -4 -2 0 2 4 6-1.5

-1

-0.5

0

0.5

1

1.5

sin(x)n = 1n = 2n = 3

f x =





Lecture 5 : Roots of Equations

� Graphical Methods

� Bracketing Mehods

� Open Methods

� MATLAB Methods

What Are Roots?What Are Roots?

Quadratic formula: f(x) = ax2 + bx + c = 0

aacbb

x2

42 −±−=

x

f(x)

0 x1 x2

Roots = Values of x that make f(x) = 0

Open Methods: require only a single starting value of x without bracketing the root.

� Fixed-point iteration

� Newton-Raphson method

� Secant method

Root Finding MethodsRoot Finding Methods

Two major classes of methods:

Bracketing methods: start with guesses that bracket, or contain, the root and then systematically reduce the width of the bracket

� Graphical method

� False-position method

� Bisection method

f(xf(x ) = 0) = 0

x = ?x = ?

GRAPHICAL METHODSGRAPHICAL METHODS

Plot the function and observe where it crosses the x axis.

x

f(x)

0

x* = ?

Graphical method : NOT precise

ZOOM

ZOOM

Double root?

BracketingBracketing MethodMethod

Coarse level search for roots over large interval

1) Subdividing large interval into smaller subinterval

2) Examine sign at the end of each subinterval

+ + + + + + + + +- - - - - - - - - -

+ + + +

+ + + + - - - + +f (x)

xx1 x2

Possible ways in an intervalPossible ways in an interval

Part (a) and (c) : f(xl) and f(xu) same sign

No roots or Even number of roots

Part (b) and (d) : f(xl) and f(xu) different signs

One roots or Odd number of roots

Exceptions:Exceptions:

(a) Multiple roots

(b) Discontinuous function

0.00

Example: Graphical Methods

Plot the function and observe where it crosses x axis

( )0.147667.38( ) 1 40xf x e

x−= − −

f(x)

x4 8 12

16

200

-10

20

40

x f(x)

48

121620

34.11517.653

6.067-2.269-8.401

Observe: x ≈ 15 → f (15) = -0.4133

Interpolation (False-position method)

Assume as linear function for short interval

12 16

6.607

2.269

Interpolation:6.607

12 (16 12) 14.9776.607 ( 2.269)

x = + − =− −

f (14.977) = -0.3691

f (15) = -0. 4133,f (14) = 1.582

2nd Interpolation: 1.58214 (15 14) 14.793

1.582 ( 0.4133)x = + − =

− −f (14.793) = -0.013

Repeatedly halve interval while bracketing root

f (x)

xxa

xb

1) Choose interval [ xa , xb ] which has sign-change

+

-

2) Compute midpoint of interval xm = ( xa + xb ) / 2

xm

+

3) Select subinterval which has sign-change and repeat 2)

-

Bisection Method

Example: Use bisection method find the root of equation

( )0.147667.38( ) 1 40xf x e

x−= − −

f(x)

x4 8 12

16

200

-10

20

40

Initial interval[ 12 , 16 ]

Estimate root at midpoint:

12 1614

2

14.7802 14100% 5.279%

14.7802

r

t

x

ε

+= =

−= × =

True value of the root:14.7863

f (14) = 1.582→ [ 14 , 16 ]+ -

Select interval with sign-change:

Next estimation: xr = (14+16)/2 = 1515 14

100% 6.667%14aε−

= × =

< εs = 0.5%

Termination Criteria and Error EstimatesStop computation when εa < εs , Ex. εs = 0.5%

Iteration xa xb xr εa(%) εt(%)

123456

12141414.514.7514.75

161615151514.875

141514.514.7514.87514.8125

6.6673.4481.6950.8400.422

5.2791.4871.8960.2040.6410.219

• Simple Fixed-Point Iteration

• Newton-Raphson Method

• Secant Methods

• MATLAB function: fzero

• Polynomials

Open MethodsOpen Methods

� How to construct the magic formulae g(x)?

� How can we ensure convergence?

� What makes a method converges quickly or diverge?

� How fast does a method converge?

What you should know about Open MethodsWhat you should know about Open Methods

Fixed-Point Iteration

xxgxf =⇔= )(0)(

� Also known as one-point iteration or successive substitution

� To find the root for f(x) = 0 , we reformulate f(x) = 0 so that there is an

x on one side of the equation.

� If we can solve g(x) = x , we solve f(x) = 0 .

x is known as the fixed point of g(x) .

given with)( 01 xxgx ii =+

� We solve g(x) = x by computing

until x i+1 converges to x.

Fixed-Point Iteration Procedure

3) Predict new xi+1 as a function of old xi

Iterative until satisfy

1

1

100%i ia

i

x x

xε +

+

−= ×

1) Rewrite original equation f(x) = 0 into another form x = g(x).

4) Use iteration x i+1 = g(x i) to find a value that reaches convergence.

xxxx

xxxx

+=⇒=

+=⇒=+−

sin 0sin2

3 032

22Example:

2) Select initial value x0

xi+1 = g (xi)

i xi εa (%)

0123456789

10

01

0.36790.69220.50050.60620.54540.57960.56010.57110.5649

100171.846.938.317.411.25.903.481.931.11

0 1 2 3 4 5 6 7 8 9 10 11 120

0.2

0.4

0.6

0.8

1

Number of Iteration

Ap

pro

xim

ate

d R

oo

t

True value = 0.5671

>> 0>> exp(-ans)

Example: Finding root of f(x) = x - e-x = 0 ans = 0.5671

x = e-x

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

Two-Curve Graphical Method

>> x = 0 : 0.1 : 1;

>> plot(x,exp(-x)-x)

>> x = 0 : 0.1 : 1;

>> plot(x,x,x,exp(-x))

( ) xf x e−=

( )f x x=

0 0.2 0.4 0.6 0.8 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

x

f(x)

= e

xp(-

x) -

x Root

( ) xf x e x−= −

Example: Beam Deflection

( )xLxLxEIL

wy 43250 2

120−+−=

Given:

w0 = 1.75 kN/cm

E = 50,000 kN/cm2;

I = 30,000 cm4; and

L = 450 cm;

Uniform beam subject to linearly increasing, distributed load.

Deflection, y, given by:w0

L Find the point of maximum deflection

( ) 065120

42240 =−+−= LxLxEIL

w

dx

dy

Max. deflection occurs at x where

i

iii

xL

xLxgx

2

44

16

5)(

+==+

Using fixed point iteration, we can rearrange this equation:

( )xLxLxEIL

wy 43250 2

120−+−=

Start at x0 = 100 cm and L = 450 cm

>> L = 450

>> 100

>> (L^4+5*ans^4)/(6*L^2*ans)

x1 = 341.6152

x2 = 262.8564

x3 = 203.1365

x4 = 201.4507

...

Enter

Converge @ x = 201.2461 cm

Max. deflection occurs at x = 201 cm

>> x = 201;

>> w0 = 1.75;

>> EI = 5e4*3e4;

>> y=w0/(120*EI*L)*(-x^5+

2*L^2*x^3-L^4*x)

y = -0.1141 cm Ans.

032)( 2 =−−= xxxf

There are infinite ways to construct g(x) from f(x).

(ans: x = 3 or -1)For example,

So which one is better ?

32)(

32

32

0322

2

+=⇒

+=⇒

+=⇒

=−−

xxg

xx

xx

xx

Case a:

2

3)(

2

3

03)2(

0322

−=⇒

−=⇒

=−−⇒

=−−

xxg

xx

xx

xx

Case b:

2

3)(

2

3

32

032

2

2

2

2

−=⇒

−=⇒

−=⇒

=−−

xxg

xx

xx

xx

Case c:

Case a:

>> 4

>> sqrt(2*ans+3)

3x2x i1i +=+

x1 = 3.3166

x2 = 3.1037

x3 = 3.0344

x4 = 3.0114

x5 = 3.0038

Enter

Converge!

Case b:

>> 4

>> 3/(ans-2)

x1 = 1.5000

x2 = -6

x3 = -0.3750

x4 = -1.2632

x5 = -0.9194

x6 = -1.0376

x7 = -0.9909

x8 = -1.0031

Enter

Converge, but slower

Case c:

>> 4

>> (ans^2-3)/2

x1 = 6.5000

x2 = 19.6250

x3 = 191.0703

Enter

Diverge!

)2x/(3x i1i −=+ 2/)3x(x 3i1i −=+

Convergence

00 0.2 0.4 0.6 0.8 1

0.2

0.4

0.6

0.8

1

x

y

y1 = xy2 = g(x)

0.3679

0.6922

0.5005

0.6062

Divergence

00 0.2 0.4 0.6 0.8 1

0.2

0.4

0.6

0.8

1

x

y

y1 = xy2 = g(x)

x0

(((( )))) (((( )))) (((( )))) (((( ))))(((( )))) (((( )))) ⋯++++−−−−

′′′′′′′′++++−−−−′′′′++++==== ++++++++++++

2i1ii1iii1i xx

!2f

xx xfxfxfξξξξ

• King of the root-finding methods

• Newton-Raphson method

• Based on Taylor series expansion

NewtonNewton ’’s Methods Method

Truncate the Taylor series to get

At the root, f(x i+1) = 0 , so

Newton-Raphson Method

)xx)(x(f)x(f)x(f i1iii1i −′+= ++

)xx)(x(f)x(f0 i1iii −′+= +

)x(f)x(f

xxi

ii1i ′−=+

Newton-Raphson Method

� Use the slope of f(x) to predict the location of the root.

f(xi)

xixi+1

slope = f’(xi)f(x)

Root

x

� xi+1 is the point where the tangent at xi intersects x-axis.

1ii

ixx

0)x(f

+−

−=

)x(f)x(f

xxi

ii1i

′−=+


xe1)x(f −+=′)x(f)x(f

xxi

ii1i

′−=+

>> 0

>> ans-(ans-exp(-ans))/(1+exp(-ans))

x1 = 0.5000

x2 = 0.5663

x3 = 0.5671

Enter

Converge!

i

i

x

xi

ie1

exx

−

−

+

−−=

>> 0>> exp(-ans)

Fixed-Point Iteration:

x1 = 1

x2 = 0.3679

x3 = 0.6922

x4 = 0.5005

x5 = 0.6062

x6 = 0.5454

x7 = 0.5796

x8 = 0.5601

x9 = 0.5711

Enter

root

x*

04x3xxf 4 ====−−−−++++====)(

xixi+1

Newton’s method - tangent line

x1

f(x1)

Approximation Sequence of Newton’s Method

f(x)

xx3

x2

f(x2)

Failure of Newton’s Method

Case 1: Inflection point in vicinity of root

f(x)

x

x1x2 x3


Case 2: Oscillate around local maximum or minimum

f(x)

x


Case 3: Jump away for several roots

f(x)

x


Case 4: Disaster from zero slope

f(x)

x

• No general convergence criteria for Newton-Raphson

method.

• Convergence depends on function nature and accuracy of initial guess.

– A guess that's close to true root is always a better choice

– Good knowledge of the functions or graphical analysis can help

you make good guesses

• Good software should recognize slow convergence or divergence.

– At the end of computation, the final root estimate should alwaysbe substituted into the original function to verify the solution.

Overcoming the Failure of Newton’s Method

• Newton-Rahpson method converges quadratically (when it converges).

– Except when the root is a multiple roots

• When the initial guess is close to the root, Newton-Rahpson method usually converges.

– To improve the chance of convergence, we could use a

bracketing method to locate the initial value for the Newton-

Raphson method.

Other Facts

for function whose derivatives are difficult to evaluate

Derivative approximated by backward finite difference

( )( ) ( )1

1

i ii

i i

f x f xf x

x x−

−

−′ ≅

−1

( )

( )i

i ii

f xx x

f x+ = −′

( )11

1

( )

( ) ( )i i i

i ii i

f x x xx x

f x f x−

+

−

−= −

−

f(xi)

xixi-1x

f(xi-1)

Secant Method

Example: Use secant method to estimate root of e-x - x = 0

Initial estimate x-1 = 0 and x0 = 1.0

True root = 0.56714329. . .

First iteration:x-1 = 0 f (x-1) = 1.00000

x0 = 1 f (x0) = -0.63212

1

0.63212(0 1)1 0.61270 8.0%

1 ( 0.63212) tx ε− −

= − = =− −

Second iteration:x0 = 1 f (x0) = -0.63212

x1 = 0.61270 f (x1) = -0.07081

2

0.07081(1 0.61270)0.61270 0.56384 0.58%

0.63212 ( 0.07081) tx ε− −

= − = =− − −

Third iteration:x1 = 0.61270 f (x1) = -0.07081

x2 = 0.56384 f (x2) = 0.00518

3

0.00518(0.61270 0.56384)0.56384 0.56717

0.07081 ( 0.00518)

0.0048%t

x

ε

−= − =

− − −

=

Secant Method

Advantage of the secant method -

• It can converge even faster and it doesn’t

need to bracket the root

Disadvantage of the secant method -

• It is not guaranteed to converge!

• It may diverge (fail to yield an answer)

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

-1 0 1 2 3 4 5 6 7

ln(x)

secant

Convergence not Guaranteed

no sign check,

may not bracket the root

y = ln xx1

x2

BisectionFalse position

Secant

Newton’s

Convergence criterion 10 -14

Bisection -- 47 iterationsFalse position -- 15 iterationsSecant -- 10 iterations

Newton’s -- 6 iterations

• Bracketing methods – reliable but slow

• Open methods – fast but possibly unreliable

• MATLAB fzero – fast and reliable

fzero: find real root of an equation (not suitable for double root!)

fzero( function, x0)

fzero( function, [ x0 x1])

MATLAB Function: fzero


>> f = @(x)x-exp(-x);

Write an anonymous function f:

>> z = fzero(f,0)

Then find the zero near 0:

z =

0.5671

Open Methods:

- Fixed-Point Iteration

- Newton’s Method

- Secant Method

( )0.147667.38( ) 1 40xf x e

x−

= − −

f(x)

x4 8 12

16

200

-10

20 Initial interval[ 12 , 16 ]

Example: Finding root of a function ans = 14.7863

>> f = @(x)(667.38/x)*(1-exp(-0.147*x))-40;

Write an anonymous function f:

>> z = fzero(f,[12 16])

Then find the zero between 12-16:

z =

14.7863

Bracket Method:

- Bisection Method

20 1 2( ) n

n nf x a a x a x a x= + + + +⋯

n = order of polynomial

a’s = constant coefficients

where

Roots of polynomials:

(1) n th-order equation has n real or complex roots

(2) If n is odd, at least one root is real.

(3) Complex roots exist with conjugate pairs (λ + µi and λ − µi)

Roots of Polynomials

= 0

Bisection, False-position, Newton-Raphson, Secant methods cannot be

easily used to determine all roots of higher-order polynomials.

0 2 4 6 8 10-1

-0.5

0

0.5

1

1.5

2

2.5

3y

=f(

x)

x

distinctreal roots

repeatedreal roots

complexroots

Roots of Polynomials

MATLAB Function: roots

• Evaluation the root as an eigenvalue problem

• Zeros of a nth-order polynomial

>> x = roots(c) - roots

>> c = poly(r) - inverse function

012

21n

1nn

n cxcxcxcxc)x(p +++++= −− ⋯

]ccccc[c vector tCoefficien 0121nn ⋯−=

Example: Finding root of a function

>> roots([1 -3 2])

ans =

2

1

p(x) = x2 – 3x + 2 = 0

>> roots([1 -10 25])

ans =

5

5

p(x) = x2 – 10x + 25 = 0

>> roots([1 -17 72.5])

ans =

8.5000 + 0.5000i

8.5000 – 0.5000i

p(x) = x2 – 17x + 72.5 = 0

Lecture Lecture 66 Linear Algebraic EquationsLinear Algebraic Equations

� Basic Concepts

� Gaussian Elimination

� Backward Substitution

� LU Decomposition





Linear Algebraic EquationsLinear Algebraic Equations

In many problems of engineering interest, you will need to be

able to solve a set of linear algebraic equations.

The methods can be broadly classified as:

Direct methods Iterative methods

� Gauss-Elimination

� Gauss-Jordan

� LU Decomposition

� Jacobi

� Gauss-Seidel

Linear Algebraic EquationsLinear Algebraic Equations

a11x1 + a12x2 + . . . a1nxn = b1

a21x1 + a22x2 + . . . a2nxn = b2

an1x1 + an2x2 + . . . annxn = bn

.

.

....

n equations

n unknowns

A system of n linear equations in n unknowns,

where all the a’s and b’s are constants.

We need to find all the x’s such that all the above equations are satisfied.

Example:

9475

2382

753

321

321

321

=−+

−=+−

=−+

ccc

ccc

ccc

Matrix Form:

−=

−−

−

9

2

7

475

382

531

3

2

1

c

c

c

A x = bSymbolic Notation

Augment Form:

−−−

−

9475

2382

7531

System of linear algebraic equations A x = b

=

mmmnmm

n

n

b

b

b

x

x

x

aaa

aaa

aaa

⋮⋮

⋯

⋮⋱⋮⋮

⋯

⋯

2

1

2

1

21

22221

11211

where aij and b i are constants, i = 1,2,…,m, j = 1,2,…,n

m = number of rows

n = number of columns

Solution :Solution : A x = b

A-1A x = A-1b

x = A-1b

MATLAB :MATLAB :

>> x = inv(A)*b

- Use matrix inversion,

>> x = A\b

- Use backslash operator,

Requirements for a Solution , Am××××n

m rows and n columns

m = nSolving Ax = b for n unknowns from n equations

m > nOverdetermined systems(e.g., least-squares problems)

m < nUnderdetermined systems(e.g., optimization)

Example 1 – Unique solution

Consider the system

2x + 3y = 8

5x - 6y = 30

Geometrically this corresponds to

two lines crossing in one point.

>> x=-10:1:10;

>> y1 = (8-2*x)/3;

>> y2 = (5*x-30)/6;

>> plot(x,y1,x,y2)

Example 2 – No solution

Consider the system

2x + 3y = 8

4x + 6y = 30

Geometrically this corresponds to

two parallel lines never crossing.

>> x=-10:1:10;

>> y1 = (8-2*x)/3;

>> y2 = (30-4*x)/6;

>> plot(x,y1,x,y2)

Example 3 – Ill-conditioned

Consider the system

2x + 3y = 8

4x + 7y = 15

Geometrically this corresponds to two almost parallel lines barely

crossing. Sensitive to round-off error.

>> x=-10:1:10;

>> y1 = (8-2*x)/3;

>> y2 = (15-4*x)/7;

>> plot(x,y1,x,y2)

rank(A) = number of linearly independent columns in A

For matrix A with m rows and n columns, and Ax = b has a solution

- if rank(A) < n , → infinite number of solutions

- if rank(A) = n , → unique solution

For a system of n equations in n unknowns written in the form Ax = b,

the solution x exists and is unique for any b if and only if rank( A) = n.

Matrix RankMatrix Rank

>> A = [3 -4 1;6 10 2;9 -7 3];>> rank(A) ans = 2

ConsistencyConsistency TestTest

11 12 1 1

21 22 2 21 2

1 2

n

nn

m m mn m

a a a b

a a a bAx b x x x

a a a b

= ⇔ + + + =

⋯⋮ ⋮ ⋮ ⋮

Element of vector x are coefficients in the linear equation

Augmented matrix:

11 12 13 1 1

11 11 11 11 2

11 11 11 11 3

11 11 11 11

n

m

a a a a b

a a a a b

A a a a a b

a a a a b

=

⋯

⋯

ɶ ⋯

⋮ ⋮ ⋮ ⋱ ⋮ ⋮

⋯

Consistent when and have the same rankA Aɶ

= [Ab]

Practical Model: y = α x + β

Example: An Inconsistent System from Data Fitting

Experiment data:x

y

1

2

2

1

3

0.5

0 1 2 3 4-2

-1

0

1

2

3

1 1 2

2 1 1

3 1 0.5

αβ

=

Matrix Form:

>> A=[1 1; 2 1; 3 1]; b=[2; 1; 0.5];

>> rank(A)

>> rank([A b])

Consistency test :

5.03

12

2

=+=+=+

βαβαβα

Data → Model:

2nd Experiment:

=

0

1

2

13

12

11

βα

Consistency test:

>> A = [1 1; 2 1; 3 1]; b = [2; 1; 0];

>> rank(A)

>> rank([A b])

0 1 2 3 4-2

-1

0

1

2

3

Exact solution

=

+

−⇒

0

1

2

1

1

1

)3(

3

2

1

)1(

[ ] [ ]TT31−=βα

Nonsingular CaseNonsingular Case

A x = b x = A-1 b

� If the coefficient matrix A is nonsingular, then it is invertible and we can

solve Ax = b as follows:

� This solution is therefore unique. Also, if b = 0, it follows that the unique

solution to Ax = 0 is x = A-1

0 = 0.

� Thus if A is nonsingular, then the only solution to Ax = 0 is the trivial

solution x = 0.

Matrix

elimination

row operation

Upper triangular

backward

substitution

Solution

Naive Gauss EliminationNaive Gauss Elimination

The most basic systematic scheme for solving system of linear equations.

The procedure consisted of two steps:

1) Forward elimination of the linear equation matrix using row operationto obtain an upper triangular matrix.

2) Backward substitution to solve for the unknowns

Forward eliminationForward elimination

′′′′′′′⇒

333

22322

1131211

3333231

2232221

1131211

ba

baa

baaa

baaa

baaa

baaa

Backward substitutionBackward substitution

1121231311

2232322

3333

/)(

/)(

/

axaxabx

axabx

abx

−−=

′′−′=

′′′′=

SubstitutionsSubstitutions

Backward substitution:Backward substitution:

Forward substitution:Forward substitution:

Upper triangular matrix

nn

nn a

bx =

1,2,,2,1),(1

1

⋯−−=−= ∑+=

nnixaba

xn

ijjiji

iii

Lower triangular matrix

11

11 a

bx =

nixaba

xi

jjiji

iii ,,3,2,1),(

1 1

1

⋯=−= ∑−

=

Pivot row

Pivot element

EXAMPLE : Naive Gauss Elimination

6443

7766

123

321

321

321

−=+−−=+−−=−+−

xxx

xxx

xxx

−−−−−−−

⇒6443

7766

1123 R1

R2

R3

Forward elimination (Row operation):

R2 + 2×R1, R3+R1:

−−−−−−−

⇒7320

9520

1123

R3 - R2:

−−−−−−

⇒2200

9520

1123

Backward substitution:

22

952

123

2200

9520

1123

3

32

321

=−−=+−−=−+−

⇒

−−−−−−

x

xx

xxx

22

952

123

3

32

321

=−

−=+−

−=−+−

x

xx

xxx

12

23 −=

−=⇒ x

2)59(2

132 =−−

−=⇒ xx

2)21(3

1321 =+−−

−=⇒ xxx

>> A = [-3 2 -1;6 -6 7;3 -4 4]>> b = [-1;-7;-6]>> x=A\b

MATLAB:

EXAMPLE : Gauss-Jordan Elimination

Pivot row

Pivot element

6443

7766

123

321

321

321

−=+−−=+−−=−+−

xxx

xxx

xxx

−−−−−−−

⇒6443

7766

1123 R1

R2

R3

Forward elimination (Row operation):

R2 + 2×R1, R3+R1:

−−−−−−−

⇒7320

9520

1123

R3 - R2:

−−−−−−

⇒2200

9520

1123

No backward substitution !

R1/-3

R2/-2R3/-2

Divide diagonal to one:

−

−

−

1100

5.45.210

33.033.067.01

R1 + 0.67R2R2 + 2.5R3

Use “1” diagonal to eliminateto identity matrix:

−

−

1100

2010

33.333.101

R1 + 1.33R3

−1100

2010

2001 ⇒⇒⇒⇒ X1 = 2

⇒⇒⇒⇒ X2 = 2

⇒⇒⇒⇒ X3 = -1

next elimination fail

GaussianGaussian Elimination with PivotingElimination with Pivoting

To prevent failure from zero diagonal elements

Augmented matrix: [ ]

2 4 2 2 4

1 2 4 3 5

3 3 8 2 7

1 1 6 3 7

A A b

− − − − = = − − − − −

ɶ

R1

R2

R3

R4

2 4 2 2 4

0 0 5 2 7

0 3 5 5 1

0 3 5 4 5

− − − − − −

R2 – R1/2,R3 + 3R1/2,R4 + R1/2

First row operation:

P I V O T I N GP I V O T I N G

Exchange row to avoid zero pivot element

2 4 2 2 4

0 0 5 2 7

0 3 5 5 1

0 3 5 4 5

− − − − − −

2 4 2 2 4

0 3 5 4 5

0 3 5 5 1

0 0 5 2 7

− − − − − −

R3=R3-R2

2 4 2 2 4

0 3 5 4 5

0 0 0 1 4

0 0 5 2 7

− − − − − − −

2 4 2 2 4

0 3 5 4 5

0 0 5 2 7

0 0 0 1 4

− − − − − −

x = A \ b

MATLAB

Solving Systems with the Backslash OperatorSolving Systems with the Backslash Operator

A x = b x = A-1 b

A \ . . . means multiply on the left by the inverse of A

>> A = [2 4 -2 -2;1 2 4 -3;-3 -3 8 -2;-1 1 6 -3]

>> b = [-4;5;7;7]

>> x = A\b

x =

1.00002.00003.00004.0000

LULU DecompositionDecomposition

� When Gauss elimination is used to solve a linear system, the row operations are applied simultaneously

to the RHS vector. [ Ax = b 1 ]

� If the system is solved again for a different RHS vector,

the row operations must be repeated. [ Ax = b 2 ]

� The LU decomposition eliminates the need to repeat the row operators each time a new RHS vector is used.

*RHS = Right Hand Side = b

A LU


To solve Ax = b , we start by decompose A into L and U such

that A = LU where

=

=

33

2322

131211

33231

2221

11

00

00

00

u

uu

uuu

lll

ll

l

UL

lower triangularmatrix

upper triangularmatrix

−−−

−

243

131

152

=

−

45.33

05.01

002

−

−

100

110

5.05.21Example:

−−−

1020

610

321


Main idea is to record the steps used in Gaussian elimination.

Consider the matrix:

−−−

=1020

1252

321

A

Gaussian elimination:

R2 – 2××××R1 (2)

For recording

Recording Method:

A

Gaussian elimination (con’t):

−−−

2)2()0(

61)2(

321R3 – 0××××R1

R3 + 2××××R1

−=⇒

120

012

001

L

Lower triangular matrix:

Mysterious coincidence !!!

ALU =

−−−

=

−−

−=

1020

1252

321

200

610

321

120

012

001

>> L = [1 0 0;2 1 0;0 -2 1]

>> U = [1 -2 3;0 -1 6;0 0 2]

>> A = L*U

A =

1 -2 3

2 -5 12

0 2 -10

L

U

Gauss elimination as Gauss elimination as LULU decompositiondecomposition

Gauss elimination can be used to decompose A into L and U as illustrated for a three-equation system,

=

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa Forward

eliminationU=

′′′′

33

2322

131211

00

0

a

aa

aaa

Store row operation @ zeros position:

′′′′

333231

232221

131211

)()(

)(

aff

aaf

aaa

,11

2121 a

af =where ,

11

3131 a

af =

22

3232 and

aa

f′′

=

[A] → [L][U]

′′′′=

33

2322

131211

00

0

a

aa

aaa

U

=

1

01

001

3231

21

ff

fL

EXAMPLE : LU Decomposition with Gauss Elimination

−−

−−=

443

766

123

A

From previous example,

−−

−−=

200

520

123

UAfter forward elimination,

=

333231

232221

131211

aaa

aaa

aaa

′′′′=

33

2322

131211

00

0

a

aa

aaa

,23

6

11

2121 −=

−==

aa

f

,13

3

11

3131 −=

−==

aa

f

122

22

3232 =

−−

=′′

=aa

f

Lower triangular matrix,

−−=

=

111

012

001

1

01

001

3231

21

ff

fL

CheckCheck LULU == AA or not ?or not ?

After the second step of forward elimination

−−

−−⇒

320

520

123

′′′′=

3332

2322

131211

0

0

aa

aa

aaa

First step: R2+2R1

Second step: R3+R1

Third step: R3-R2

Solving Linear System by Solving Linear System by LULU DecompositionDecomposition

The linear system Ax = b can be solved in 3 steps:

(1) Find the LU decomposition of matrix A : A = LU

A x = b ⇒⇒⇒⇒ L U x = b (define Ux = y)

y

(2) Solve the system Ly = b :

=

3

2

1

3

2

1

333231

2221

11

b

b

b

y

y

y

LLL

0LL

00L

Forward Substitution

y1

y2

y3

(3) Solve the system Ux = y :

Backward Substitution

x1

x3

x2

=

3

2

1

3

2

1

33

2322

131211

y

y

y

x

x

x

U00

UU0

UUU

Solving Linear System by Solving Linear System by LULU DecompositionDecomposition

� LU Decomposition replaces the solving single Ax = b system

by solving two systems of Ly = b and Ux = y

� However, the latter systems are easy to solve since the

coefficient matrices are triangular.

Example:Example: Using Using LULU to solve equationsto solve equations

Ax = b → LUx = bLy = b

Ux = y

−−−

=

−−

−−

6

7

1

443

766

123

2

2

1

x

x

x

=

3

2

1

3

2

1

3231

21

1

01

001

b

b

b

y

y

y

ff

f

Step 2 : Ly = b : Forward Substitution

−−−

=

−−

6

7

1

111

012

001

3

2

1

y

y

y

11 by =

21122 fyby −=

32231133 fyfyby −−=

11 −=y

9)2)(1(72 −=−−−−=y

21)9()1)(1(63 =×−−−−−−=y

Step 1 : A = LU : Decomposition � done in previous example

Step 3 : Ux = y : Backward Substitution

=

′′′′

3

2

1

3

2

1

33

2322

131211

00

0

y

y

y

x

x

x

a

aa

aaa

−−

=

−−

−−

2

9

1

200

520

123

3

2

1

x

x

x

33

33 a

yx

′′=

22

23322 a

axyx

′′−

=

11

13312211 a

axaxyx

−−=

12

23 −=

−=x

22

)5)(1(92 =

−−−−

=x

23

)1)(1(2211 =

−−−−×−−

=x

MATLAB Function: MATLAB Function: lulu

>> [L, U] = lu(A)

−−−

=

−−

−−

6

7

1

443

766

123

2

2

1

x

x

x

>> A = [-3 2 -1;6 -6 7;3 -4 4];>> b = [-1; -7; -6];>> [L,U] = lu(A)

LU Decomposition:LU Decomposition:

>> y = L\b>> x = U\y

Solve equations:Solve equations:

2 3

2

3

For Gauss elimination with pivoting:

PA = LU P = identity matrix with same rows switched as A in pivoting.

For example PA : (switch R2 ↔ R3)

−−

−=

−−−

1252

1020

321

1020

1252

321

010

100

001

PAx = Pb ≡ b’ → LUx = b’Ly = b’

Ux = y

LULU Decomposition with pivotingDecomposition with pivoting

ALU =

−−−

=

−−

−=

1020

1252

321

200

610

321

120

012

001

>> A = [1 -2 3;2 -5 12;0 2 -10];

>> [L,U] = lu(A)

>> [L,U,P] = lu(A)

Example:Example: LULU decomposition with pivotingdecomposition with pivoting

From previous example:

P A = L U

001

100

010

−

−

−

1020

1252

321

=

−

−

−

321

1020

1252

R1�R3,

=

125.05.0

010

001

−

−

−

5.000

1020

1252

R2�R1, R3�R2

Next, we want to find the solution of the system: Ax = b

=

−

−

−

3

2

2

x

x

x

1020

1252

321

3

2

1

PAx = Pb ≡ b’ → LUx = b’Ly = b’

Ux = y

001

100

010

3

2

2

=

2

3

2 >> b_dash = P*b

>> y = L\b_dash

>> x = U\y

>> x = A\b

Compare with,

� Linear Fit

� Goodness of Fit

� Quadratic Fit

� Polynomial Fit

� MATLAB polyfit & polyval

Numerical Numerical Methods for Civil EngineersMethods for Civil Engineers

By Asst.Prof.Dr.Mongkol JIRAVACHARADET



Lecture Lecture 77 Curve FittingCurve Fitting

LINEAR REGRESSIONLINEAR REGRESSION

x

y

��ก��

y = αααα x + ββββ

- ��ก�� !��

��ก� �!�� [xi, yi] ��ก��"#��!�� !��

- �$��#�%�� &$�� !��ก��

Error Between Model and ObservationError Between Model and Observation

x

yData: [ xi yi ]

Error: ei = yi – α xi – β

Criteria for a “Best” Fit :

�� %��$�&�� ΣΣΣΣei ��

Model: β+α= xy

y i

x i

y i

BESTBEST line with error minimized ?line with error minimized ?

��ก'�� ei ��&�$��%��$��$��( yi ��%�$��%��)��ก*��

yye ii ˆ−=

��

βα += ixy �$�$��ก�$�%� ��+�$�,�ก

��%�,"#��%��ก��ก�� '��&(��ก��(

�-��ก��ก��ก��!�� ±

��(��

�$�%�'��./��ก��'��)��

ERRORERROR DefinitionDefinition

yye ii ˆ−=

��+��#�(��!�&��ก'��

!�+��ก��&��ก'�� ก�$� least squareleast square

( )∑= 2ieS

LeastLeast --Square Fit of a Straight LineSquare Fit of a Straight Line

��ก� �!�� (xi, yi) ��%��: β+α= ii xy

��: iii yye −= ii xy α−β−=

&��ก'��: ...eeeS 23

22

21r +++=

( )2ii

2ir xyeS α−β−Σ=Σ=

�� ก�� -��12!��%�(�1(3��$�ก�,4��2

0Sr =α∂

∂

0Sr =β∂

∂

��ก��ก��!�� x i ��% y i ��$�� %��$�

0)x)(xy(2S

iiir =−β−α−Σ=

α∂∂

0)1)(xy(2S

iir =−β−α−Σ=

β∂∂

��$�%��$�

iii2i yxxx Σ=Σβ+Σα

ii ynx Σ=β+Σα

�� n ��'��!��

Σ

Σ=

β

α

Σ

ΣΣ

i

ii

i

i2i

y

yx

nx

xx

��!��ก��(ก

�ก��ก��+��-��ก��-��$� αααα ��% ββββ

( )22

1

1

i i i i

i i

x y x yn

x xn

α∑ − ∑ ∑

=∑ − ∑

y xβ α= −

xyxy and of mean the are and where

1xy i i i iS x y x y

n= Σ − Σ ΣDefine:

( )22 1xx i iS x x

n= Σ − Σ

( )22 1yy i iS y y

n= Σ − Σ

y xα β= +

Approximated y for any x is

xy

xx

S

Sα =

Example: Fit a straight line to x and y values

xi yi

1234567

0.52.52.04.03.56.05.5 4286.3

7

24

47

28

7

==

==

=

y

x

n

2

(119.5) (28)(24) / 70.8393

(140) (28) / 7

3.4286 0.8393(4) 0.0714

α

β

−= =

−

= − =

xi2 xi yi

149

16253649

0.55.06.0

16.017.536.038.5

yi2

0.256.25

416

12.2536

30.25

Σ 28 24 140 119.5 105

Least-square fit:

0714.0x8393.0y +=

1 2 3 4 5 6 70

1

2

3

4

5

6

x

y

MATLAB:

>> x = 1:7;

>> y = [0.5 2.5 2.0 4.0 3.5 6.0 5.5];

>> yhat = 0.8393*x+0.0714;

>> plot(x,y,'o',x,yhat)

Goodness of FitGoodness of Fit

Sum of the square of the errors:

( )22

1 1

n n

r i i ii i

S e y xβ α= =

= = − −∑ ∑

Standard errors of the estimation: / 2r

y x

Ss

n=

−

Sum of the square around the mean: ( )2

1

n

t ii

S y y=

= −∑

Standard deviation:2

ty

Ss

n=

−

( )2 /xx yy xy xxS S S S= −

yyS=

Coefficient of determination 2 t r

t

S Sr

S

−=

For perfect fit Sr = 0 and r = r2 = 1

y

sy sy/x

Linear regressionsy > sy/x

2xy

xx yy

S

S S=

r2 ��$��ก��$� y ��ก(��กก��$� x

Example: Error analysis of the linear fit

xi yi

1234567

0.52.52.04.03.56.05.5

0.8393

0.0714

αβ

=

=

3.4286y =

/

22.7143

7 22.131

2.9911

7 20.773

y

y x

s

s

=−

=

=−

=

( yi - β - α xi)2

8.57650.86222.04080.32650.00516.61224.2908

0.16870.56260.34730.32650.58960.79720.1993

Σ 28 24 22.7143 2.9911

( )2

iy y−

St Sr

Since sy/x < sy , linear regression has merit.

22.7143 2.99110.868 0.932

22.7143r

−= = =

Linear model explains 86.8% of original uncertainty.

OR Example: Error analysis of the linear fit

xi yi

1234567

0.52.52.04.03.56.05.5

Σ 28 24

xi2

149

16253649

140

xi yi

0.55.06.0

16.017.536.038.5

119.5

yi2

0.256.25

416

12.2536

30.25

105

2140 28 / 7 28xxS = − =

2105 24 / 7 22.7yyS = − =

119.5 28 24 / 7 23.5xyS = − × =

2(28 22.7 23.5 ) / 28

2.977

rS = × −

=

/

22.72.131

7 2

2.9770.772

7 2

y

y x

s

s

= =−

= =−

Since sy/x < sy , linear regression has merit.

22 23.5

0.86928 22.7

r = =×

Linear model explains 86.9% of original uncertainty.

NonNon --linear Curve Fitlinear Curve Fit

��,�ก�)�� $�� ก�� Linear regression

ก5�%��$��ก�,��(�� error �$��!��

0 1 2 3 4 5-10

0

10

20

30

40

50

60

70

ก��%��)��*��&��ก�$�

22i2i10i

2ir )xaxaay(eS −−−Σ=Σ=

...eeeS 23

22

21r +++=

Quadratic FitQuadratic Fit

��%��)� �!�� (xi, yi) ��*��ก'��: 2i2i10i xaxaay ++=

��: iii yye −=

&��ก'��:

�� ก�� -��12!��%�(�1(3��$�ก�,4��2

2i2i10i xaxaay −−−=

)xaxaay(2aS 2

i2i10i0

r −−−Σ−=∂∂

= 0

)xaxaay(x2aS 2

i2i10ii1

r −−−Σ−=∂∂

= 0

)xaxaay(x2aS 2

i2i10i2i

2

r −−−Σ−=∂∂

= 0

��$�%��$�

i22i1i0 ya)x(a)x(an Σ=Σ+Σ+

ii23i1

2i0i yxa)x(a)x(a)x( Σ=Σ+Σ+Σ

i2i2

4i1

3i0

2i yxa)x(a)x(a)x( Σ=Σ+Σ+Σ

��!��ก��(ก

Σ

Σ

Σ

=

ΣΣ

ΣΣΣ

ΣΣΣ

i

ii

i2i

0

1

2

i2i

i2i

3i

2i

3i

4i

y

yx

yx

a

a

a

nxx

xxx

xxx

�ก��ก��(ก*�� Gaussian elimination �-��$��%�(�1(3 a0, a1 ��% a2

Example: Fit a second-order polynomial to the data.

xi yi

012345

2.17.7

13.627.240.961.1

Σ 15 152.6

( yi - a0 - a1xi - a2xi2)2

0.143321.002861.081580.804910.619510.09439

3.74657

544.44314.47140.03

3.12239.22

1272.11

2513.39

( )2

iy y−

4

2 2

3

2 15 979

6 152.6 585.6

2.5 55 585.6

25.433 225

i i

i i i

i i i

i

m x x

n y x y

x x x y

y x

= ∑ = ∑ =

= ∑ = ∑ =

= ∑ = ∑ =

= ∑ =

From the given data:

Simultaneous linear equations

0

1

2

6 15 25 152.6

15 55 225 585.6

55 225 979 2488.8

a

a

a

=

Solving these equation gives a0 = 2.47857,a1 = 2.35929, and

a2 = 1.86071.

Least-squares quadratic equation:

y = 2.47857 + 2.35929x + 1.86071x2

Coefficient of determination:

2513.39 3.746570.99851 0.99925

2513.39r

−= = =

Polynomial FitPolynomial Fit

��%��)� �!�� (xi, yi) *��*-�(*��ก�� m:

mim

2i2i10i xaxaxaay ++++= ⋯

�'��*��ก��ก��%,,��ก��(��'�� m+1:

Σ

Σ

Σ

Σ

=

ΣΣΣΣ

ΣΣΣΣ

ΣΣΣΣ

ΣΣΣ

++

+

+

imi

i2i

ii

i

m

2

1

0

m2i

2mi

1mi

mi

2mi

4i

3i

2i

1mi

3i

2ii

mi

2ii

yx

yx

yx

y

a

a

a

a

xxxx

xxxx

xxxx

xxxn

⋮⋮

⋯

⋮⋱⋮⋮⋮

⋯

⋯

⋯

MATLAB MATLAB polyfitpolyfit FunctionFunction

For second-order polynomial, we can define

211 1

1222 2

2

32

-1

1

1, ,

1

and show that ( ' ) ' or =

mm m

yx xc

yx xc

cyx x

−

= = =

= 1

A Y C

C A A A Y C A Y

⋮⋮ ⋮ ⋮

>> C = polyfit(x,y,n)

>> [C,S] = polyfit(x,y,n)

x = independent variabley = dependent variablen = degree of polynomial

C = coeff. of polynomial indescending power

S = data structure for polyvalfunction

Fit norm Fit QR

Solving by MATLAB Solving by MATLAB polyfitpolyfit FunctionFunction

>> x = [0 1 2 3 4 5];

>> y = [2.1 7.7 13.6 27.2 40.9 61.1];

>> c = polyfit(x,y,2)

>> [c,s] = polyfit(x,y,2)

>> st = sum ((y-mean(y)).^2)

>> sr = sum((y-polyval(c,x)).^2)

>> r = sqrt((st-sr)/st)

MATLAB MATLAB polyvalpolyval FunctionFunction

Evaluate polynomial at the points defined by the input vector

Example: y = 1.86071x2 + 2.35929x + 2.47857

y = c(1)*x n + c(2)*x (n-1) +...+ c(n)*x + c(n+1)

>> y = polyval(c,x)

where x = Input vector

y = Value of polynomial evaluated at x

c = vector of coefficient in descending order

>> c = [1.86071 2.35929 2.47857]

0 1 2 3 4 50

10

20

30

40

50

60

70

x

y

Polynomial InterpolationPolynomial Interpolation

>> y2 = polyval(c,x)

>> plot(x,y,’o’,x,y2)

Example: Which order is the best polynomial fit?

xy

00

10.84

20.91

30.14

4-0.76

5-0.96

6-0.28

0 1 2 3 4 5 6-1

-0.5

0

0.5

1

1st order (linear) Fit :

>> x = [0 1 2 3 4 5 6];

>> y = [0 .84 .91 .14 -.76 -.96 -.28];

>> c1 = polyfit(x,y, 1);

>> st = sum ((y-mean(y)).^2);

>> sr = sum((y-polyval(c1,x)).^2);

>> r1 = sqrt((st-sr)/st)

r1 = 0.6528

>> y1 = polyval(c1,x);

>> plot(x,y,’o’,x,y1)

0 1 2 3 4 5 6-1

-0.5

0

0.5

1

3rd

0.9951

2nd

0.6695

Order

r

1st

0.6528

0 1 2 3 4 5 6-1

-0.5

0

0.5

1

Data1st2nd3rd

c3 = [0.0886 -0.8262 1.7262 -0.0362]

>> c2 = polyfit(x,y,2)

>> c3 = polyfit(x,y,3)

Exponential FitExponential Fit

0E+0

1E+5

2E+5

3E+5

4E+5

5E+5

6E+5

0 5 10 15 20Years

Pop

ulat

ion

��,�ก�)�� !��ก6)%�� exponential function

��ก��7��'��%��ก�"#��-(��

��$�� ก�8 �%��5��$��$� 15 �8

��ก��+��ก��,��ก��$��5�ก��-(��!#+�

��ก��+ ก��%��)!��

��ก6)%��+�#�$��!��ก


1

10

100

1000

10000

100000

1000000

0 10 20Years

Pop

ulat

ion

• 9��-�5�!��(�*��ก� y �� log ��ก�

• �%��5��$�ก��-(��%��ก��%��ก��$��

• ��+�9��!��ก$��ก5�%��9��%��)��,, least-square ��(�


��%��)� �!�� (xi, yi) *��7.ก2��ก"2*��

ixbi eay =

��$*��$7.ก2��กก��(�#�:��1��(��+��!��ก��

ii xbalnyln +=

*��$� ln yi !��$�%!�� 9�ก��$� a ��% b ��*��(1� least square

β+α= ii xz

�� ,ylnz ii = α = b, ��% β = ln a �� a = eβ

Example: Exponential Fit

xy

0266

1162

298

370

439

521

612

0 1 2 3 4 5 60

50

100

150

200

250

300

x i

0123456

y i

2661629870392112

zi = ln y i

5.5845.0884.5854.2493.6643.0452.485

x i2

0149

162536

x izi

05.0889.17012.74614.65415.22314.909

zi2

31.17525.88421.02218.05013.4229.2696.175

ΣΣΣΣ 21 28.698 91 71.789 124.996

Least Square fit of x i and z i:

n = 7

3721

x ==

1.47698.28

z ==

2i

2i

iiii

)x(n1

x

zxn1

zx

Σ−Σ

ΣΣ−Σ=α 261.0

7/2191

7/698.2821789.782 −=

−

×−=

xz α−=β = 4.1 – (-0.261)(3) = 4.883

0 1 2 3 4 5 60

50

100

150

200

250

300

ixbi eay =

β+α= ii xz

��$��

α = b, ��% a = eβ

= -0.261 xi + 4.883

�� = e4.883 = 132.026

ix261.0i e026.132y −=

>> xi=0:0.1:6;

>> yi=132.026*exp(-0.261*xi);

>> plot(x,y,'o',xi,yi)

Least Square fit of x i and z i: (by MATLAB)

>> clear

>> x = 0:6;

>> y = [266 162 98 70 39 21 12];

>> z = log(y);

>> c = polyfit(x,z,1);

>> b = c(1);

>> a = exp(c(2));

>> yhat = a*exp(b*x);

>> st = sum ((y-mean(y)).^2);

>> sr = sum((y-yhat).^2);

>> r = sqrt((st-sr)/st)

r = 0.9968

>> xi = 0:0.1:6;

>> yi = a*exp(b*xi);

>> plot(x,y,'o',xi,yi)

0 1 2 3 4 5 60

50

100

150

200

250

300

Lecture Lecture 88 InterpolationInterpolation

LinearLinear InterpolationInterpolation

QuadraticQuadratic InterpolationInterpolation

PolynomialPolynomial InterpolationInterpolation

PiecewisePiecewise Polynomial InterpolationPolynomial Interpolation

Numerical MethodsNumerical Methods for for CivilCivil EngineersEngineers




Visual InterpolationVisual Interpolation

Vehicle speed is approximately 49 km/h

0

20

4060

80

100

120

km/h

Interpolation between data points

known datay

x

Consider a set of xy data collected during an experiment.

We use interpolation technique to estimate y at x where there’s no data.

What is the corresponding value of y for this x ?

BASIC IDEASBASIC IDEAS

From the known data ( xi , yi ), interpolate

Determining coefficient a1, a2, . . . , an of basis function F(x)

F(x) = a1ΦΦΦΦ1(x) + a2ΦΦΦΦ2(x) + . . . + anΦΦΦΦn(x)

Polynomials are often used as the basis functions.

F(x) = a1 + a2 x + a3 x2 + . . . + an xn-1

ixxxFy ≠= ˆ for )ˆ(ˆ

nn

ii

ii

yx

yx

yx

yx

⋮⋮

⋮⋮

11

11

++

interpolateyx ˆˆ →

y

xx

y

F(x)

Interpolation vInterpolation v ..ss.. Curve FittingCurve Fitting

known datay

x

interpolationcurve fit

Curve fitting: fit function & data not exactly agree

Interpolation: function passes exactly through known data

Interpolation Interpolation && ExtrapolationExtrapolation

Interpolation approximate within the range of independent variable

of the given data set.

Extrapolation approximate outside the range of independent variable

of the given data set.

xx1 x2

y

Linear Interpolation

y

x

The most common way to estimate a data point between 2 points.

The function is estimated by a straight line drawn between them.

Interpolated Point

Linear, Quadratic, and Cubic Interpolations

Linear = 1st order

f(x) = a0 + a1x

2 points

Quadratic = 2nd order

f(x) = a0 + a1x + a2x2

3 points

Cubic = 3rd order

f(x) = a0 + a1x+ a2x2 + a3x3

4 points

General formula for an (General formula for an ( nn –– 1)th1)th --order polynomial:order polynomial:

f(x) = a0 + a1x+ a2x2 +…+ anxn-1

Linear InterpolationLinear Interpolation

Connect two data points with a straight line

x0 x1

y0

y1

x

yxF ˆ)ˆ( =

Using similar triangles:

01

01

0

0

ˆˆ

xxyy

xxyy

−

−=

−

−

)ˆ(ˆ)ˆ( 001

010 xx

xxyy

yyxF −−

−+==

QuadraticQuadratic InterpolationInterpolation

Second-order polynomial interpolation using 3 data points

(x0, y0)

(x1, y1)

(x2, y2)

Convenient form:

))(()()( 1020102 xxxxbxxbbxf −−+−+=

22102 )( xaxaaxf ++=

where 1020100 xxbxbba ++=

120211 xbxbba ++=

22 ba =00002 )( Substitute ybyxf =→=

01

011112 )( Substitute

xxyy

byxf−

−=→=

02

01

01

12

12

2222 xxxxyy

xxyy

b y)x(f Substitute−

−

−−

−

−

=→=

EXAMPLE : Linear & Quadratic Interpolation

Estimate ln (2) by using linear & quadratic interpolation

ln(1) = 0ln(2) = 0.6932ln(4) = 1.3863ln(6) = 1.7918

Linear interpolation from x1 = 1 to x2 = 6

3584.0)12(16

07918.10)2(1 =−

−

−+=f %3.48 =→ tε

Linear interpolation from x1 = 1 to x2 = 4

4621.0)12(14

03863.10)2(1 =−

−

−+=f %3.33 =→ tε

Quadratic interpolation from x1 = 1 to x2 = 4 and x3 = 6

05187.016

4621.046

3863.17918.1

b

4621.014

03863.1b ,0b

2

10

−=−

−−

−

=

=−

−==

Substitute b0, b1 and b2 into equation

f2(x) = 0 + 0.4621(x - 1) - 0.05187(x - 1)(x - 4)

which can be evaluated at x = 2 for

f2(2) = 0 + 0.4621(2 - 1) - 0.05187(2 - 1)(2 - 4)

f2(2) = 0.5658 → εt = 18.4%

f2(x) = – 0.05187x2 + 0.7251x – 0.6696

1 2 3 4 5 6 7

1

2

>> x = 1:0.01:7;

>> y = log(x);

>> y2= -0.05187*x.^2+ ...

0.7251*x-0.6696;

>> plot(x,y,x,y2)

MATLAB :

PolynomialPolynomial InterpolationInterpolation

In general, n+1 data points can be fitted by an nth-order

polynomial of the form

)xx()xx)(xx(b)xx(bb)x(f n10n010n −−−++−+= ⋯⋯

The following equations are used to evaluate the coefficients:

)x(fb 00 =

01

011 xx

)x(f)x(fb

−

−=

02

01

01

12

12

2 xxxx

)x(f)x(fxx

)x(f)x(f

b−

−

−−

−

−

=

]x[f 0=

]x,x[f 01=

]x,x,x[f 012=

⋮

]x,x,,x[fb 01nn …=

Finite Divided DifferenceFinite Divided Difference

The bracketed function evaluations are finite divided differences and are defined as

01

0101 xx

)x(f)x(f]x,x[f

−

−=

02

0112012 xx

]x,x[f]x,x[f]x,x,x[f

−

−=

⋮

0n

011n12n01n xx

]x,x,,x[f]x,x,,x[f]x,x,,x[f

−

−= − ……

…

1st finite divided difference

2nd finite divided difference

nth finite divided difference

These equations are recursive, i.e. higher order differences

are computed by taking differences of lower-order differences.

Divided Difference TableDivided Difference Table

The computations are organized in the divided-

difference table:

x i

x0

x1

x2

x3

f(x i)

f(x0)

f(x1)

f(x2)

f(x3)

First Second Third

f [x1,x0] f [x2,x1,x0] f [x3,x2,x1,x0]

f [x3,x2]

f [x2,x1] f [x3,x2,x1]

]x,x,,x[f)xx()xx)(xx(

]x,x,x[f)xx)(xx(]x,x[f)xx()x(f)x(f

01nn10

012100100n

…⋯⋯ −−−++

−−+−+=

Newton’s divided difference interpolating polynomial:

EXAMPLE : 3rd Order Polynomial Interpolation

Use the polynomial interpolation to estimate ln (2)

ln(1) = 0ln(2) = 0.6932ln(4) = 1.3863ln(5) = 1.6094ln(6) = 1.7918

x i

1

4

5

6

f(x i)

0

1.3863

1.6094

1.7918

First Second Third

0.4621 -0.0797 0.0133

0.1834

0.2231 -0.0397

f3(x) = 0 + 0.4621(x – 1) – 0.0797(x – 1)(x – 4) + 0.0133(x – 1)(x – 4)(x – 5)

For x = 2, f3(2) = 0.7013 εεεεt = 1.17%


� Although the finite divided difference is well suited for

determining intermediate values between points, they

do not provide a polynomial in conventional form:

nx xaxaxaaxf ++++= ⋯

2210)(

� Since n+1 data points are required to determine n+1

coefficients, simultaneous linear systems of equations

can be used to calculate “a”s.

nnnnnn

nn

nn

xaxaxaaxf

xaxaxaaxf

xaxaxaaxf

+++=

+++=

+++=

⋯

⋮

⋯

⋯

2210

12121101

02020100

)(

)(

)(

Where “x”s are the knowns and “a”s are the unknowns.

VandermondeVandermonde Matrix EquationMatrix Equation

=

)x(f

)x(f

)x(f

a

a

a

xxx1

xxx1

xxx1

n

1

0

n

1

0

nn

2nn

n2

222

n1

211

⋮⋮

⋯

⋮⋮⋮⋮

⋯

⋯

Rewritten the equations in the matrix form:


Finding Pn-1(x) of degree n-1 that passes through n known data pairs

Pn-1(x) = c1xn-1 + c2xn-2 + . . . + cn-1x + cn

Vandermonde Systems n pairs of (x, y)n equationsn unknowns

Polynomial pass through each of data points

Linear = 1 : 2 pt.Quadratic = 2 : 3 pt.Cubic = 3 : 4 pt.

Example: Construct a quadratic interpolating function

y = c1x2 + c2x + c3

that pass through (x, y) support points (-2, -2), (-1, 1), and (2, -1)

Substitute known points into equation:

-2 = c1 (-2)2 + c2 (-2) + c3

1 = c1 (-1)2 + c2 (-1) + c3

-1 = c1 (2)2 + c2 (2) + c3

Rewritten in matrix form:

1

2

3

4 2 1 2

1 1 1 1

2 2 1 1

c

c

c

− − − = −

VandermondeVandermonde MatrixMatrix

21 1 1 122 2 2 223 3 3 3

1

1

1

x x c y

x x c y

x x c y

=

>> x = [-2 -1 2]’;

>> A = [x.^2 x ones(size(x))];

or use the built-in vander function

>> A = vander([-2 -1 2]);

>> y = [-2 1 -1]’;

>> c = A\y

c =

-0.9167

0.2500

2.1667

MATLAB : MATLAB : polyfitpolyfit and and polyvalpolyval FunctionsFunctions

Find y = c1x2 + c2x + c3 that pass through (x, y) support points

(-2, -2), (-1, 1), and (2, -1).

>> x = [-2 -1 2];>> y = [-2 1 -1];>> c = polyfit(x,y,2)c =

-0.9167 0.2500 2.1667

We can then use the polyval function to perform an interpolation as in

Ex. To interpolate y at x = 1

>> polyval(c,1)

ans =

1.5000

Also, we can use the polyval function to plot the result as in

>> xhat = -3:0.1:3;

>> yhat = polyval(c,xhat);

>> plot(xhat,yhat,x,y,'o')

-3 -2 -1 0 1 2 3-8

-6

-4

-2

0

2

4

Polynomials WigglePolynomials Wiggle

2nd-order

3rd-order

4th-order

5th-order

y

x

xi 1 2 3 4 5 6 7 8 9 10

yi 3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0

MATLAB’s Command Lines to Demonstrate Polynomials Wi ggle

>> x = [1 2 3 4 5 6 7 8 9 10];

>> y = [3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0];

>> x0 = 1:0.1:10;

>> y2 = polyval(polyfit(x(4:6), y(4:6), 2), x0);




>> axis([0 10 -5 5])

>> plot(x, y, ‘o’)

>> hold on

>> plot(x0, y2)

>> plot(x0, y3)

>> plot(x0, y4)

>> plot(x0, y5)

Piecewise Polynomial Interpolation

Using a set of lower degree interpolants on subinterval of the whole domain

= breakpoint or knoty

x

Piecewise-linear interpolation

Piecewise-quadratic interpolation

Piecewise-cubic interpolation

f’(x) and f’’(x) continuous at breakpoint = cubic spline

MATLAB’s Built-in Interpolation Functions

Funnction Description

interp1 1-D interpolation with piecewise polynomials.

interp2 2-D interpolation with nearest neighbor, bilinear,or bicubic interpolants.

interp3 3-D interpolation with nearest neighbor, bilinear,

or bicubic interpolants.

interpft 1-D interpolation of uniformly spaced data using

Fourier Series (FFT).

interpn n-D extension of methods used by interp3.

spline 1-D interpolation with cubic-splines using

not-a-knot or fixed-slope end conditions.

interp1 Built-in Function

1-D interpolation with one of the following 4 methods:

1. Nearest-neighbor uses piecewise-constant function.

Interpolant discontinue at midpoint between knots

2. Linear interpolation uses piecewise-linear polynomials.

3. Cubic interpolation uses piecewise-cubic polynomials.

Interpolant and f’ (x) are continuous.

4. Spline interpolation uses cubic splines. This option performs

the same interpolation as built-in function spline .

How to use How to use interpinterp 11 ??

>> yhat = interp1(y, xhat)

>> yhat = interp1(x, y, xhat)

>> yhat = interp1(x, y, xhat, method)

where y = tabulated values to be interpolated.

x = independent values. If not given x=1:length(y).

xhat = values at which interpolant be evaluated.

method = ‘nearest’ , ‘linear’ , ‘cubic’ or ‘spline’

>> x = [1 2 3 4 5 6 7 8 9 10];

>> y = [3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0];

>> xhat = 1:0.1:10; % eval interpolant at xhat

>> yn = interp1(x, y, xhat, ‘nearest’);

>> plot(x, y, ‘o’, xhat, yn); pause;

>> yl = interp1(x, y, xhat, ‘linear’);

>> plot(x, y, ‘o’, xhat, yl); pause;

>> yc = interp1(x, y, xhat, ‘cubic’);

>> plot(x, y, ‘o’, xhat, yc); pause;

>> ys = interp1(x, y, xhat, ‘spline’);

or >> ys = spline(x, y, xhat);

>> plot(x, y, ‘o’, xhat, ys);

Example: Interpolation with piecewise-polynomials

EXAMPLE : Linear interpolation by interp1 function

Estimate the value of y when x is equal to 3.5

x

y

0

15

1

10

2

9

3

6

4

2

5

0

>> x = 0:5;

>> y = [15 10 9 6 2 0];

>> interp1(x,y,3.5)

ans =

4

Plot Graph:

>> xhat = 0:0.2:5;

>> yhat = interp1(x,y,xhat);

>> plot(x,y,'bo-',xhat,yhat,'rx')

0 1 2 3 4 50

5

10

15

-- Basic IdeasBasic Ideas

-- Symbolic Symbolic vsvs.. Numerical IntegrationNumerical Integration

-- Trapezoid RuleTrapezoid Rule

-- SimpsonSimpson’’s Rules Rule

-- MATLAB quad and quadMATLAB quad and quad88 FunctionsFunctions


Mongkol JIRAVACHARADETS U R A N A R E E INSTITUTE OF ENGINEERINGUNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING

Lecture Lecture 99 Numerical IntegrationNumerical Integration

BASICBASIC IDEASIDEAS

)( curveunder Area)( xfdxxfIb

a

== ∫

Approximated by piecewise-linear:

Area = Sum of trapezoidal regions

f (x)

a b

Trapezoidalregion

Node

x

Symbolic Integration with MATLAB

>> I = int(f) % Indefinite integral

>> I = int(f, v) % Designating integration variable

>> I = int(f, a, b) % Definite integral on close interval

>> I = int(f, v, a, b)where f = symbolic expression

Variables must be define as symbolic by sym or syms>> x = sym(‘x’), y = sym(‘y’), z = sym(‘z’)or >> syms x y z

Symbolic Math ToolboxSymbolic Math ToolboxSymbolic Math Toolboxes incorporate symbolic computation into the numericenvironment of MATLAB.

dxcxIb

a∫ −= )( :Example 3

>> syms x a b c>> I = int(x^3-c,x,a,b)

I =

1/4*b^4-c*b-1/4*a^4+c*a

What is Integration?What is Integration?

dxxfIb

a∫= )(

The integral is equivalent to the area under the curve.f(x)

xa b

I

Cross-sectional area of a river

Examples of how integration is used to evaluate areasExamples of how integration is used to evaluate areas

Net force due to wind blowing against the building

NewtonNewton--Cotes FormulasCotes FormulasReplace a complicated function with a polynomial that is easy to integrate

dxxfIb

a∫= )( dxxf

b

an∫≅ )(

nn

nnn xaxaxaaxf 1

110)( where +

− ++++= L

f(x)

xa b

Approximate area by a straight line

f(x)

xa b

Approximate area by a parabola

Trapezoidal RuleTrapezoidal Rule

xba

f(a)

f(b)

f(x)

f1(x)

1st degree polynomial = Linear line

)()()()()(1 axab

afbfafxf −−−

+=

2)()()()(1

bfafabdxxfIb

a

+−== ∫

Area of trapezoid = width x average height

width = b – a = h

average height = 2

)()( bfaf +

f1

f2

x1 x2

f(x)

x3 x4 x5

f3f4 f5

∫ =5

1

)(x

x

dxxf ∫ +2

1

)(x

x

dxxf ∫ +3

2

)(x

x

dxxf ∫ +4

3

)(x

x

dxxf ∫5

4

)(x

x

dxxf

Composite Trapezoidal RuleComposite Trapezoidal RuleImprove accuracy by dividing interval into subintervals

f(x)

x1 = a

x1 x2 x3 x4 x5 x6

xn = bh =

b – an

There are n segment with equal width:

h =b – a

n

dxxfdxxfdxxfIn

n

x

x

x

x

x

x∫∫∫

−

+++=1

3

2

2

1

)()()( L

2)()(

2)()(

2)()(

1

3221

nn xfxfh

xfxfhxfxfhI

++

++

++

=

−

L

22213221 nn ffhffhffhI +

+++

++

= −L

++= ∑

−

=n

n

ii fffhI

1

21 2

2

Substitute h =b – a

n

MEAN

f(x)

x1 x2 x3 x4 x5 x6

f1

f2 f3

f4 f5

f6

width average height

n

fffabI

n

n

ii

2

2)(

1

21

++

−=∑

−

=

Example 9-1 Use Trapezoidal rule to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=

from a = 0 to b = 0.8. Note that the exact value of the integral can be determinedanalytically to be 1.640533.

or using MATLAB’s Symbolic Math Toolbox :

>> x = sym(‘x’)>> I=int(0.2+25*x-200*x^2+675*x^3-900*x^4+400*x^5,x,0,0.8)

I = 3076/1875 = 1.6405

Single Trapeziod: f(0) = 0.2 and f(0.8) = 0.232

1728.02

232.02.0)08.0(

2)()()(

=+

−=

+−=

bfafabI

%5.89%1006405.1

1728.06405.1=×

−=tε

f(x)

0 0.8

Error

Integral estimate

2 Trapeziods: n = 2 (h = 0.4) :

f(0) = 0.2 f(0.4) = 2.456 f(0.8) = 0.232

n

fffabI

n

n

ii

2

2)(

1

21

++

−=∑

−

=

0688.14

232.0)456.2(22.0)8.0( =++

=I

%9.34%1006405.1

0688.16405.1=×

−=tε

Trapezoidal numerical integration

>> z = trapz(y) % Integral of y with unit spacing>> z = trapz(x, y) % Integral of y with respect to x

o

o

90

0

Example: sin x dx∫>> angle = 0:15:90;>> x = (pi*angle/180);>> y = sin(x);>> z = trapz(x,y)>> z =

0.9943

MATLABMATLAB’’ss trapztrapz functionfunction

Example 9-2 Use MATLAB’s trapz function to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=


For n = 2 : >> x = linspace(0,0.8,3);>> fx = 0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;>> I = trapz(x,fx)I =1.0688

n

23456

I

1.06881.36391.48481.53991.5703

εt

34.916.59.496.134.28

Using higher-order polynomials to connect the points

f(x)

xSimpson’s 1/3 ruleQuadratic connecting 3 points

f(x)

xSimpson’s 3/8 ruleCubic connecting 4 points

SimpsonSimpson’’s Ruless Rules

f(x)

x0

Quadratic connecting 3 points

x2x1

f(x0)f(x2)

f(x1) f2(x)2 2

0 0

2( ) ( )x x

x x

I f x dx f x dx= ≅∫ ∫

1 22 0

0 1 0 2

0 21

1 0 1 2

0 12

2 0 2 1

( )( )( ) ( )( )( )( )( ) ( )

( )( )( )( ) ( )

( )( )

x x x xf x f xx x x xx x x x f x

x x x xx x x x f x

x x x x

− −=

− −− −

+− −

− −+

− −

[ ] 2 00 1 2( ) 4 ( ) ( ) ,

3 2x xhI f x f x f x h −

≅ + + =

SimpsonSimpson’’s 1/3 Ruless 1/3 Rules

Example 9-3 Use Simpson’s 1/3 rule to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=


Solution: n = 2 (h = 0.4) :

f(0) = 0.2 f(0.4) = 2.456 f(0.8) = 0.232

3675.1)232.0)456.2(42.0(34.0

=++=I

%6.16%1006405.1

3675.16405.1=×

−=tε

Composite SimpsonComposite Simpson’’s 1/3 Ruless 1/3 Rules

number even, where)()()(2

4

2

2

0

=−

=+++= ∫∫∫−

nn

abhdxxfdxxfdxxfIn

n

x

x

x

x

x

x

L

[ ] [ ] [ ]nnn fffhfffhfffhI +++++++++= −− 12432210 43

43

43

L

+++

−= ∑ ∑

−

=

−

=

)()(2)(4)(3

)( 1

5,3,1

2

6,4,20 n

n

i

n

jji xfxfxfxf

nabI

14 2 4

24

2 4 2 41

1 4 1

1 4 1

1 4 1

4

1 4 1

1 1

a b

Example 9-4 Use composite Simpson’s 1/3 rule with n = 4 to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=


Solution: n = 4 (h = 0.2) :

f(0) = 0.2 f(0.2) = 1.288 f(0.4) = 2.456 f(0.6) = 3.464 f(0.8) = 0.232

624.1)232.0)456.2(2)464.3288.1(42.0()4(3

8.0=++++=I

%04.1%1006405.1

624.16405.1=×

−=tε

+++

−= ∑ ∑

−

=

−

=

)()(2)(4)(3

)( 1

5,3,1

2

6,4,20 n

n

i

n

jji xfxfxfxf

nabI

2( ) ( )b b

a a

I f x dx f x dx= ≅∫ ∫f(x)

x

Cubic connecting 4 points

[ ]

( )0 1 2 3

3 0

3 ( ) 3 ( ) 3 ( ) ( )8

/ 3

hI f x f x f x f x

h x x

≅ + + +

= −

x0 x1 x2 x3a b

SimpsonSimpson’’s 3/8 Ruless 3/8 Rules

[ ])()(3)(3)(8

)(3210 xfxfxfxfabI +++

−=or…

31 ( )12

h f ξ′′

5 (4)1 ( )90

h f ξ

5 (4)3 ( )80

h f ξ

Trapezoid

Simpson’s 1/3

Simpson’s 3/8

Simpson’s 3/8 rule is used when number of segments is odd.

h b a= −

2b ah −

=

3b ah −

=

Method Et Et

Intervalwidth

3( ) ( )12

b a f ξ− ′′

5(4)( ) ( )

2880b a f ξ

−

5(4)( ) ( )

6480b a f ξ

−

Truncation ErrorsTruncation Errors

Apply Simpson’s 1/3 and 3/8 RulesTo handle multiple application with odd number of intervals

f(x)

x

1/3 rule 3/8 rule

Example 9-5 Use Simpson’s 1/3 + 3/8 rule with n = 5 to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=

from a = 0 to b = 0.8. Note that the exact value of the integral can be determined analytically to be 1.640533.

f(x)

x

1/3 rule 3/8 rule

0 0.16 0.32 0.48 0.64 0.8

Solution: n = 5 (h = 0.16) :

f(0) = 0.2 f(0.16) = 1.297 f(0.32) = 1.743 f(0.48) = 3.186f(0.64) = 3.182 f(0.8) = 0.232

%274.0%1006405.1

645.16405.1=×

−=tε

For the first two segments use Simpson’s 1/3 :

380.0)743.1)297.1(42.0(316.0

=++=I

For the last three segments use Simpson’s 3/8 :

265.1

)232.0)182.3186.3(3743.1(848.0

=

+++=I

Total integral : I = 0.380 + 1.265 = 1.645

>> q = quad(’f',a,b)Approximates the integral of f(x) from a to b within a relativeerror of 1e-3 using an adaptive recursive Simpson's rule.

quad : low order method, quad8: high order method

’f' is a string containing the name of the function.

>> quad(‘sin’, 0, pi/2)

Function f must return a vector of output values if given a vector of input values.

MATLABMATLAB’’ss quadquad and and quad8quad8 FunctionsFunctions

0.82 3 4 5

0

Example: 0.2 25 200 675 900 400x x x x x dx+ − + − +∫

function p = poly5(x)p=0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;

poly5.m

>> quad(‘poly5’, 0, 0.8)

ans =

1.6405

-1-0.5

00.5

1

-1

-0.5

0

0.5

10

2

4

6

8

10

Example: Computing the Length of a Curve

[ ]

( ) sin(2 ), ( ) cos( ), ( )

where 0, 3

x t t y t t z t t

t π

= = =

∈

>> t = 0:0.1:3*pi;>> plot3(sin(2*t), cos(t), t)

Length of the curve:Norm of derivative

32 2

0

4cos(2 ) sin( ) 1t t dtπ

+ +∫

hcurve.m

function f = hcurve(t)f = sqrt(4*cos(2*t).^2 + sin(t).^2 + 1);

>> len = quad(‘hcurve’,0,3*pi)

len =17.2220

Example: Double Integrationmax max

min min

( , )y x

y x

f x y dx dy∫ ∫

>> xmin = pi; xmax = 2*pi;>> ymin = 0; ymax = pi;>> result = dblquad(‘integrnd’,xmin,xmax,ymin,ymax);result =

-9.8698

For example: f (x, y) = y sin(x) + x cos(y)integrnd.m

function out = integrnd(x,y)out = y*sin(x) + x*cos(y);

- Basic Ideas

- Euler’s Method

- Higher Order One-step Methods

- Predictor-Corrector Approach

- Runge-Kutta Methods

- Adaptive Stepsize Algorithms


Lecture Lecture 1010 Ordinary Differential EquationsOrdinary Differential Equations




Definitions and TerminologyDefinitions and Terminology

Differential equation (DE) : An equation contains the derivates

of one or more dependent variables with respect to one or more independent variables.

yx2.0dxdy

=Example:

Ordinary DE (ODE): An eq. contains only ordinary derivates

of one or more dependent variables with respect to a single independent variable.

Example: ,ey5dxdy x=+ yx2

dtdy

dtdx

+=+

( , )dy

f t ydt

=

where t = independent variable

y = dependent variable

Initial Condition y(t0) = y0

Rate of change of one variable with respect to another.

2

0d x dx

ma cv kx m c kxdt dt

+ + = + + =

Example: Mass-spring system m

k

c

x

Ordinary Differential EquationsOrdinary Differential Equations

Leonhard Euler 1707 - 1783

Leonhard Euler made a huge number of contributions to mathematics, almost half after he was totally blind.

Leohard Euler

It was Euler who originated the following notations:

e (base of natural log)

( )f x (function notation)

π (pi) i ( )1−

(summation)∑

New value = Old value + slope x step size

From Taylor series:2

00 0 0 0

( )( ) ( ) ( ) ( ) ( )

2!

t ty t y t t t y t y t

−′ ′′= + − + +⋯

Retaining only first derivative: 0 0 0( ) ( , )y t y h f t y= +

where h = (t - t0), y0 = y(t0), and f(t0, y0) = y’(t0)

t

y

t0 t1

h

Predict

Trueerror

1 0 0 0( , )y y h f t y= +

2 1 1 1( , )y y h f t y= +

1 1 1( , )i i i iy y h f t y− − −= +

EULEREULER’’S METHODS METHOD

Example: Manual Calculation with Euler’s Method

2 , (0) 1dy

t y ydt

= − = Exact solution:

( )212 1 5

4ty t e−= − +

Set step size: h = 0.2

f (ti-1, yi-1)

NA

0-2(1) = -2.0

0.2-2(0.6) = -1.0

0.4-2(0.4) = -0.4

ti

0.0

0.2

0.4

0.6

Euleryi = yi-1+ h f (ti-1, yi-1)

initial cond. = 1.0

1.0+0.2(-2.0) = 0.6

0.6+0.2(-1.0) = 0.4

0.4+0.2(-0.4) = 0.32

Exact

1.0000

0.6879

0.5117

0.4265

Error

0

-0.0879

-0.1117

-0.1065

f (t, y) = t – 2y

Implementing Euler’s Method

function [t,y] = odeEuler(diffeq,tn,h,y0)% Input: diffeq = (string) name of m-file that% evaluate right hand side of ODE% tn = stopping value of independent variable% h = stepsize% y0 = initial condition at t=0

t = (0:h:tn)’;n = length(t);y = y0*ones(n,1);

for i=2:ny(i) = y(i-1)+h*feval(diffeq,t(i-1),y(i-1));

end

odeEuler.m

Example: 2 , (0) 1, 0.2dy

t y y hdt

= − = =

function dydt = rhs1(t,y)dydt = t-2*y;

rhs1.m

>> [t,y] = odeEuler(‘rhs1’,0.6,0.2,1.0)

t =0

0.20000.40000.6000

y =1.00000.60000.40000.3200

>> y0 = (1/4)*(2*t-ones(size(t))+5*exp(-2*t))

y0 =

1.00000.68790.51170.4265

Exact Solution :

>> t0=0:0.01:0.6;>> y0 = (1/4)*(2*t0-ones(size(t0))+5*exp(-2*t0));>> plot(t0,y0,t,y)

HeunHeun ’’ss MethodMethod :: estimate slope from derivatives at the beginning and at the end of the interval.

Improvements of Improvements of EEuleruler’’ss MMethodethod

t

y

ti-1

slope:k1 = f(ti-1, yi-1)

ti

Euler:y0

i = yi-1 + hk1

h

slope:k2 = f(ti, y0

i)

t

y

ti-1 ti

h

2),(),(

2 slope Average

01121 iiii ytfytfkk +

=+

= −−

++= − 2

211

kkhyy ii

Use slope:(k1 + k2)/2

1 1 1

2 1 1 1

1 21

( , )

( , )

2

i i

i i

i i

k f t y

k f t h y hk

k ky y h

− −

− −

−

=

= + +

+= +

yi-1

ti-1

h

ti

True

Euler

Heun

HeunHeun’’ss MMethodethod

( , )dy

f t ydt

= 0iy

Heun’s Method:

1 1 1

2 1 1 1

1 21

( , )

( , )

2

i i

i i

i i

k f t y

k f t h y hk

k ky y h

− −

− −

−

=

= + +

+= +

1 1 1( , )i i i iy y h f t y− − −= +Euler’s Method:

01 1 1Predictor: ( , )i i i iy y h f t y− − −= +

01 1

1

( , ) ( , )Corrector:

2i i i i

i i

f t y f t yy y h − −

−

+= +

PredictorPredictor--Corrector ApproachCorrector Approach

Iterating the Corrector of Iterating the Corrector of HeunHeun’’ss MethodMethod

To improve the estimation

01 1

1

( , ) ( , )

2i i i i

i

f t y f t yy h − −−

++

iy

Example: Use Heun’s method to integrate

0.84 0.5 , (0) 2xy e y y′ = − =

From x = 0 to 4, step size = 1

Analytical solution: ( )0.8 0.5 0.542

1.3x x xy e e e− −= − +

01 1 1

0 01

Predictor, ( , )

2 1 (4 0.5(2)) 5

i i i iy y h f t y

y e

− − −= +

= + − =

01 1

1

0 0.8(1)

1

( , ) ( , )Corrector,

2(4 0.5(2)) (4 0.5(5))

2 1 6.70112

i i i ii i

f t y f t yy y h

e ey

− −−

+= +

− + −= + =

True value: ( )0.8(1) 0.5(1) 0.5(1)42 6.1946

1.3y e e e− −= − + =

6.1946 6.7011Relative error, 100% 8.18%

6.1946tE−

= × =

0 0.8(1)nd

1

(4 0.5(2)) (4 0.5(6.7011))2 Corrector, 2 1

26.2758, 1.31%t

e ey

E

− + −= +

= =

0 0.8(1)rd

1

(4 0.5(2)) (4 0.5(6.2758))3 Corrector, 2 1

26.3821, 3.03%t

e ey

E

− + −= +

= =

Error may increases for large step sizes.

Iteration using up-arrow in MATLAB

>> y = 2+1*(4*exp(0)-0.5*2) % Predictor of y1

y = 5

>> y = 2+(1/2)*((4*exp(0)-0.5*2)+(4*exp(0.8*1)-0.5*y) )

y = 6.7011 % 1st Corrector of y1

EnterPress and

y = 6.7011 % 2nd Corrector of y1

y = 6.2758 % 3rd Corrector of y1

y = 6.3821 % 4th Corrector of y1

y = 6.3555 % 5th Corrector of y1

y = 6.3609 % until no change

MATLAB’s Implementation

function dydx = func1(x,y)dydx = 4*exp(0.8*x)-0.5*y;

func1.m

function [x,y] = odeHeun(diffeq,xn,h,y0,iter)% Input: iter = number of corrector iterations

x = (0:h:xn)’;n = length(x);y = y0*ones(n,1);for i=2:n

y(i) = y(i-1)+h*feval(diffeq,x(i-1),y(i-1));for j=1:iter

y(i) = y(i-1)+h*(feval(diffeq,x(i-1),y(i-1)) ...

+ feval(diffeq,x(i),y(i)))/2;end

end

odeHeun.m

Et =0.002.683.093.173.18

y_true =2.00006.194614.843933.677275.3390

For x = 0 to 4, step size = 1,y(0) = 2, Iteration = 15

>> [x,y] = odeHeun(‘func1’,4,1,2,15)

x =01234

y =2.00006.360915.302234.743377.7351

Compare with Euler’s Method:

>> [x,y] = odeEuler(‘func1’,4,1,2)

x =01234

y =2.00005.000011.402225.513256.8493

Et =0.0019.2823.1924.2424.54

Comparison of True Solution with Euler’s and Heun’s Method

>> xtrue = (0:0.1:4)’;>> ytrue = (4/1.3)*(exp(0.8*xtrue)-exp(-0.5*xtrue)).. .

+ 2*exp(-0.5*xtrue);>> [xeuler,yeuler] = odeEuler(‘func1’,4,1,2);>> [xheun,yheun] = odeHeun(‘func1’,4,1,2,15);>> plot(xtrue,ytrue,xeuler,yeuler,’o’,xheun,yheun,’ +’)

0 1 2 3 40

20

40

60

80

x

y

True Euler Heun 15

11

m

i i l ll

y y kγ−=

= +∑General Formula using Weighted Average of slopes

For Heun’s method γ1 = γ2 = 0.5

The weights must satisfy1

1m

ll

γ=

=∑

Second-Order RK (Ralston’s) Method:

1 1 2

1 1 1

2 1 1 1

1 2

3 3

( , )

3 3,

4 4

i i

i i

i i

y y h k k

k f x y

k f x h y k h

−

− −

− −

= + +

=

= + +

RRUNGEUNGE--KUTTA (RK)KUTTA (RK) METHODSMETHODS

Third-Order Runge-Kutta Methods

( )1 1 2 346i i

hy y k k k−= + + +

where

( )

1 1 1

2 1 1 1

3 1 1 1 2

( , )

1 1,

2 2

, 2

i i

i i

i i

k f x y

k f x h y k h

k f x h y k h k h

− −

− −

− −

=

= + +

= + − +

Fouth-Order Runge-Kutta Methods

where

( )

1 1 1

2 1 1 1

3 1 1 2

4 1 1 3

( , )

1 1,

2 2

1 1,

2 2

,

i i

i i

i i

i i

k f x y

k f x h y k h

k f x h y k h

k f x h y k h

− −

− −

− −

− −

=

= + +

= + +

= + +

The most popular RK methods

( )1 1 2 3 42 26i i

hy y k k k k−= + + + +Classical 4th-order RK:

Example: Use 4th-order RK method to integrate

0.84 0.5 , (0) 2xy e y y′ = − =

From x = 0 to 4, step size = 1

Step 1:x0 = 0, y0 = 2

01

0.8(0.5)2

0.8(0.5)3

0.8(1)4

4 0.5(2) 3

14 0.5(2 3 1) 4.2173

21

4 0.5(2 4.2173 1) 3.91302

4 0.5(2 3.9130 1) 5.9457

k e

k e

k e

k e

= − =

= − + × × =

= − + × × =

= − + × =

( )1

12 3 2 4.2173 2 3.9130 5.9457

66.2011

y = + + × + × +

=

True value: ( )0.8(1) 0.5(1) 0.5(1)42 6.1946

1.3y e e e− −= − + =

6.1946 6.2010Relative error, 100% 0.1038%

6.1946tE−

= × =

MATLAB’s Implementation

function [x,y] = odeRK4(diffeq,xn,h,y0)

x = (0:h:xn)’;n = length(x);y = y0*ones(n,1);for i=2:n

k1 = feval(diffeq,x(i-1),y(i-1));k2 = feval(diffeq,x(i-1)+h/2,y(i-1)+k1*h/2);k3 = feval(diffeq,x(i-1)+h/2,y(i-1)+k2*h/2);k4 = feval(diffeq,x(i-1)+h,y(i-1)+k3*h);y(i) = y(i-1)+h/6*(k1+2*k2+2*k3+k4);

end

odeRK4.m

For x = 0 to 4, step size = 1, y(0) = 2

>> [x,y] = odeRK4(‘func1’,4,1,2)

x =01234

y =2.00006.201014.862533.721375.4392

y_true =2.00006.194614.843933.677275.3390

Et(%) =0.000.10380.12500.13090.1328

MATLAB’s ode23 and ode45 Routines

ode23 use second- and third-order RK simultaneously

ode45 use fourth- and fifth-order RK simultaneously

>> x0 = 0; xn = 4; y0 = 2;>> [x45,y45] = ode45(‘func1’,[x0,xn],y0);>> xtrue = (0:0.1:4)’;>> ytrue = (4/1.3)*(exp(0.8*xtrue)...

- exp(-0.5*xtrue))+ 2*exp(-0.5*xtrue);>> plot(xtrue,ytrue,x45,y45)

For x = 0 to 4,y(0) = 2

Example: 0.84 0.5 , (0) 2xy e y y′ = − = func1.m

Adaptive Adaptive StepsizeStepsize AlgorithmsAlgorithms

�� Basic Ideas

�� One-dimension- Golden-Section Search- Quadratic Interpolation- Newton’s Method

�� Multidimension- Direct Methods- Gradient Methods

NumericalNumerical Methods for Methods for CivilCivil EngineersEngineers




Lecture 11 Optimization

BASIC IDEAS

Root Root

Root

( ) 0f x =

Maximum

Minimum

( ) 0

( ) 0

f x

f x

′ =

′′ <

( ) 0

( ) 0

f x

f x

′ =

′′ >

0

f (x)

x

Maximum : Find x* such that f(x*) ≥≥≥≥ f(x) for all x

Minimum : Find x* such that f(x*) ≤≤≤≤ f(x) for all x

Examples of Optimization

• Minimize weight of structure subject to constraint on its

strength, or maximize its strength subject to constraint of

its weight

• Design vehicle for minimum energy consumption subject

to constraint on capacity and speed

• Design machine for maximum efficiency with less energy

consumption

• Optimal planning and scheduling of a construction project

• Budget or financial planning of any organization

0

f (x)

x

maximum

minimum

maximum

minimum

Local

LocalGlobal

Global

Global & Local Optimization

x* is global minimum if f(x*) ≤≤≤≤ f(x) for all x

x* is local minimum if f(x*) ≤≤≤≤ f(x) for all x near x*

Golden Section Search

� Similar to Bisection root finding method

� Define interval that contains one answer

� Need 3 points to detect maximum or minimum

� Pick 4th points then choose the first or last three points.

IntermediatePoint

x*xL xU

MAX

MIN

Continue the process to

narrow down the interval

Choice of Intermediate Points

0

f (x)

xx*

MAX

xL xU

L0

First iteration:L0 = L1 + L2

L2L1

- Define interval

- Pick 3 pts. from 4 pts.

Interval L0 is narrow down to L1

Choice of Intermediate Points

1 2

0 1

L L=

L LL2

Second iteration:

0

f (x)

xx*

MAX

xL

L0

First iteration:L0 = L1 + L2

L2L1

xU

- Move xU to new position

- Use same old 3 pts. sopick the new 4th pt.

From two condition, 1 2

1 2 1

L L

L L L=

+

Set R = L2 / L1,

211 or 1 0R R R

R+ = + − =

Solving for positive root,

1 1 4( 1) 5 10.61803

2 2R

− + − − −= = = …

R = 0.61803 = Golden ratio since it allows optima to be

found efficiently

The Parthenon in Athens

x

0.61803 x

This golden ratio were considered aesthetically pleasing

by the Greeks.

f (x1)f (x2)Eliminate

3) If f (x1) > f (x2),

Initial Step of the Golden-section Search

f (x)

x

Maximum

2) Choose two interior points x1 and x2

1) Guess initial bracket xL and xU

xL xUx1x2

dd

at the distance d

where )xx(2

15d LU −

−= )xx(618.0 LU −≈

x1 = xL + d

x2 = xU – d

eliminate [ xL, x2 ] and set x2 = xL for next step

Next Step to Complete the Algorithm

f (x)

x

Maximum

xUx2xL

Old x2 Old x1

4) Only new x1 need to be determined,

x1

new )xx(2

15d LU −

−=

x1 = xL + d

Example: Golden-Section Search to find maximum

2

( ) 2sin , 0 and 410 L U

xf x x x x= − = =

Solution: (1) Create two interior points

1

2

5 1(4 0) 2.472

20 2.472 2.472

4 2.472 1.528

d

x

x

−= − =

= + =

= − =

(2) Evaluate function at interior points,2

1

2

2.472( ) (2.472) 2sin(2.472) 0.63

10( ) (1.528) 1.765

f x f

f x f

= = − =

= =

xL x1 x2 xU

d d

(3) Because f(x2) > f(x1), eliminate upper part,

New xU = x1 = 2.472

New x1 = x2 = 1.528xL x2 x1 xU

OLD

xL x1 xU

NEW

x2

(4) Compute new x2

2

5 1(2.472 0) 1.528

22.472 1.528 0.944

d

x

−= − =

= − =

(5) Evaluate function at x2

1 2

2

New ( ) Old ( ) 1.765

New ( ) (0.944) 1.531

f x f x

f x f

= =

= =

Because f(x1) > f(x2), eliminate lower part, . . .

2 0 2.4721

4

5

6

7

8

0.9443

1.3050

1.3050

1.3050

1.3901

1.3050

1.5279

1.4427

1.3901

1.4427

1.7595

1.7647

1.7755

1.7742

1.7755

1.5279

1.6656

1.5279

1.4427

1.4752

1.7647

1.7136

1.7647

1.7755

1.7732

1.8885

1.8885

1.6656

1.5279

1.5279

0.5836

0.3607

0.2229

0.1378

0.0851

Tabulated results:

xL x2 f(x 2) x1 f(x 1) xU di

1.7647 0.6300

3 0.9443 2.4721

1 0 4.0000

1.5279 1.7647 1.8885 1.5432 0.9443

0.9443 1.5310 1.5279 1.7647 1.5279

1.5279 2.4721 2.4721

Quadratic Interpolation

2nd polynomial provides good approximation near optimum

xx0 x1 x3 x2

f (x) True maximum Quadraticapproximationof maximum

2 2 2 2 2 20 1 2 1 2 0 2 0 1

30 1 2 1 2 0 2 0 1

( )( ) ( )( ) ( )( )

2 ( )( ) 2 ( )( ) 2 ( )( )

f x x x f x x x f x x xx

f x x x f x x x f x x x

− + − + −=

− + − + −

Example: Quadratic Interpolation to find maximum

2

0 1 2( ) 2sin , 0, =1 and 410

xf x x x x x= − = =

Solution: (1) Evaluate function values,

0 0

1 1

2 2

0 ( ) 0

1 ( ) 1.5829

4 ( ) 3.1136

x f x

x f x

x f x

= =

= =

= = −

2 2 2 2 2 2

3

0(1 4 ) 1.5829(4 0 ) ( 3.1136)(0 1 )1.5055

2(0)(1 4) 2(1.5829)(4 0) 2( 3.1136)(0 1)x

− + − + − −= =

− + − + − −

(2) Because f(1.5055) = 1.7691, employ golden-section search,

eliminate lower guess x0 = 0,

0 0

1 1

2 2

1 ( ) 1.5829

1.5055 ( ) 1.7691

4 ( ) 3.1136

x f x

x f x

x f x

= =

= =

= = −

2 2 2 2 2 2

3

1.5829(1.5055 4 ) 1.7691(4 1 ) ( 3.1136)(1 1.5055 )

2(1.5829)(1.5055 4) 2(1.7691)(4 1) 2( 3.1136)(1 1.5055)

1.4903

x− + − + − −

=− + − + − −

=

After 5 iterations, x = 1.4276 gives maximum value of 1.7757

Newton’s Method

1

( )( ) 0

( )i

i ii

f xf x x x

f x+= → = −′

At optimal point *, ( *) ( *) 0x f x g x′ = =

Define new function, ( ) ( )g x f x′=

1 1

( ) ( )( ) 0

( ) ( )i i

i i i ii i

g x f xg x x x x x

g x f x+ +

′= → = − → = −

′ ′′

Example: Use Newton’s method to find maximum of

2

0( ) 2sin , Initial guess: 2.510

xf x x x= − =

Solution: (1) Evaluate 1st and 2nd derivatives of the function,

( ) 2cos51

( ) 2sin5

xf x x

f x x

′ = −

′′ = − −

(2) Substitute into Newton’s formula,

1

2cos / 5

2sin 1/ 5i i

i ii

x xx x

x+

−= −

− −

i x f(x)

01234

2.50.995081.469011.427641.42755

0.571941.578591.773851.775731.77573

(3) Substituting initial guess,

1

2cos 2.5 2.5 / 52.5 0.99508, (0.99508) 1.57859

2sin 2.5 1/ 5x f

−= − = =

− −

(4) Second iteration,

2

2cos0.995 0.995 / 50.995 1.46901, (1.46901) 1.77385

2sin 0.995 1/ 5x f

−= − = =

− −

After 4 iterations, result

converges rapidly to the

true value.

Multidimension Optimization

2D searches: Ascending a mountain (maximum) orDescending into a valley (minimization)

Example: 2-D Topographic Map of 3-D Mountain

x

y Line of constant f

GRADIENT METHODS

DIRECT METHODS

� Random Search

� Univariate and Pattern Searches

� Gradients and Hessians

� Steepest Ascent Method

Write an M-file objfun1.m

function f = objfun1(x,y)

f = y-x-2*x^2-2*x*y-y^2;

Random Search

Repeatedly evaluates function at randomly selected points

Example: 22 22),( yxyxxyyxf −−−−=

Domain boundaries: x = -2 to 2 and y = 1 to 3

>> rand % Uniformly distributed random numbers % on the interval (0.0,1.0).

ans =0.2311

Random search algorithm: rndsrch.m

function [maxx,maxy,maxf] = rndsrch(objfun,xmin,…xmax,ymin,ymax)

iter = 100;maxf = -inf;

for i = 1:iterx = xmin + (xmax - xmin)*rand;y = ymin + (ymax - ymin)*rand;fn = feval(objfun,x,y);if fn > maxf

maxf = fn;maxx = x;maxy = y;

endend

>>[maxx,maxy,maxf]=rndsrch('objfun1',-2,2,1,3)

100 -0.8050 1.3092 1.2120

200 -1.1409 1.7708 1.2133

500 -1.0006 1.4983 1.2500

1000 -1.0253 1.5030 1.2489

TRUE -1.0000 1.5000 1.2500

iter maxx maxy maxf

50 -0.9024 1.5913 1.2048

maxx = -1.0609 maxy = 1.7063 maxf = 1.2251

12

From (1) move along x axis with y constant to max at (2)

3

Univariate and Pattern Searches

- More efficient and still no require derivative evaluation

- Change one variable at time while others constant

y

x

4

Pattern directions

Powell’s Method

Use pattern directions to find optimum efficiently

y

x

1

2

- Pt. 1 and 2 are obtained by 1-Dsearch from different starting pts.

- Line formed by 1 and 2 will bedirected toward maximum

Conjugate direction

GRADIENT METHODS

Use derivative to locate optima

Gradient :

f∆h∆

y∆

x∆

Slope along axis f

hh

∆=

∆

f ff

x y

∂ ∂∇ = +

∂ ∂i j

The vector is called “delf “ = directional derivative of f (x , y)

= Steepest direction

To determine whether maximum, minimum or saddle

22 2 2

2 2

f f fH

x y x y

∂ ∂ ∂= − ∂ ∂ ∂ ∂

2

2

2

2

If 0 and 0 local minimum

If 0 and 0 local maximum

If 0 saddle point

fH

x

fH

xH

∂• > > →

∂

∂• > < →

∂

• < →

Hessian

SteepestSteepest AscentAscent

Steepest Ascent Method

Climbing a hill with maximum slope

y

x0 : Starting point (x0, y0)

1 Follow steepest ascent pathuntil f (x, y) stop increasing

Repeat the process untilthe maximum is reached

Determine: 1) The best direction of steepest ascent

2) The best value along that search direction

Example: Steepest ascent method

2( , ) , Starting point (2,2)f x y xy=

0 1 2 3 40

1

2

3

4

x

y

>> contour(x,y,fxy)

>> surf(x,y,fxy)

>> mesh(x,y,fxy)

x=0:0.1:4;y=0:0.1:4;fxy=zeros(length(y),length(x));for i=1:length(y)

for j=1:length(x)fxy(i,j)=x(j)*y(i)^2;

endend

M-file to create fxy: fxy.m

Determine elevation: 2(2,2) 2 2 8f = × =

Evaluate partial derivatives,

2 22 4

2 2(2)(2) 8

fy

xf

xyy

∂= = =

∂

∂= = =

∂

Determine the gradient, 4 8f∇ = +i j

Direction, 1 08tan 63.4

4θ − = =

relative to x axis

63.40

Magnitude of 2 2 is 4 8 8.944f∇ + =

We will gain 8.944 units of elevation rise for a unit distance

advanced along this steepest path.

63.4o

0 1 2 3 4x

0

1

2

3

4

y

Relationship between steepest direction and x-y coo rdinate

4 8f∇ = +i jy

x2

2

6

10

h =

0

h =

1

Starting at x0, y0 to any point x, y

in the gradient direction :

Then, we go along h axis to find a

maximum of f(x, y)

hyf

yy

hxf

xx

0

0

∂

∂+=

∂

∂+=

Define a new function g(h) = f(x, y) )hyf

y,hxf

x(f 00∂

∂+

∂

∂+

Find maximum by setting g’(h*) = 0 to find h*

Example: Developing 1-D function along a gradient direction

2 2( , ) 2 2 2 , Starting point ( 1,1)f x y xy x x y= + − − −

Partial derivative at starting point:

2 2 2 2(1) 2 2( 1) 6

2 4 2( 1) 4(1) 6

fy x

xf

x yy

∂= + − = + − − =

∂

∂= − = − − = −

∂

Function along h axis:

0 0

2 2

2

( , ) ( 1 6 ,1 6 )

2( 1 6 )(1 6 ) 2( 1 6 ) ( 1 6 ) 2(1 6 )

( ) 180 72 7

f ff x h y h f h h

x y

h h h h h

g h h h

∂ ∂+ + = − + −∂ ∂

= − + − + − + − − + − −

= − + −

Locate the maximum,

( *) 360 * 72 0

* 0.2

g h h

h

′ = − + =

=

Locate (x, y) coordinates according to h = 0.2,

1 6(0.2) 0.2

1 6(0.2) 0.2

x

y

= − + =

= − = −

Set new starting point to (0.2, -0.2) and repeat the process.

MATLAB Optimization Toolbox

fminunc : the unconstrained optimization function

2 2( , ) (4 2 4 2 1), Starting point ( 1,1)xf x y e x y xy y= + + + + −

Write an M-file objfun2.m

function f = objfun2(x,y)

f = exp(x)*(4x^2+2*y^2+4*x*y+2*y+1);

>> x0 = [-1,1]; % Starting point>> options = optimset(‘LargeScale’,’off’);>> [x,fval,flag,output]=fminunc(‘objfun2’,x0,options );

x =0.5000 -1.0000

Welcome to - TumCivil.com Engfanatic Club : … Methods for Civil Engineers Welcome to Lecture 1 –...

Documents

Transcript of Welcome to - TumCivil.com Engfanatic Club : … Methods for Civil Engineers Welcome to Lecture 1 –...