Lecture 1 - Introduction to Numerical...

Lecture 1Introduction to Numerical Computing

J. Chaudhry (Zeb)

Department of Mathematics and StatisticsUniversity of New Mexico

J. Chaudhry (Zeb) (UNM) Math/CS 375 1 / 29

Numerical Computing?

Numerical Analysis?


Numerical Computation vs. Symbolic Computation

Numerical Computation: involve numbers directlyI manipulate numbers to produce a numerical result

Symbolic Computation: symbols represent numbersI manipulate symbols according to mathematical rules to produce a symbolic

result

Example (numerical)(17.36)2 − 1

17.36 + 1= 16.36

Example (symbolic)x2 − 1x + 1

= x − 1


Analytic Solution vs. Numerical Solution

Analytic Solution (a.k.a. symbolic): The exact numerical or symbolicrepresentation of the solution

I may use special characters such as π, e, or tan (83)

Numerical Solution: The computational representation of the solutionI entirely numerical

Example (analytic)1413π

tan (83)

Example (numerical)

0.25

0.33333 . . . (?)

3.14159 . . . (?)

0.88472 . . . (?)


Numerical Computation and Approximation

Numerical Approximation is needed to carry out the steps in thenumerical calculation. The overall process is a numerical computation.

Example (symbolic computation, numerical solution)12+

13+

14− 1 =

112

= 0.083333333 . . .

Example (numerical computation, numerical approximation)

0.500 + 0.333 + 0.250 − 1.000 = 0.083


Method vs. Algorithm vs. Implementation

Method: a general (mathematical) framework describing the solutionprocessAlgorithm: a detailed description of executing the methodImplementation: a particular instantiation of the algorithm

Is it a “good” method?Is it a robust algorithm?Is it a fast implementation?


The Big Theme

Accuracy Cost


History: Numerical Algorithms

date to 1650 BC: The Rhind Papyrus of ancient Egypt contains 85problems; many use numerical algorithms (T. Chartier, Davidson)

Approximates π with(8/9)2 ∗ 4 ≈ 3.1605


History: Archimedes

287-212BC developed the ”Method ofExhaustion”Method for determining π

I find the length of the permieter of apolygon inscribed inside a circle ofradius 1/2

I find the perimiter of a polygoncircumscribed outside a circle ofradius 1/2

I the value of π is between these twolengths


History: Method of Exhaustion

A circle is not a polygonA circle is a polygon with an infinite number of sidesCn = circumference of n-sided polygon inscribed in circle of radius 1/2limn→∞ Cn = π

Archimedes deterimined

22371

<π <227

3.1408 <π < 3.1429

two places of accuracy....

http://www.pbs.org/wgbh/nova/archimedes/pi.html


http://www.pbs.org/wgbh/nova/archimedes/pi.html

History: Method of Machin

Around 1700, John Machin discovered the trig identity

π = 16 arctan (15) − 4 arctan (

1239

)

Led to calculation of the first 100 digits of πUses the Taylor series of arctan in the algorithm

arctan (x) = x −x3

3+

x5

5−

x7

7. . .

Used until 1973 to find the first Million digits


Numerical Analysis

Definition (Trefethen)Study of algorithms for the problems of continuous mathematics

We’ve been doing this since Calculus (and before!)Riemann sum for calculating a definite integralNewton’s MethodTaylor’s Series expansion + truncation


Big Questions

How algorithms work and how they failWhy algorithms work and why they fail

Connects mathematics and computer scienceNeed mathematical theory, computer programming, and scientific inquiry


Numerical Analysis

A Numerical Analyst needscomputational knowledge (e.g. programming skills)understanding of the application (physical intuition for validation)mathematical ability to construct a meaningful algorithm


Numerical Analysis

Numerical focus:Approximation An approximate solution is sought. How close is this to the

desired solution?Efficiency How fast and cheap (memory) can we compute a solution?

Stability Is the solution sensitive to small variations in the problem setup?Error What is the role of finite precision of our computers?


Numerical Analysis

Why?Numerical methods improve scientific simulationSome disasters attributable to bad numerical computing (Douglas Arnold)

I The Patriot Missile failure, in Dharan, Saudi Arabia, on February 25, 1991which resulted in 28 deaths, is ultimately attributable to poor handling ofrounding errors.

I The explosion of the Ariane 5 rocket just after lift-off on its maiden voyage offFrench Guiana, on June 4, 1996, was ultimately the consequence of asimple overflow.

I The sinking of the Sleipner A offshore platform in Gandsfjorden nearStavanger, Norway, on August 23, 1991, resulted in a loss of nearly onebillion dollars. It was found to be the result of inaccurate finite elementanalysis.


I thought we were studying ”Numerical Computing”?!

Numerical analysis is the study of numerical computing

Numerical Analysis: understanding general behavior of numericalcomputingNumerical Computing: understanding how to apply certain methods tosolve specific tasksAs computational scientists, we need to understand the concepts ofnumerical analysis and implementation aspects of the numericalcomputing

We thus focus onMatlab implementation

I fast learning curveI quick time-to-production: low development timesI a major development environment in scientific computingI integrated graphics

Errors in computationSpecific methods for solving linear and nonlinear systems, root finding,integrating, interpolation, etc.J. Chaudhry (Zeb) (UNM) Math/CS 375 17 / 29

Applications: Mathematics, Engineering, ComputerScience...

In Engineering

Biomolecular Systems

Rocket Simulations

Crack propagation


Applications: Mathematics, Engineering, ComputerScience...

In Computer Science?Machine learning: Training deep neural networks (iterativemethods)AI: transitions, state systems, patterns (eigenvalues, linearalgebra)Informatics: Google matrix, Amazon recommendations(eigenvalues, linear algebra, sparse matrices, iterative methods)Graphics: image compression, representation ofcurves/surfaces/lighting (interpolation, differentiation, etc)Security: ssh (random numbers)Economics/Finance: modeling/simulation of financial data(monte carlo)Scientific computing: design of algorithms for highperformance/parallel computing.


Course Info

http://math.unm.edu/˜jehanzeb/math375fa19.html

Book: Numerical Mathematics and Computing, SeventhEdition, by Cheney and Kincaid (or 6th)

Zeb’s contact:

Office: SMLC 310

Office Hours: Tuesday: 11:00 am - 1:00 pm, Wednesday: 10:00 am - 11:00 am

email: [email protected]


http://math.unm.edu/~jehanzeb/math375fa19.html

Assessment


Grades:Homeworks 50%Midterm 20%Final Exam 30%

Homework:Assigned approximately weeklysubmitted as a reportsource code has to be includedno graphics by handaccepted for 50% credit up to one week laterno dropped scores (or maybe at most one)collaborate, groups of 2cite collaborators at the top, submit one report



Finally. . .


Schedule and Notes:on the webeverything is tentativefinal exam is fixedGrades posted on UNM Learn

Questions?



Preliminaries...

A computational experiment...relative vs absolute errorMatlab

GoalsTo think like a numerical analystUnderstanding error in numerical computingUtilizing Matlab


Typical Process

GoalFind f ′(1.0) for function f (x) = ex.

ProblemDon’t (want to need to) know f ′(x) explicitly.

MethodUse definition.

f ′(x) = limh→0

f (x + h) − f (x)h


Typical Process continued

MethodUse definition.

f ′(x) = limh→0

f (x + h) − f (x)h

Listing 1: Algorithm (psuedocode)1

2 h=13 x=14 for i=1 to 20 do5 h = h/26 y = (f (x + h) − f (x))/h7 err = |(f ′(x) − y)|8 end

Listing 2: Implementation (Matlab)1 f= @(x) (exp(x));

2 df= @(x) (exp(x));

3 h=1;

4 x=1;

5 for i=1:20

6 h=h/2;

7 y=(f(x+h) - f(x))/h;

8 err(i) = abs(df(x)-y);

9 end


Measuring Error

Here we used the absolute error:

Error(xa) = |xt − xa|

= |true value − approximate value|

This doesn’t tell the whole story. For example, if the values are large, likebillions, then an Error of 100 is small. If the values are smaller, sayaround 10, then an Error of 100 is large. We need the relative error:

Rel(xa) =

∣∣∣∣xt − xa

xt

∣∣∣∣=

∣∣∣∣ true value − approximate valuetrue value

∣∣∣∣More on types of errors in future...


What is MATLAB?

both a computing environment and a languageinitially developed as an easy interface to LAPACK: Linear AlgebraPackage (FORTRAN libraries)MATrix LABoratoryWritten in C. For matrix computations, it calls C/Fortran libraries == FastMatlab + “Toolboxes”

I Symbolic Math Toolbox: mathematical manipulation of symbolsI Partial Differential Equation Toolbox: tools for solving PDEs in 2-DI Statistics Toolbox: statistical data analysisI Image processing toolbox: visualization and image analysisI Bioinformatics toolbox: computational molecular biologyI Compiler: application developmentI many many more.

http://www.mathworks.com


http://www.mathworks.com

Why Matlab?

The Good:Fast development timesno compiling, easy debuggingaccessible syntax and language constructsin-house graphics capabilitiestons of basic ”libraries” or functions availablemany more complicated “toolboxes” can be added

The Bad:small coding mistakes can result in slow codeloops are computationally intensivelanguage is limited: no templates etc.

The Ugly:proprietary (but the language format is open)expensive (but free for UNM students/faculty?)the open source substitute, Octave, is not fully compatible


What is Python?

Full fledged, easy to use programming language

The Good:All the good stuff that Matlab has!Large range of librariesFree to use forever!Language has full object oriented supportJupyter notebooks are great for displaying text, code and figures in thesame file

The Bad:Installing it and the associated libraries can be a pain (although manygood installers now available)Slightly steeper learning curve compared to Matlab


Lecture 1 - Introduction to Numerical...

Documents

Transcript of Lecture 1 - Introduction to Numerical...