MATH20602 Numerical Analysis 1 - School of …mlotz/teaching/math20602-old/... · 2014-01-27 ·...
Transcript of MATH20602 Numerical Analysis 1 - School of …mlotz/teaching/math20602-old/... · 2014-01-27 ·...
M\crManchester Numerical Analysis
NA
MATH20602Numerical Analysis 1
Martin Lotz
School of MathematicsThe University of Manchester
Manchester, January 27, 2014
Outline
General Course Information
Introduction to Numerical Analysis
Prerequisites from Calculus
Outline
General Course Information
Introduction to Numerical Analysis
Prerequisites from Calculus
Organisation
The course website can be found under
http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/
I Tutorials start in second week (on February 4).
I Problem sets in week n are discussed in week n+ 1.
I All the material presented in the lecture will be made availableonline as the course progresses.
I Lecture notes will appear on the website towards the end ofeach week.
I Contact: [email protected]
1 / 24
Organisation
The course website can be found under
http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/
I Tutorials start in second week (on February 4).
I Problem sets in week n are discussed in week n+ 1.
I All the material presented in the lecture will be made availableonline as the course progresses.
I Lecture notes will appear on the website towards the end ofeach week.
I Contact: [email protected]
1 / 24
Organisation
The course website can be found under
http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/
I Tutorials start in second week (on February 4).
I Problem sets in week n are discussed in week n+ 1.
I All the material presented in the lecture will be made availableonline as the course progresses.
I Lecture notes will appear on the website towards the end ofeach week.
I Contact: [email protected]
1 / 24
Organisation
The course website can be found under
http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/
I Tutorials start in second week (on February 4).
I Problem sets in week n are discussed in week n+ 1.
I All the material presented in the lecture will be made availableonline as the course progresses.
I Lecture notes will appear on the website towards the end ofeach week.
I Contact: [email protected]
1 / 24
Organisation
The course website can be found under
http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/
I Tutorials start in second week (on February 4).
I Problem sets in week n are discussed in week n+ 1.
I All the material presented in the lecture will be made availableonline as the course progresses.
I Lecture notes will appear on the website towards the end ofeach week.
I Contact: [email protected]
1 / 24
Organisation
The course website can be found under
http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/
I Tutorials start in second week (on February 4).
I Problem sets in week n are discussed in week n+ 1.
I All the material presented in the lecture will be made availableonline as the course progresses.
I Lecture notes will appear on the website towards the end ofeach week.
I Contact: [email protected]
1 / 24
Example classes
Example classes are there to deepen the understanding of thecourse material.
I Problems sheets should ideally be looked at before theexample class.
I In the example classes,I you will be given some time to work on the problems,I you should ask questions if some parts are not clear,I you will get feedback on your attempts at the problems,I we will go through a selection of problems and their solutions.
2 / 24
Example classes
Example classes are there to deepen the understanding of thecourse material.
I Problems sheets should ideally be looked at before theexample class.
I In the example classes,I you will be given some time to work on the problems,I you should ask questions if some parts are not clear,I you will get feedback on your attempts at the problems,I we will go through a selection of problems and their solutions.
2 / 24
Example classes
Example classes are there to deepen the understanding of thecourse material.
I Problems sheets should ideally be looked at before theexample class.
I In the example classes,I you will be given some time to work on the problems,I you should ask questions if some parts are not clear,I you will get feedback on your attempts at the problems,I we will go through a selection of problems and their solutions.
2 / 24
Important dates
Reserve these dates:
I Wed, March 12: Revision class for midterm test
I Mon, March 17: Midterm (coursework) test (1 hour)
I Wed, May 14: Revision class for final exam
I Date of final exam will be announced when available!
3 / 24
MATLAB
I Matlab, or an equivalent system, is an indispensable tool forthis course.
I Availability:I On campus computersI Student version available at low cost
I The course page contains links to Matlab documentation
I Alternatives: Python, Octave, Mathematica, Sage
I An easy to parse introduction to Matlab is available here:
http://www.guettel.com/teaching/matlab/
4 / 24
MATLAB
I Matlab, or an equivalent system, is an indispensable tool forthis course.
I Availability:I On campus computersI Student version available at low cost
I The course page contains links to Matlab documentation
I Alternatives: Python, Octave, Mathematica, Sage
I An easy to parse introduction to Matlab is available here:
http://www.guettel.com/teaching/matlab/
4 / 24
MATLAB
I Matlab, or an equivalent system, is an indispensable tool forthis course.
I Availability:I On campus computersI Student version available at low cost
I The course page contains links to Matlab documentation
I Alternatives: Python, Octave, Mathematica, Sage
I An easy to parse introduction to Matlab is available here:
http://www.guettel.com/teaching/matlab/
4 / 24
MATLAB
I Matlab, or an equivalent system, is an indispensable tool forthis course.
I Availability:I On campus computersI Student version available at low cost
I The course page contains links to Matlab documentation
I Alternatives: Python, Octave, Mathematica, Sage
I An easy to parse introduction to Matlab is available here:
http://www.guettel.com/teaching/matlab/
4 / 24
MATLAB
I Matlab, or an equivalent system, is an indispensable tool forthis course.
I Availability:I On campus computersI Student version available at low cost
I The course page contains links to Matlab documentation
I Alternatives: Python, Octave, Mathematica, Sage
I An easy to parse introduction to Matlab is available here:
http://www.guettel.com/teaching/matlab/
4 / 24
MATLAB
I Matlab, or an equivalent system, is an indispensable tool forthis course.
I Availability:I On campus computersI Student version available at low cost
I The course page contains links to Matlab documentation
I Alternatives: Python, Octave, Mathematica, Sage
I An easy to parse introduction to Matlab is available here:
http://www.guettel.com/teaching/matlab/
4 / 24
Course Structure
The lecture consists of the following parts
1 Introduction and foundational material (Lectures 1-2)
2 Interpolation (Lectures 3-6)
3 Numerical Integration/Quadrature (Lectures 7-9)
4 Numerical Linear Algebra (Lectures 10-15)
5 Nonlinear Equations (Lectures 16-19)
5 / 24
Course Structure
The lecture consists of the following parts
1 Introduction and foundational material (Lectures 1-2)
2 Interpolation (Lectures 3-6)
3 Numerical Integration/Quadrature (Lectures 7-9)
4 Numerical Linear Algebra (Lectures 10-15)
5 Nonlinear Equations (Lectures 16-19)
5 / 24
Course Structure
The lecture consists of the following parts
1 Introduction and foundational material (Lectures 1-2)
2 Interpolation (Lectures 3-6)
3 Numerical Integration/Quadrature (Lectures 7-9)
4 Numerical Linear Algebra (Lectures 10-15)
5 Nonlinear Equations (Lectures 16-19)
5 / 24
Course Structure
The lecture consists of the following parts
1 Introduction and foundational material (Lectures 1-2)
2 Interpolation (Lectures 3-6)
3 Numerical Integration/Quadrature (Lectures 7-9)
4 Numerical Linear Algebra (Lectures 10-15)
5 Nonlinear Equations (Lectures 16-19)
5 / 24
Course Structure
The lecture consists of the following parts
1 Introduction and foundational material (Lectures 1-2)
2 Interpolation (Lectures 3-6)
3 Numerical Integration/Quadrature (Lectures 7-9)
4 Numerical Linear Algebra (Lectures 10-15)
5 Nonlinear Equations (Lectures 16-19)
5 / 24
Course Structure
The lecture consists of the following parts
1 Introduction and foundational material (Lectures 1-2)
2 Interpolation (Lectures 3-6)
3 Numerical Integration/Quadrature (Lectures 7-9)
4 Numerical Linear Algebra (Lectures 10-15)
5 Nonlinear Equations (Lectures 16-19)
5 / 24
Learning Outcomes
After the lecture you should have
I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;
I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;
I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;
I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;
I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.
6 / 24
Learning Outcomes
After the lecture you should have
I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;
I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;
I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;
I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;
I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.
6 / 24
Learning Outcomes
After the lecture you should have
I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;
I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;
I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;
I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;
I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.
6 / 24
Learning Outcomes
After the lecture you should have
I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;
I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;
I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;
I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;
I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.
6 / 24
Learning Outcomes
After the lecture you should have
I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;
I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;
I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;
I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;
I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.
6 / 24
Learning Outcomes
After the lecture you should have
I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;
I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;
I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;
I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;
I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.
6 / 24
Outline
General Course Information
Introduction to Numerical Analysis
Prerequisites from Calculus
What is Numerical Analysis?
Numerical analysis is concerned with the development and analysisof algorithms for solving problems based on continuousmathematics.
I This includes computing integrals, solving differentialequations, approximating functions based on data samples,minimising functions.
I Applications can be found in any field in science andengineering, such as aircraft design, chemical processes,weather forecasting, signal processing, Google, ...
7 / 24
What is Numerical Analysis?
Numerical analysis is concerned with the development and analysisof algorithms for solving problems based on continuousmathematics.
I This includes computing integrals, solving differentialequations, approximating functions based on data samples,minimising functions.
I Applications can be found in any field in science andengineering, such as aircraft design, chemical processes,weather forecasting, signal processing, Google, ...
7 / 24
What is Numerical Analysis?
Numerical analysis is concerned with the development and analysisof algorithms for solving problems based on continuousmathematics.
I This includes computing integrals, solving differentialequations, approximating functions based on data samples,minimising functions.
I Applications can be found in any field in science andengineering, such as aircraft design, chemical processes,weather forecasting, signal processing, Google, ...
7 / 24
Top Ten Algorithms
1 Monte Carlo method or Metropolis algorithm, devised by John von Neumann,Stanislaw Ulam, and Nicholas Metropolis;
2 simplex method of linear programming, developed by George Dantzig;
3 Krylov Subspace Iteration method, developed by Magnus Hestenes, EduardStiefel, and Cornelius Lanczos;
4 Householder matrix decomposition, developed by Alston Householder;
5 Fortran compiler, developed by a team lead by John Backus;
6 QR algorithm for eigenvalue calculation, developed by J Francis;
7 Quicksort algorithm, developed by Anthony Hoare;
8 Fast Fourier Transform, developed by James Cooley and John Tukey;
9 Integer Relation Detection Algorithm, developed by Helaman Ferguson andRodney Forcade;
10 fast Multipole algorithm, developed by Leslie Greengard and Vladimir Rokhlin;
(List assembled by Dongarra and Sullivan)
8 / 24
Numerical Analysis and Manchester: The Baby
The world’s first stored-program computer, the Small Scale ExperimentalMachine (SSEM), also called The Baby, was developed at Manchester
University in 1948. (Picture credit: MOSI)
9 / 24
Numerical Analysis and Manchester: Alan Turing
Alan Turing not only developed the mathematical foundations of
programming, he also did pioneering work in numerical analysis by
introducing the concept of Condition Number.
10 / 24
The Challenges of Numerical Analysis
A particular challenge for Numerical Analysis is the fact thatcomputers are finite devices.
I Most numerical problems can’t be solved in a finite amount oftime.
I Most real numbers can’t be represented in a finite amount ofspace.
Since none of the numbers which we take out fromlogarithmic and trigonometric tables admit of absoluteprecision, but are all to a certain extent approximate only,the results of all calculations performed by the aid ofthese numbers can only be approximately true.- C.F. Gauss (1809)
11 / 24
The Challenges of Numerical Analysis
A particular challenge for Numerical Analysis is the fact thatcomputers are finite devices.
I Most numerical problems can’t be solved in a finite amount oftime.
I Most real numbers can’t be represented in a finite amount ofspace.
Since none of the numbers which we take out fromlogarithmic and trigonometric tables admit of absoluteprecision, but are all to a certain extent approximate only,the results of all calculations performed by the aid ofthese numbers can only be approximately true.- C.F. Gauss (1809)
11 / 24
The Challenges of Numerical Analysis
A particular challenge for Numerical Analysis is the fact thatcomputers are finite devices.
I Most numerical problems can’t be solved in a finite amount oftime.
I Most real numbers can’t be represented in a finite amount ofspace.
Since none of the numbers which we take out fromlogarithmic and trigonometric tables admit of absoluteprecision, but are all to a certain extent approximate only,the results of all calculations performed by the aid ofthese numbers can only be approximately true.- C.F. Gauss (1809)
11 / 24
The Challenges of Numerical Analysis
A particular challenge for Numerical Analysis is the fact thatcomputers are finite devices.
I Most numerical problems can’t be solved in a finite amount oftime.
I Most real numbers can’t be represented in a finite amount ofspace.
Since none of the numbers which we take out fromlogarithmic and trigonometric tables admit of absoluteprecision, but are all to a certain extent approximate only,the results of all calculations performed by the aid ofthese numbers can only be approximately true.- C.F. Gauss (1809)
11 / 24
The E word
An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...
I Modelling errors;
I Measurement errors;
I Rounding errors (most numbers cannot be stored exactly tocomputer precision);
I Truncation or approximation errors.
http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html
12 / 24
The E word
An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...
I Modelling errors;
I Measurement errors;
I Rounding errors (most numbers cannot be stored exactly tocomputer precision);
I Truncation or approximation errors.
http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html
12 / 24
The E word
An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...
I Modelling errors;
I Measurement errors;
I Rounding errors (most numbers cannot be stored exactly tocomputer precision);
I Truncation or approximation errors.
http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html
12 / 24
The E word
An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...
I Modelling errors;
I Measurement errors;
I Rounding errors (most numbers cannot be stored exactly tocomputer precision);
I Truncation or approximation errors.
http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html
12 / 24
The E word
An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...
I Modelling errors;
I Measurement errors;
I Rounding errors (most numbers cannot be stored exactly tocomputer precision);
I Truncation or approximation errors.
http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html
12 / 24
The E word
An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...
I Modelling errors;
I Measurement errors;
I Rounding errors (most numbers cannot be stored exactly tocomputer precision);
I Truncation or approximation errors.
http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html
12 / 24
The E word
A further source of error... photocopiers!
Scan on a Xerox Workcentre 7535. More information can be found on
Prof. Nick Higham’s blog.
13 / 24
The E word
A further source of error... photocopiers!
Scan on a Xerox Workcentre 7535. More information can be found on
Prof. Nick Higham’s blog.
13 / 24
Types of errors
I Modelling and measurement errors fall outside the scope ofnumerical analysis, but we need to be aware of their presence!
I Rounding errors arise due to the fact that computers operatewith limited storage space, and results of calculations arealways rounded to the neares representable number (seefloating point numbers).
I Truncation errors arise when replacing functions or equationswith an approximations. For example, the Taylorapproximation of a function gives
f(x) ≈f(x0) + f ′(x0)(x− x0) +1
2f ′′(x0)(x− x0)2 + · · ·
+f (n)(x0)
n!(x− x0)n.
I Errors can accumulate!
14 / 24
Types of errors
I Modelling and measurement errors fall outside the scope ofnumerical analysis, but we need to be aware of their presence!
I Rounding errors arise due to the fact that computers operatewith limited storage space, and results of calculations arealways rounded to the neares representable number (seefloating point numbers).
I Truncation errors arise when replacing functions or equationswith an approximations. For example, the Taylorapproximation of a function gives
f(x) ≈f(x0) + f ′(x0)(x− x0) +1
2f ′′(x0)(x− x0)2 + · · ·
+f (n)(x0)
n!(x− x0)n.
I Errors can accumulate!
14 / 24
Types of errors
I Modelling and measurement errors fall outside the scope ofnumerical analysis, but we need to be aware of their presence!
I Rounding errors arise due to the fact that computers operatewith limited storage space, and results of calculations arealways rounded to the neares representable number (seefloating point numbers).
I Truncation errors arise when replacing functions or equationswith an approximations. For example, the Taylorapproximation of a function gives
f(x) ≈f(x0) + f ′(x0)(x− x0) +1
2f ′′(x0)(x− x0)2 + · · ·
+f (n)(x0)
n!(x− x0)n.
I Errors can accumulate!
14 / 24
Types of errors
I Modelling and measurement errors fall outside the scope ofnumerical analysis, but we need to be aware of their presence!
I Rounding errors arise due to the fact that computers operatewith limited storage space, and results of calculations arealways rounded to the neares representable number (seefloating point numbers).
I Truncation errors arise when replacing functions or equationswith an approximations. For example, the Taylorapproximation of a function gives
f(x) ≈f(x0) + f ′(x0)(x− x0) +1
2f ′′(x0)(x− x0)2 + · · ·
+f (n)(x0)
n!(x− x0)n.
I Errors can accumulate!
14 / 24
Types of errors
I Modelling and measurement errors fall outside the scope ofnumerical analysis, but we need to be aware of their presence!
I Rounding errors arise due to the fact that computers operatewith limited storage space, and results of calculations arealways rounded to the neares representable number (seefloating point numbers).
I Truncation errors arise when replacing functions or equationswith an approximations. For example, the Taylorapproximation of a function gives
f(x) ≈f(x0) + f ′(x0)(x− x0) +1
2f ′′(x0)(x− x0)2 + · · ·
+f (n)(x0)
n!(x− x0)n.
I Errors can accumulate!
14 / 24
Measuring errors
In order to quantify errors in our solutions we need to define ameasure for the error
I If x̂ is an approximation to a quantity x then the absoluteerror is defined by
|x− x̂|
I The relative error is defined by
|x̂− x||x|
, x 6= 0
15 / 24
Measuring errors
In order to quantify errors in our solutions we need to define ameasure for the error
I If x̂ is an approximation to a quantity x then the absoluteerror is defined by
|x− x̂|
I The relative error is defined by
|x̂− x||x|
, x 6= 0
15 / 24
Relative errors
Example
Relative vs absolute measures
An error of 1cm makes a big difference for small objects, but is not considered an error at all on a cosmic scale.
16 / 24
Significant Digits
When doing calculations by hand, we will be using the concept ofsignificant figures.
I Starting with the first non-zero digit on the left, count all thefigures on the right of it, including trailing zeros if they are tothe right of the decimal point.
I 1.2048, 12.040 and 0.012040 all have 5 significant figures.
I Concept can be made more precise using the normalisedscientific notation.
I An approximation x̂ of a number x is correct to n significantfigures, if both number round to the same number to ksignificant digits.
17 / 24
Significant Digits
When doing calculations by hand, we will be using the concept ofsignificant figures.
I Starting with the first non-zero digit on the left, count all thefigures on the right of it, including trailing zeros if they are tothe right of the decimal point.
I 1.2048, 12.040 and 0.012040 all have 5 significant figures.
I Concept can be made more precise using the normalisedscientific notation.
I An approximation x̂ of a number x is correct to n significantfigures, if both number round to the same number to ksignificant digits.
17 / 24
Significant Digits
When doing calculations by hand, we will be using the concept ofsignificant figures.
I Starting with the first non-zero digit on the left, count all thefigures on the right of it, including trailing zeros if they are tothe right of the decimal point.
I 1.2048, 12.040 and 0.012040 all have 5 significant figures.
I Concept can be made more precise using the normalisedscientific notation.
I An approximation x̂ of a number x is correct to n significantfigures, if both number round to the same number to ksignificant digits.
17 / 24
Significant Digits
When doing calculations by hand, we will be using the concept ofsignificant figures.
I Starting with the first non-zero digit on the left, count all thefigures on the right of it, including trailing zeros if they are tothe right of the decimal point.
I 1.2048, 12.040 and 0.012040 all have 5 significant figures.
I Concept can be made more precise using the normalisedscientific notation.
I An approximation x̂ of a number x is correct to n significantfigures, if both number round to the same number to ksignificant digits.
17 / 24
Significant Digits
When doing calculations by hand, we will be using the concept ofsignificant figures.
I Starting with the first non-zero digit on the left, count all thefigures on the right of it, including trailing zeros if they are tothe right of the decimal point.
I 1.2048, 12.040 and 0.012040 all have 5 significant figures.
I Concept can be made more precise using the normalisedscientific notation.
I An approximation x̂ of a number x is correct to n significantfigures, if both number round to the same number to ksignificant digits.
17 / 24
Floating Point Arithmetic
I On a computer, numbers are stored as a bounded sequence ofbits, or 0 and 1 digits, usually using 32 bits (single precision)or 64 bits (double precision) per number.
I In double precision, a number is represented as
x = ±f × fe,
whereI f is a number in [0, 1], represented using 52 bits,I e is an exponent, represented using 11 bits.I The remaining bit is used as a sign.
I The largest representable number is of the order ±10308, thesmallest of the order 10−308.
I Floating point numbers form a finite subset of the realnumbers and are not uniformly spaced!
18 / 24
Floating Point Arithmetic
I On a computer, numbers are stored as a bounded sequence ofbits, or 0 and 1 digits, usually using 32 bits (single precision)or 64 bits (double precision) per number.
I In double precision, a number is represented as
x = ±f × fe,
whereI f is a number in [0, 1], represented using 52 bits,I e is an exponent, represented using 11 bits.I The remaining bit is used as a sign.
I The largest representable number is of the order ±10308, thesmallest of the order 10−308.
I Floating point numbers form a finite subset of the realnumbers and are not uniformly spaced!
18 / 24
Floating Point Arithmetic
I On a computer, numbers are stored as a bounded sequence ofbits, or 0 and 1 digits, usually using 32 bits (single precision)or 64 bits (double precision) per number.
I In double precision, a number is represented as
x = ±f × fe,
whereI f is a number in [0, 1], represented using 52 bits,I e is an exponent, represented using 11 bits.I The remaining bit is used as a sign.
I The largest representable number is of the order ±10308, thesmallest of the order 10−308.
I Floating point numbers form a finite subset of the realnumbers and are not uniformly spaced!
18 / 24
Floating Point Arithmetic
I On a computer, numbers are stored as a bounded sequence ofbits, or 0 and 1 digits, usually using 32 bits (single precision)or 64 bits (double precision) per number.
I In double precision, a number is represented as
x = ±f × fe,
whereI f is a number in [0, 1], represented using 52 bits,I e is an exponent, represented using 11 bits.I The remaining bit is used as a sign.
I The largest representable number is of the order ±10308, thesmallest of the order 10−308.
I Floating point numbers form a finite subset of the realnumbers and are not uniformly spaced!
18 / 24
Floating Point Arithmetic
I On a computer, numbers are stored as a bounded sequence ofbits, or 0 and 1 digits, usually using 32 bits (single precision)or 64 bits (double precision) per number.
I In double precision, a number is represented as
x = ±f × fe,
whereI f is a number in [0, 1], represented using 52 bits,I e is an exponent, represented using 11 bits.I The remaining bit is used as a sign.
I The largest representable number is of the order ±10308, thesmallest of the order 10−308.
I Floating point numbers form a finite subset of the realnumbers and are not uniformly spaced!
18 / 24
Big O notation
Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.
I Running time is measured as a function of a parameter n thatdescribes the problem.
I We are usually more interested in the order of magnitude thanexact operation count.
I We normally don’t care if an algorithm uses 0.5n2 or 20n2
operations, but will consider a difference between n log(n) orn4 operation significant.
I Given function f(n) and g(n), we say that
f(n) ∈ O(g(n)) or f(n) = O(g(n)),
if there exists C > 0 and n0, such that for n ≥ n0 we havef(n) < C · g(n).
I Examples: n log(n) = O(n2), n3 + 10308n ∈ O(n3).
19 / 24
Big O notation
Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.
I Running time is measured as a function of a parameter n thatdescribes the problem.
I We are usually more interested in the order of magnitude thanexact operation count.
I We normally don’t care if an algorithm uses 0.5n2 or 20n2
operations, but will consider a difference between n log(n) orn4 operation significant.
I Given function f(n) and g(n), we say that
f(n) ∈ O(g(n)) or f(n) = O(g(n)),
if there exists C > 0 and n0, such that for n ≥ n0 we havef(n) < C · g(n).
I Examples: n log(n) = O(n2), n3 + 10308n ∈ O(n3).
19 / 24
Big O notation
Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.
I Running time is measured as a function of a parameter n thatdescribes the problem.
I We are usually more interested in the order of magnitude thanexact operation count.
I We normally don’t care if an algorithm uses 0.5n2 or 20n2
operations, but will consider a difference between n log(n) orn4 operation significant.
I Given function f(n) and g(n), we say that
f(n) ∈ O(g(n)) or f(n) = O(g(n)),
if there exists C > 0 and n0, such that for n ≥ n0 we havef(n) < C · g(n).
I Examples: n log(n) = O(n2), n3 + 10308n ∈ O(n3).
19 / 24
Big O notation
Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.
I Running time is measured as a function of a parameter n thatdescribes the problem.
I We are usually more interested in the order of magnitude thanexact operation count.
I We normally don’t care if an algorithm uses 0.5n2 or 20n2
operations, but will consider a difference between n log(n) orn4 operation significant.
I Given function f(n) and g(n), we say that
f(n) ∈ O(g(n)) or f(n) = O(g(n)),
if there exists C > 0 and n0, such that for n ≥ n0 we havef(n) < C · g(n).
I Examples: n log(n) = O(n2), n3 + 10308n ∈ O(n3).
19 / 24
Big O notation
Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.
I Running time is measured as a function of a parameter n thatdescribes the problem.
I We are usually more interested in the order of magnitude thanexact operation count.
I We normally don’t care if an algorithm uses 0.5n2 or 20n2
operations, but will consider a difference between n log(n) orn4 operation significant.
I Given function f(n) and g(n), we say that
f(n) ∈ O(g(n)) or f(n) = O(g(n)),
if there exists C > 0 and n0, such that for n ≥ n0 we havef(n) < C · g(n).
I Examples: n log(n) = O(n2), n3 + 10308n ∈ O(n3).
19 / 24
Big O notation
Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.
I Running time is measured as a function of a parameter n thatdescribes the problem.
I We are usually more interested in the order of magnitude thanexact operation count.
I We normally don’t care if an algorithm uses 0.5n2 or 20n2
operations, but will consider a difference between n log(n) orn4 operation significant.
I Given function f(n) and g(n), we say that
f(n) ∈ O(g(n)) or f(n) = O(g(n)),
if there exists C > 0 and n0, such that for n ≥ n0 we havef(n) < C · g(n).
I Examples: n log(n) = O(n2), n3 + 10308n ∈ O(n3).
19 / 24
Outline
General Course Information
Introduction to Numerical Analysis
Prerequisites from Calculus
Intermediate Value Theorem
If you start above ground and end up below ground, thenyou have to cross the surface at some point.
Let f be continuous on [a, b]. Then f is bounded on [a, b] and if ysatisfies
infx∈[a,b]
f(x) ≤ y ≤ supx∈[a,b]
f(x),
then there exists ξ ∈ [a, b] such that f(ξ) = y. In particular, theinfimum and supremum are achieved.
20 / 24
Taylor’s Theorem
Let f be a function on [a, b], such that n derivatives of f exist andare continuous on [a, b]. Assume further that f (n) is differentiableon (a, b). Let x, x0 be in [a, b]. Then there exists ξ ∈ (a, b) suchthat
f(x) = f(x0) + f ′(x0)(x− x0) +1
2f ′′(x0)(x− x0)2 + · · ·
+f (n)(x0)
n!(x− x0)n +
f (n+1)(ξ)
(n+ 1)!(x− x0)n+1
21 / 24
Taylor Expansion: Example
sin(x) = x− x3
3!+x5
5!− · · ·
0 0.5 1 1.5 2 2.5 3 3.5−3
−2
−1
0
1
2
3
4
Taylor expansion of sin(x).
22 / 24
Mean Value Theorem
A special case is the Mean Value Theorem:Let f be continuous on [a, b] and differentiable on (a, b). Thenthere exists a number ξ ∈ (a, b) such that
f(x) = f(x0) + f ′(ξ)(x− x0).
This can also be written as
f ′(ξ) =f(x)− f(x0)
x− x0.
23 / 24
Rolle’s Theorem
Yet another special case is Rolle’s Theorem:
If you end up at the same height at which you started,then the path wasn’t all up or all down.
Let f be continuous on [a, b] and differentiable on (a, b), and suchthat f(a) = f(b). Then there exists a number ξ ∈ (a, b) such thatf ′(ξ) = 0.
24 / 24