Harder_Numerical Analysis for Engineering

download Harder_Numerical Analysis for Engineering

of 5

Transcript of Harder_Numerical Analysis for Engineering

  • 8/3/2019 Harder_Numerical Analysis for Engineering

    1/5

    Top of Form

    google-uwdir-key Search

    Bottom of Form

    Numerical Analysis for EngineeringSkip to the content of the web site. ECE Home

    Undergraduate Home

    My Home

    Numerical Analysis

    Table of Contents

    0 Introduction

    1 Error Analysis

    2 Numeric Representation 3 Iteration

    4 Linear Algebra

    4.1 PLU Decomposition

    4.2 PLU Decomposition on Tridiagonal Matrices

    4.3 Positive-definite Matrices

    4.4 Cholesky Decomposition

    4.5 QR Decomposition

    4.6 Eigenvalues and Eigenvectors

    4.7 Norms

    4.8 Ill-conditioned Matrices

    4.9 The Jacobi Method

    4.10 The Gauss-Seidel Method

    4.11 Overrelaxation Techniques

    5 Interpolation

    6 Least Squares

    7 Taylor Series

    8 Bracketing

    9 The Five Techniques

    10 Root Finding

    11 Optimization

    12 Differentiation

    13 Integration

    14 Initial-value Problems

    15 Boundary-value Problems

    http://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/https://ece.uwaterloo.ca/Home/https://ece.uwaterloo.ca/Undergrad/https://ece.uwaterloo.ca/~dwharder/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/Contents/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/00Introduction/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/01Error/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/02Numerics/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/03Iteration/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/lup/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/tri/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/posdef/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/cholesky/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/qr/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/eigen/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/norms/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/illconditioned/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/jacobi/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/gauss/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/overrelax/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/05Interpolation/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/06LeastSquares/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/07TaylorSeries/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/08Bracketing/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/09TheFiveTechniques/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/11Optimization/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/12Differentiation/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/13Integration/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/14IVPs/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/15BVPs/https://ece.uwaterloo.ca/Home/https://ece.uwaterloo.ca/Undergrad/https://ece.uwaterloo.ca/~dwharder/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/Contents/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/00Introduction/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/01Error/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/02Numerics/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/03Iteration/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/lup/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/tri/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/posdef/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/cholesky/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/qr/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/eigen/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/norms/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/illconditioned/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/jacobi/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/gauss/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/overrelax/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/05Interpolation/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/06LeastSquares/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/07TaylorSeries/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/08Bracketing/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/09TheFiveTechniques/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/10RootFinding/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/11Optimization/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/12Differentiation/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/13Integration/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/14IVPs/https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/15BVPs/http://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/
  • 8/3/2019 Harder_Numerical Analysis for Engineering

    2/5

    Appendices

    4.3 Positive-definite Matrices

    Introduction Theory HOWTO Error Analysis Examples Questions Applications in Engineering Matlab Maple

    Introduction

    Positive-definite matrices have some nice properties which we will exploit in finding more convenient

    decompositions which require less memory.

    Background

    Useful background for this topic includes:

    Matrix multiplication and inner (dot) products

    References

    Bradie, Section 3.7, Special Matrices, p.211.

    Weisstein,http://mathworld.wolfram.com/PositiveDefiniteMatrix.html.

    Theory

    A symmetric n n matrix M is said to be positive definite if for all nonzero vectors v, the product

    vTMv > 0.

    Looking more closely at this product, we see that it is the dot product ofvT and Mv. You will recall

    that the dot product of two vectors is equal tovTMv = ||v||2 ||Mv||2 cos(),where is the angle

    between v and Mv, and and thus, for the dot product to be positive, it means that the image ofv

    must have an angle less than 90o, i.e., || < /2.

    It is unreasonable to test every possible vector v to determine if a matrix is positive definite,

    however, there are some cases which appear quite often in engineering where it is quite simple to

    state that a matrix is positive definite. These are covered on the howtos. The following are some

    interesting theorems related to positive definite matrices:

    Theorem 4.2.1

    A matrix is invertible if and only if all of the eigenvalues are non-zero.

    Proof: Please refer to your linear algebra text.

    Theorem 4.2.2A positive definite matrix M is invertible.

    Proof: if it was not, then there must be a non-zero vectorx such that Mx = 0. ThereforexTMx = 0

    which contradicts our assumption aboutM being positive definite.

    Theorem 4.2.3

    All the eigenvalues with corresponding real eigenvectors of a positive definite matrix M are positive.

    https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/Appendices/http://mathworld.wolfram.com/PositiveDefiniteMatrix.htmlhttps://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/Appendices/http://mathworld.wolfram.com/PositiveDefiniteMatrix.html
  • 8/3/2019 Harder_Numerical Analysis for Engineering

    3/5

    Proof: ifx is an eigenvector ofM then Mx = xand therefore xTMx = ||x||2. For this to be positive,

    it follows that > 0.

    HOWTO

    Determining Positive-definiteness

    A symmetric matrix is positive definite if:

    1. all the diagonal entries are positive, and

    2. each diagonal entry is greater than the sum of the absolute values of all other entries in the

    corresponding row/column.

    An arbitrary symmetric matrix is positive definite if and only ifeach of its principal submatrices has

    a positive determinant. This is known as Sylvester's criterion.

    Note that only the last case does the implication go both ways.

    Diagonal Dominance

    A matrix M is row diagonally dominantif

    for i= 1, ..., n, column diagonally dominantif

    for i= 1, ..., n, and diagonally dominant if it is both row and column diagonally dominant.

    Thus, we may restate our first definition as:

    A symmetric row diagonally-dominant matrix with positive diagonal entries is positive definite.

    Error Analysis

    [an error occurred while processing this directive]

    Examples

    Example 1

    Show that ifA is invertible, then ATA is positive definite.

    IfA is invertible, then Av 0 for any vector v 0. Therefore vT(ATA)v = (vTAT)(Av) which is the

    vectorAv dotted with itself, that is, the square of the norm (or length) of the vector. As Av 0, the

    norm must be positive, and thereforevT(ATA)v > 0.

    Questions

    Question 1

    Create a random positive-definite n n matrix in which is also diagonally dominant.

  • 8/3/2019 Harder_Numerical Analysis for Engineering

    4/5

    ( rand(n, n) + (n - 1)*eye( n ) )

    Question 3

    Which of the following matrices are positive definite?

    a.

    b.

    c.

    Applications to Engineering

    The conductance matrix of a RLC circuit is positive definite. Additionally, we will see that the matrix

    defined when performing least-squares fitting is also positive definite.

    As an alternate example, the Hurwitz criteria for the stability of a differential equation requires that

    the constructed matrix be positive definite. This is seen in signals and linear systems.

    MatlabA test for positive definiteness requires that the matrix is symmetric and that all the eigenvalues are

    positive.

    all( all( M == M' ) ) & min( eig( M ) ) > 0

    Alternatively, one may use the test

    M = [...]; % assume M is square

    isposdef = true;

    for i=1:length(M)

    if ( det( M(1:i, 1:i) )

  • 8/3/2019 Harder_Numerical Analysis for Engineering

    5/5

    Douglas Wilhelm HarderDepartment of Electrical and Computer Engineering

    University of Waterloo200 University Avenue WestWaterloo, Ontario, CanadaN2L 3G1Phone: +1 519 888 4567 extension 37023Fax: +1 519 746 3077

    Contact the Author

    Department of Electrical and Computer Engineering

    Faculty of Engineering

    mailto:[email protected]://ece.uwaterloo.ca/http://engineering.uwaterloo.ca/http://www.corel.com/http://www.firefox.com/http://www.vim.org/https://ece.uwaterloo.ca/Computing/Logo/http://www.eng.uwaterloo.ca/~engdev/mailto:[email protected]://ece.uwaterloo.ca/http://engineering.uwaterloo.ca/