Square or Rectangular Matrices

Factorizations and Decompositions

QRA QR factorization

RQA ˆˆ Reduced QR factorization

TVUA SVD

Reduced SVDTVUA ˆˆˆ

Square Matrices

Factorizations and Decompositions

LUPA LU factorization

RRA T Cholesky factorization

TUTUA Schur Factorization

Arnoldi iterationnTnn AQQH

Lanczos iterationnTnn AQQT

Eigenvalue decomposition1 XXA

Core Problem

bAx

Three fundamental problems are considered

1) Square system of linear equations

bAx 2) Least Square Problem: m>n

xAx 3) Eigenvalue Problem (2 hw4)

Core Problem

bAx Three fundamental problems are considered

1) Square system of linear equations

Direct Methods

LU factorization

Cholesky factorization

Iterative Methods

stationarysplitting

JacobiGauss-SeidelSOR

Krylov

CG (SPD)MINRES (Symm)GMRES (general)

Lanczos

Arnoldi

Core Problem

bAx Three fundamental problems are considered

Normal Equation QR-factorization

2) Least Square Problem: m>n

bAAxA TT bAx

bxRQ ˆˆ

SVD

bAx

bxVU T ˆˆˆ

Core Problem

xAx Three fundamental problems are considered

QR-algorithm

PHASE-I

3) Eigenvalue Problem

iii

iii

QRA

RQA

1

TUTUA Schur Factorization

TQAQH TQAQT

PHASE-II

algorithmQR

shift

Divide and Conqur 21,TTT

Applications

Data Fitting Least square Problem

Image processing Trrr

TT vuvuvuA 222111

SVD

bAx BVP (PDE or ODE)linear

FDM, FEM, FVM

Non-linear0)( xF

Applications

Non-linear system of equations: 0)( xF)('

)(1

n

nnn xf

xfxx Newton Raphson Method: Single variable

)()(' 11 nnnn xFxFxx

)()(' 1 nnnn xFxxxF bAx

j

iij x

FA

jacobian

In Newton Raphson Method, each iteration requires solving one linear system of equations

Norms and Stability

Vector p-norm)1(

/1

1

2

pxxpn

iip

Matrix p-norm

n

m

Rxnm x

AxA

n

0,

sup

Frobenius norm: 21

1

2

2

n

jiFaA

A ofcolumn th -j ja

)( AAtrA T

F

Condition number1)()( AAAAcond

Norms and Stability

A well-conditioned Problem

Frobenius norm: 21

1

2

2

n

jiFaA

A ofcolumn th -j ja

)( AAtrA T

F

Condition number1)()( AAAAcond

Is one with the property that all small changes in initial data lead to only small changes in the solution

An ill-conditioned ProblemIs one with the property that some small changes in initial data lead to a large change in the solution

MATLABLU factorization: lu, luinc

Cholesky factorization: chol, cholinc

Schur Factorization: schur

Arnoldi, Lanczos: hess

Conjugate Gradient: pcg

Eigenvalue decomposition: eig

QR factorization: qr

Singula Value Decompsition: svd

Minimal Residuals: minres

Generalized Minimal Residuals: gmres

Solving linear system: A\b

Least Square Problem: lsqr

References

Focuses on the use of iterative methods for solving large sparse systems of linear equations

an in-depth, up-to-date view of practical algorithms for solving large-scale linear systems of equations

Trefethen: encyclopedia

PDE in Fluid and its iterative

References

software library for numerical linear algebra

LAPACKLINPACKNAG (The Numerical Algorithms Group)

IMSLBLAS

www.netlib.org

software library for numerical linear algebra http://www.netlib.org/utk/people/JackDongarra/la-sw.html

software library for numerical linear algebra

Paralleldistributed methods

FORTRANc

Iterative methods for sparse and structured linear systems

JOURNALS

SIAM

SIAM Journal on Scientific Computing (SISC)SIAM Journal on Matrix Analysis and Applications (SIMAX)SIAM Journal on Numerical Analysis (SINUM)SIAM Review (SIREV)

Elsevier

Linear Algebra and its Applications (LAA)Journal of Computational and Applied Mathematics (JCAM)Applied Numerical Mathematics (APNUM)Applied Mathematics and Computation (AMC)Mathematical and Computer Modeling (MCM)Mathematics and Computers in Simulation (MCM)Computers and Mathematics with Applications

Wiley InterScience

Numerical Linear Algebra with Applications (NLAA)

Springer

Numerische MathematikComputingFoundations of Computational MathematicsNumerical AlgorithmsJournal of Scientific Computing

Free journals

Electronic Transactions on Numerical Analysis (ETNA)Electronic Journal of Linear Algebra (ELA)Taylor & Francis

0.589: International Journal on Computer Mathematics

Acta NumericaAMS

Mathematics of Computation

Oxford University PressIMA Journal of Numerical AnalysisJournal of Computational MathematicsComputational Methods in Applied MathematicsLinear and Multilinear Algebra (LAMA)NA-Digest