Linear Algebra Review
description
Transcript of Linear Algebra Review
-
Linear Algebra Review*CS479/679 Pattern Recognition Dr. George Bebis
-
n-dimensional VectorAn n-dimensional vector v is denoted as follows:
The transpose vT is denoted as follows:
-
Inner (or dot) productGiven vT = (x1, x2, . . . , xn) and wT = (y1, y2, . . . , yn), their dot product defined as follows:
or(scalar)
-
Orthogonal / Orthonormal vectorsA set of vectors x1, x2, . . . , xn is orthogonal if
A set of vectors x1, x2, . . . , xn is orthonormal if
-
Linear combinationsA vector v is a linear combination of the vectors v1, ..., vk if:
where c1, ..., ck are constants.
Example: vectors in R3 can be expressed as a linear combinations of unit vectors i = (1, 0, 0), j = (0, 1, 0), and k = (0, 0, 1)
-
Space spanningA set of vectors S=(v1, v2, . . . , vk ) span some space W if every vector in W can be written as a linear combination of the vectors in S
- The unit vectors i, j, and k span R3
w
-
Linear dependenceA set of vectors v1, ..., vk are linearly dependent if at least one of them is a linear combination of the others.
(i.e., vj does not appear on the right side)
-
Linear independenceA set of vectors v1, ..., vk is linearly independent if no vector can be represented as a linear combination of the remaining vectors, i.e.:
Example:
-
Vector basisA set of vectors (v1, ..., vk) forms a basis in some vector space W if:(1) (v1, ..., vk) are linearly independent(2) (v1, ..., vk) span W
Standard bases:R2R3Rn
-
Matrix OperationsMatrix addition/subtractionMatrices must be of same size.
Matrix multiplication
Condition: n = qm x nq x pm x p
-
Identity Matrix
-
Matrix Transpose
-
Symmetric MatricesExample:
-
Determinants2 x 23 x 3n x nProperties:
-
Matrix InverseThe inverse A-1 of a matrix A has the property: AA-1=A-1A=I
A-1 exists only if
TerminologySingular matrix: A-1 does not existIll-conditioned matrix: A is close to being singular
-
Matrix Inverse (contd)Properties of the inverse:
-
Matrix traceProperties:
-
Rank of matrixEqual to the dimension of the largest square sub-matrix of A that has a non-zero determinant.
Example:
has rank 3
-
Rank of matrix (contd)Alternative definition: the maximum number of linearly independent columns (or rows) of A.
i.e., rank is not 4!Example:
-
Rank of matrix (contd)
-
Eigenvalues and EigenvectorsThe vector v is an eigenvector of matrix A and is an eigenvalue of A if:
i.e., the linear transformation implied by A cannot change the direction of the eigenvectors v, only their magnitude.
(assume non-zero v)
-
Computing and vTo find the eigenvalues of a matrix A, find the roots of the characteristic polynomial:
Example:
-
PropertiesEigenvalues and eigenvectors are only defined for square matrices (i.e., m = n)Eigenvectors are not unique (e.g., if v is an eigenvector, so is kv)Suppose 1, 2, ..., n are the eigenvalues of A, then:
-
Matrix diagonalizationGiven an n x n matrix A, find P such that: P-1AP= where is diagonal
Take P = [v1 v2 . . . vn], where v1,v2 ,. . . vn are the eigenvectors of A:
-
Matrix diagonalization (contd)Example:
-
Only if P-1 exists (i.e., P must have n linearly independent eigenvectors, that is, rank(P)=n)
If A is diagonalizable, then the corresponding eigenvectors v1,v2 ,. . . vn form a basis in Rn
Are all n x n matrices diagonalizable P-1AP ?
-
Matrix decompositionLet us assume that A is diagonalizable, then A can be decomposed as follows:
-
Special case: symmetric matrices
The eigenvalues of a symmetric matrix are real and its eigenvectors are orthogonal.P-1=PTA=PDPT=
Pattern Recognition*George Bebis***************************