Lec 04 Linear Algebra
description
Transcript of Lec 04 Linear Algebra
-
Linear Algebra topicsL. Lanari
Control Systems
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 2
Matrices
M =
m11 m12 m1qm21 m22 m2q...
... ...
mp1 mp,q1 mpq
M = {mij : i = 1, . . . , p & j = 1, . . . , q}
MT =mij : m
ij = mji
1 1
v : n 1vT : 1 n
M : p p
M : p q (p = q)
transpose
vector (column)
row vector
scalar
square matrix
rectangular matrix
Terminologyand notation
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 3
Matrices
m11 m12 m1n0 m22
. . . m2n
0. . .
. . ....
0 0 mnn
M1 0 00 M2 00 0 M3
M11 M120 M22
M = diag{Mi}, i = 1, . . . , k
m1 0 0
0 m2. . . 0
0. . .
. . . 00 0 mn
M = diag{mi}, i = 1, . . . , pdiagonal matrix
block diagonal matrix
upper triangular matrix
upper block triangular matrix
lower triangular matrix
strictly lower triangular matrix
lower block triangular matrix
strictly lower block triangular matrix
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 4
Matrices
determinant of a diagonal matrix = product of the diagonal terms
det
M11 M120 M22
= det(M11) det(M22)
det
M1 00 M2
= det(M1) det(M2) special case
special case
useful for the eigenvalue computation
a11 a1n...
. . ....
an1 ann
1 0...
. . ....
0 n
=
1 0...
. . ....
0 n
a11 a1n...
. . ....
an1 ann
a111 a1nn
.... . .
...an11 annn
=
a111 a1n1
.... . .
...an1n annn
gives
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 5
Matrices
det(M) = 0
A1
1= A
(AB)1 = B1A1
M = diag{mi} = M1 = diag{1
mi}
non-singular(invertible)square matrix
A1A = AA1 = I
A0 = I
detA= det
AT
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 6
generic transformationu
AuA
u
u
Au
Auor
particular directionssuch that Au is parallel to u
Au = u
scalar(scaling factor)
Eigenvalues & Eigenvectors
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 7
Aui = iui (iI A)ui = 0
det(iI A) = 0
pA() = det(I A) = n + an1n1 + an2n2 + + a1+ a0
i
pA() = det(I A) = 0
eigenvalue
for non-trivial solution
characteristic polynomial (order n = dim(A))
eigenvalue solution of
Eigenvalues & Eigenvectors
(scaling)
i
eigenvector ui
ui = 0
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 8
pA() = det(I A) = n + an1n1 + an2n2 + + a1+ a0
ii R
i C i
ma(i)
(i,i )
generic eigenvalue
polynomial with real coefficients.
algebraicmultiplicity
Eigenvalues & Eigenvectors
i Ri C
ui
uii
real components
ui complex components
pA() = 0
also is a solution pairs
therefore
if then
ieigenvalue pA() = 0solution of with multiplicity
The set of the n solutions of is defined as the spectrum of A: (A)
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 9
Eigenvalues & Eigenvectors
special cases
m11 m12 m1n0 m22
. . . m2n
0. . .
. . ....
0 0 mnn
m1 0 0
0 m2. . . 0
0. . .
. . . 00 0 mn
diagonalmatrix
triangularmatrix
eigenvalues = {mi}
eigenvalues = {mii}
elements on themain diagonal
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 10
A TAT1
Aui = iui uivTi A = v
Ti i = iv
Ti vTi
A(ui) = Aui = iui = i(ui)
Eigenvalues & Eigenvectors
det(T ) = 0
Tsame eigenvalues as A
right eigenvector
left eigenvector
eigenvalues are invariant under similarity transformations
eigenvector associated to i is not unique
all belong to the same linear subspace
(proof)
vTi uj = ijwith
similarity transformations
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 11
Eigenvalues & Eigenvectors
i eigenvalue one or more linearly independent eigenvectors
A1 =
1 00 1
, u11 =
10
, u12 =
01
A2 =
1 0 1
, with = 0, only u1 =
10
ma(i) > 1
pA() = ( 1)2
Vi = {u Rn|Au = iu}
Linear subspace (eigenspace) associated to an eigenvalue i
dim(Vi) = mg(i)
mg(i) = dim [Ker(A iI)] = n rank(A iI)
geometric multiplicity
with
A1 and A2 same eigenvalues with same m.a.
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 12
1 mg(i) ma(i) n
Eigenvalues & Eigenvectors
A1 =
1 00 1
, (A1 1I) =
0 00 0
mg(1) = 2 = ma(1)
A2 =
1 0 1
, (A2 1I) =
0 0 0
mg(1) = 1 < ma(1)
useful property
45
u1
u2
u3
P =
0.5 0 0.50 1 00.5 0 0.5
u1 =
010
u2 =
101
u3 =
101
1 = 2 = 1
3 = 0
projection matrix
ma(1) = 2 = mg(1)
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 13
Diagonalization
Def. An (n x n) matrix A is said to be diagonalizable if there exists an invertible (n x n) matrix T such that TAT -1 is a diagonal matrix
Th. An (n x n) matrix A is diagonalizable if and only if it has n independent eigenvectors
Since the eigenvalues are invariant under similarity transformations
if A diagonalizable
TAT1 = = diag{i}, i = 1, . . . , n
eigenvalues of A
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 14
Diagonalization
Aui = iui, i = 1, . . . , n
Au1 u2 un
=u1 u2 un
1 0 00 2 0
. . .0 0 n
We need to find T
i ui n linearly independent(by hyp.)
non-singular
in matrix form
AU = U
U =u1 u2 un
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 15
AU = U
AT1 = T1 A = T1T = TAT1
Diagonalization
Distinct eigenvalues (real and/or complex) A diagonalizable==
T1 = Ubeing non-singular, we can define T such thatU
therefore the diagonalizing similarity transformation is T s.t.
T1 = U =u1 u2 un
A: (n x n) is diagonalizable if and only if
mg(i) = ma(i) for every i
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 16
Diagonalization
Complex eigenvalues?i = i + ji
ui = uai + jubi
eigenvalue
eigenvector
diagonalization
or real block 2 x 2
complexelements
realelements
(i,i )
T1 =ui ui
Di = TAT1 =
i 00 i
T1 =uai ubi
Mi = TAT1 =
i ii i
2 choices
real system representation for complex eigenvalues
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 17
r = diag{1, . . . ,r}
Diagonalization
Simultaneous presence of real and complex eigenvalues
If A diagonalizable, there exists a non-singular matrix R such that
real eigenvalues
complex eigenvalues
RAR1 = diag {r,Mr+1,Mr+3, . . . ,Mq1}
Mi = TAT1 =
i ii i
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 18
Spectral decomposition or Eigendecomposition
A = U U1
U1 =
vT1vT2...vTn
U1U = I vTi uj = ij , i, j = 1, . . . , n
A =n
i=1
i ui vTi
Hyp: A diagonalizable
U =u1 u2 un
columns (linearly independent)
rows
A = U U1 =1u1 2u2 nun
vT1vT2...vTn
spectral form of A
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 19
A =n
i=1
i ui vTi
Spectral decomposition
columnrow
is the projection matrix on the invariant subspace generated by ui
(n x n) Pi = uivTi
A =
2 10 1
u1 =
10
u2 =
11
vT1 =1 1
vT2 =
0 1
P1 =
1 10 0
P2 =
0 10 1
1 = 2
2 = 1
u1
u2
P1 v
P2 v
v
v =
21
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 20
Not diagonalizable case
mg(i) < ma(i)
mg(i) nki
Jk =
i 1 0 00 i 1 0...
. . .. . .
. . ....
0 0 i 10 0 i
Rnknk
blocks of dimension
Jordanblock
of dimension nk
Null space has dimension 1Jk iI
1 mg(i) ma(i) nSince in general
if A not diagonalizable then
the knowledge of this dimension is out of scope
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 21
Not diagonalizable case
Generalized eigenvector chain of nk generalized eigenvectors
Jordan canonical form (block diagonal)
ma(i) = n =p
i=1
nk
mg(i) = p
example: unique eigenvalue i of matrix A (n x n)
TAT1 = J =
J1
. . .Jp
Jk Rnknk
T :
Jordan block of dim nkwith
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 22
Special cases
if ma(i) = 1 then mg(i) = 1
if mg(i) = 1 then only one Jordan block of dimension ma(i)
then only one Jordan block of dimension ma(i)
1 10 1
1 = 1 ma(1) = 2 mg(1) = 1
From the rank-nullity theorem
rank(A - iI)nullity(A - iI)n
if i unique eigenvalue of matrix A and if rank(iI - A) = ma(i) - 1
dim (Rn) = dim (Ker(A iI)) + dim (Im(A iI))
A iI : Rn Rn
Wednesday, October 8, 2014
-
Lanari: CS - Linear Algebra 23
Summary
A diagonalizable
A not diagonalizable
= diag{i} T s.t. TAT1 =
mg(i) = ma(i) for all i
T s.t.TAT1 = diag{Jk}
A =n
i=1
i ui vTi
r = diag{1, . . . ,r}
Mi =
i ii i
for real & complex i
alternative choice for complex i
Jk =
i 1 0...
. . .. . .
...0 i 10 0 i
Rnknk
Jordan blocks
mg(i) < ma(i)
spectral form
block diagonal
Wednesday, October 8, 2014