Eigenvalues and Eigenvectors -...
Transcript of Eigenvalues and Eigenvectors -...
![Page 1: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/1.jpg)
Eigenvalues and Eigenvectors
Hung-yi Lee
![Page 2: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/2.jpg)
Chapter 5
• In chapter 4, we already know how to consider a function from different aspects (coordinate system)
• Learn how to find a “good” coordinate system for a function
• Scope: Chapter 5.1 – 5.4
• Chapter 5.4 has *
![Page 3: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/3.jpg)
Outline
• What is Eigenvalue and Eigenvector?
• Eigen (German word): "unique to” or "belonging to"
• How to find eigenvectors (given eigenvalues)?
• Check whether a scalar is an eigenvalue
• How to find all eigenvalues?
• Reference: Textbook Chapter 5.1 and 5.2
![Page 4: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/4.jpg)
Definition
![Page 5: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/5.jpg)
Eigenvalues and Eigenvectors
• If 𝐴𝑣 = 𝜆𝑣 (𝑣 is a vector, 𝜆 is a scalar)
• 𝑣 is an eigenvector of A
• 𝜆 is an eigenvalue of A that corresponds to 𝑣
Eigen value
Eigen vector
A must be square
excluding zero vector
![Page 6: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/6.jpg)
Eigenvalues and Eigenvectors
• If 𝐴𝑣 = 𝜆𝑣 (𝑣 is a vector, 𝜆 is a scalar)
• 𝑣 is an eigenvector of A
• 𝜆 is an eigenvalue of A that corresponds to 𝑣
• T is a linear operator. If T 𝑣 = 𝜆𝑣 (𝑣 is a vector, 𝜆is a scalar)
• 𝑣 is an eigenvector of T
• 𝜆 is an eigenvalue of T that corresponds to 𝑣
excluding zero vector
excluding zero vector
![Page 7: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/7.jpg)
Eigenvalues and Eigenvectors
• Example: Shear Transform 𝑥′𝑦′
= 𝑇𝑥𝑦 =
𝑥 +𝑚𝑦𝑦
This is an eigenvector.
Its eigenvalue is 1.
(x,y) (x’,y’)
![Page 8: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/8.jpg)
Eigenvalues and Eigenvectors
• Example: Reflection
reflection operator T about the line y = (1/2)x
b1 is an eigenvector of T
b2 is an eigenvector of T
y = (1/2)x
𝑏1𝑏2
𝑇 𝑏1 = 𝑏1
𝑇 𝑏2 = −𝑏2
Its eigenvalue is 1.
Its eigenvalue is -1.
![Page 9: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/9.jpg)
Eigenvalues and Eigenvectors
• Example:
Expansion and Compression
2 00 2
0.5 00 0.5
All vectors are eigenvectors.
Eigenvalue is 2
Eigenvalue is 0.5
![Page 10: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/10.jpg)
Eigenvalues and Eigenvectors
• Example: Rotation
Do any n x n matrix or linear operator have eigenvalues?
Source of image: https://twitter.com/circleponiponi/status/1056026158083403776
![Page 11: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/11.jpg)
How to find eigenvectors (given eigenvalues)
![Page 12: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/12.jpg)
Eigenvalues and Eigenvectors
• An eigenvector of A corresponds to a unique eigenvalue.
• An eigenvalue of A has infinitely many eigenvectors.
Example:𝑣 =
100
𝑢 =01−1
−1 0 00 1 20 2 1
100
−1 0 00 1 20 2 1
01−1
=−100
=0−11
Eigenvalue= -1 Eigenvalue= -1
Do the eigenvectors correspond to the same eigenvalue form a subspace?
![Page 13: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/13.jpg)
Eigenspace
• Assume we know 𝜆 is the eigenvalue of matrix A
• Eigenvectors corresponding to 𝜆
Av = v
Av − v = 0
Av − Inv = 0
(A − In)v = 0
matrix
Eigenvectors corresponding to 𝜆are nonzero solution of
(A − In)v = 0
Eigenvectors corresponding to 𝜆
= 𝑁𝑢𝑙𝑙(A − In) − 𝟎
eigenspace
Eigenspace of 𝜆:Eigenvectors corresponding to 𝜆 + 𝟎
![Page 14: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/14.jpg)
Check whether a scalar is an eigenvalue
![Page 15: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/15.jpg)
• How to know whether a scalar is the eigenvalue of A?
Check Eigenvalues 𝑁𝑢𝑙𝑙(A − In): eigenspace of
If the dimension is 0
Eigenspace only contains {0}
No eigenvector
𝜆 is not eigenvalue
Check the dimension of eigenspace of
![Page 16: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/16.jpg)
Check Eigenvalues
• Example: to check 3 and −2 are eigenvalues of the linear operator T
𝑁𝑢𝑙𝑙(A − In): eigenspace of
𝑁𝑢𝑙𝑙(A − 3In)=? 𝑁𝑢𝑙𝑙(A + 2In)=?
11
−2−2
−23
−69
![Page 17: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/17.jpg)
Check Eigenvalues
• Example: check that 3 is an eigenvalue of B and find a basis for the corresponding eigenspace
find the solution set of (B − 3I3)x = 0
𝑁𝑢𝑙𝑙(A − In): eigenspace of
B − 3I3
find the RREF of
=
![Page 18: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/18.jpg)
Looking for Eigenvalues
![Page 19: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/19.jpg)
Looking for Eigenvalues
𝐴𝑣 = 𝑡𝑣
𝐴 − 𝑡𝐼𝑛 is not invertible
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0
A scalar 𝑡 is an eigenvalue of A
Existing 𝑣 ≠ 0 such that
𝐴𝑣 − 𝑡𝑣 = 0
𝐴 − 𝑡𝐼𝑛 𝑣 = 0
𝐴 − 𝑡𝐼𝑛 𝑣 = 0 has multiple solution
Existing 𝑣 ≠ 0 such that
Existing 𝑣 ≠ 0 such that
The columns of 𝐴 − 𝑡𝐼𝑛 are independentDependent
![Page 20: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/20.jpg)
Looking for Eigenvalues
• Example 1: Find the eigenvalues of
The eigenvalues of A are -3 or 5.
=0
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0A scalar 𝑡 is an eigenvalue of A
t = -3 or 5
![Page 21: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/21.jpg)
Looking for Eigenvalues
• Example 1: Find the eigenvalues of
The eigenvalues of A are -3 or 5.
Eigenspace of -3
Eigenspace of 5
𝐴𝑥 = −3𝑥 𝐴 + 3𝐼 𝑥 = 0
𝐴𝑥 = 5𝑥 𝐴 − 5𝐼 𝑥 = 0
find the solution
find the solution
![Page 22: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/22.jpg)
Looking for Eigenvalues
• Example 2: find the eigenvalues of linear operator
standard
matrix
𝐴 − 𝑡𝐼𝑛 =−1 − 𝑡 0 0
2 −1 − 𝑡 −10 0 −1 − 𝑡
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0A scalar 𝑡 is an eigenvalue of A
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = −1 − 𝑡 3
![Page 23: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/23.jpg)
Looking for Eigenvalues
• Example 3: linear operator on R2 that rotates a vector by 90◦
standard matrix of the 90◦-rotation:
No eigenvalues, no eigenvectors
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0A scalar 𝑡 is an eigenvalue of A
![Page 24: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/24.jpg)
Characteristic Polynomial
Eigenvalues are the roots of characteristic polynomial or solutions of characteristic equation.
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 : Characteristic polynomial of A
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0: Characteristic equation of A
A is the standard matrix of linear operator T
linear operator T
linear operator T
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0A scalar 𝑡 is an eigenvalue of A
![Page 25: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/25.jpg)
Characteristic Polynomial
• In general, a matrix A and RREF of A have different characteristic polynomials.
• Similar matrices have the same characteristic polynomials
Different Eigenvalues
𝑑𝑒𝑡 𝐵 − 𝑡𝐼
= 𝑑𝑒𝑡 𝐴 − 𝑡𝐼
𝐵 = 𝑃−1𝐴𝑃= 𝑑𝑒𝑡 𝑃−1𝐴𝑃 − 𝑃−1 𝑡𝐼 𝑃
= 𝑑𝑒𝑡 𝑃−1 𝑑𝑒𝑡 𝐴 − 𝑡𝐼 𝑑𝑒𝑡 𝑃
= 𝑑𝑒𝑡 𝑃−1 𝐴 − 𝑡𝐼 𝑃
=1
𝑑𝑒𝑡 𝑃𝑑𝑒𝑡 𝐴 − 𝑡𝐼 𝑑𝑒𝑡 𝑃
The same Eigenvalues
![Page 26: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/26.jpg)
Characteristic Polynomial
• Question: What is the order of the characteristic polynomial of an nn matrix A?
• The characteristic polynomial of an nn matrix is indeed a polynomial with degree n
• Consider det(A − tIn)
• Question: What is the number of eigenvalues of an nnmatrix A?
• Fact: An n x n matrix A have less than or equal to n eigenvalues
• Consider complex roots and multiple roots
![Page 27: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/27.jpg)
Characteristic Polynomial v.s. Eigenspace• Characteristic polynomial of A is
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛
𝜆1 𝜆2
𝑑𝑘
= 𝑡 − 𝜆1𝑚1 𝑡 − 𝜆2
𝑚2 … 𝑡 − 𝜆𝑘𝑚𝑘 ……
Factorization
Eigenvalue:
Eigenspace:
(dimension)
𝑑1 𝑑2
𝜆𝑘
multiplicity
≤ 𝑚1 ≤ 𝑚2 ≤ 𝑚𝑘
![Page 28: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/28.jpg)
Characteristic Polynomial
• The eigenvalues of an upper triangular matrix are its diagonal entries.
The determinant of an upper triangular matrix is the product of its diagonal entries.
𝑎 ∗ ∗0 𝑏 ∗0 0 𝑐
Characteristic Polynomial:
𝑑𝑒𝑡𝑎 − 𝑡 ∗ ∗0 𝑏 − 𝑡 ∗0 0 𝑐 − 𝑡
= 𝑎 − 𝑡 𝑏 − 𝑡 𝑐 − 𝑡
![Page 29: Eigenvalues and Eigenvectors - 國立臺灣大學speech.ee.ntu.edu.tw/~tlkagk/courses/LA_2019/Lecture/eigen.pdf · Eigenvalues and Eigenvectors •If =𝜆 ( is a vector, 𝜆is a](https://reader035.fdocuments.us/reader035/viewer/2022071218/604f1d036e53e776030b80f5/html5/thumbnails/29.jpg)
Summary
• If 𝐴𝑣 = 𝜆𝑣 (𝑣 is a vector, 𝜆 is a scalar)
• 𝑣 is an eigenvector of A
• 𝜆 is an eigenvalue of A that corresponds to 𝑣
• Eigenvectors corresponding to 𝜆 are nonzerosolution of (A − In)v = 0
• A scalar 𝑡 is an eigenvalue of A
excluding zero vector
Eigenvectors corresponding to 𝜆
= 𝑁𝑢𝑙𝑙(A − In) − 𝟎
eigenspace
Eigenspace of 𝜆:
Eigenvectors corresponding to 𝜆 + 𝟎
𝑑𝑒𝑡 𝐴 − 𝑡𝐼𝑛 = 0