CSci 6971: Image Registration Lecture 2: Vectors and Matrices
January 16, 2004
CSci 6971: Image Registration Lecture 2: Vectors and Matrices
January 16, 2004
Prof. Chuck Stewart, RPIDr. Luis Ibanez, KitwareProf. Chuck Stewart, RPIDr. Luis Ibanez, Kitware
Image Registration Lecture 2 2
Lecture OverviewLecture Overview
Vectors Matrices
Basics Orthogonal matrices Singular Value Decomposition (SVD)
Vectors Matrices
Basics Orthogonal matrices Singular Value Decomposition (SVD)
Image Registration Lecture 2 3
Preliminary CommentsPreliminary Comments
Some of this should be review; all of it might be review
This is really only background, and not a main focus of the course
All of the material is covered in standard linear algebra texts. I use Gilbert Strang’s Linear Algebra and
Its Applications
Some of this should be review; all of it might be review
This is really only background, and not a main focus of the course
All of the material is covered in standard linear algebra texts. I use Gilbert Strang’s Linear Algebra and
Its Applications
Image Registration Lecture 2 4
Vectors: DefinitionVectors: Definition
Formally, a vector is an element of a vector space
Informally (and somewhat incorrectly), we will use vectors to represent both point locations and directions
Algebraically, we write Note that we will usually
treat vectors column vectors and use the transpose notation to make the writing more compact
Formally, a vector is an element of a vector space
Informally (and somewhat incorrectly), we will use vectors to represent both point locations and directions
Algebraically, we write Note that we will usually
treat vectors column vectors and use the transpose notation to make the writing more compact
Image Registration Lecture 2 5
Vectors: ExampleVectors: Example
x
z
y
(-4,6,5)
(0,0,-1)
Image Registration Lecture 2 6
Vectors: AdditionVectors: Addition
Added component-wise
Example:
Added component-wise
Example:
x
z
y
p
p+qq
Geometric view
Image Registration Lecture 2 7
Vectors: Scalar MultiplicationVectors: Scalar Multiplication
Simplest form of multiplication involving vectors
In particular:
Example:
Simplest form of multiplication involving vectors
In particular:
Example:
pcp
Image Registration Lecture 2 8
Vectors: Lengths, Magnitudes, DistancesVectors: Lengths, Magnitudes, Distances
The length or magnitude of a vector is
The distance between two vectors is
The length or magnitude of a vector is
The distance between two vectors is
Image Registration Lecture 2 9
Vectors: Dot (Scalar/Inner) ProductVectors: Dot (Scalar/Inner) Product
Second means of multiplication involving vectors In particular,
We’ll see a different notation for writing the scalar product using matrix multiplication soon
Note that
Second means of multiplication involving vectors In particular,
We’ll see a different notation for writing the scalar product using matrix multiplication soon
Note that
Image Registration Lecture 2 10
Unit VectorsUnit Vectors
A unit (direction) vector is a vector whose magnitude is 1:
Typically, we will use a “hat” to denote a unit vector, e.g.:
A unit (direction) vector is a vector whose magnitude is 1:
Typically, we will use a “hat” to denote a unit vector, e.g.:
Image Registration Lecture 2 11
Angle Between VectorsAngle Between Vectors
We can compute the angle between two vectors using the scalar product:
Two non-zero vectors are orthogonal if and only if
We can compute the angle between two vectors using the scalar product:
Two non-zero vectors are orthogonal if and only if
x
z
y
pq
Image Registration Lecture 2 12
Cross (Outer) Product of Vectors Cross (Outer) Product of Vectors
Given two 3-vectors, p and q, the cross product is a vector perpendicular to both
In component form,
Finally,
Given two 3-vectors, p and q, the cross product is a vector perpendicular to both
In component form,
Finally,
Image Registration Lecture 2 13
Looking Ahead A Bit to TransformationsLooking Ahead A Bit to Transformations
Be aware that lengths and angles are preserved by only very special transformations
Therefore, in general Unit vectors will no longer be unit vectors
after applying a transformation Orthogonal vectors will no longer be
orthogonal after applying a transformation
Be aware that lengths and angles are preserved by only very special transformations
Therefore, in general Unit vectors will no longer be unit vectors
after applying a transformation Orthogonal vectors will no longer be
orthogonal after applying a transformation
Image Registration Lecture 2 14
Matrices - DefinitionMatrices - Definition
Matrices are rectangular arrays of numbers, with each number subscripted by two indices:
A short-hand notation for this is
Matrices are rectangular arrays of numbers, with each number subscripted by two indices:
A short-hand notation for this is
m rows
n columns
Image Registration Lecture 2 15
Special Matrices: The IdentitySpecial Matrices: The Identity
The identity matrix, denoted I, In or Inxn, is a square matrix with n rows and columns having 1’s on the main diagonal and 0’s everywhere else:
The identity matrix, denoted I, In or Inxn, is a square matrix with n rows and columns having 1’s on the main diagonal and 0’s everywhere else:
Image Registration Lecture 2 16
Diagonal MatricesDiagonal Matrices
A diagonal matrix is a square matrix that has 0’s everywhere except on the main diagonal.
For example:
A diagonal matrix is a square matrix that has 0’s everywhere except on the main diagonal.
For example:
Notational short-hand
Image Registration Lecture 2 17
Matrix Transpose and SymmetryMatrix Transpose and Symmetry
The transpose of a matrix is one where the rows and columns are reversed:
If A = AT then the matrix is symmetric. Only square matrices (m=n) are symmetric
The transpose of a matrix is one where the rows and columns are reversed:
If A = AT then the matrix is symmetric. Only square matrices (m=n) are symmetric
Image Registration Lecture 2 18
ExamplesExamples
This matrix is not symmetric
This matrix is symmetric
This matrix is not symmetric
This matrix is symmetric
Image Registration Lecture 2 19
Matrix AdditionMatrix Addition
Two matrices can be added if and only if (iff) they have the same number of rows and the same number of columns.
Matrices are added component-wise:
Example:
Two matrices can be added if and only if (iff) they have the same number of rows and the same number of columns.
Matrices are added component-wise:
Example:
Image Registration Lecture 2 20
Matrix Scalar MultiplicationMatrix Scalar Multiplication
Any matrix can be multiplied by a scalar Any matrix can be multiplied by a scalar
Image Registration Lecture 2 21
Matrix MultiplicationMatrix Multiplication
The product of an mxn matrix and a nxp matrix is a mxp matrix:
Entry i,j of the result matrix is the dot-product of row i of A and column j of B
Example
The product of an mxn matrix and a nxp matrix is a mxp matrix:
Entry i,j of the result matrix is the dot-product of row i of A and column j of B
Example
Image Registration Lecture 2 22
Vectors as MatricesVectors as Matrices
Vectors, which we usually write as column vectors, can be thought of as nx1 matrices
The transpose of a vector is a 1xn matrix - a row vector.
These allow us to write the scalar product as a matrix multiplication:
For example,
Vectors, which we usually write as column vectors, can be thought of as nx1 matrices
The transpose of a vector is a 1xn matrix - a row vector.
These allow us to write the scalar product as a matrix multiplication:
For example,
Image Registration Lecture 2 23
NotationNotation
We will tend to write matrices using boldface capital letters
We will tend to write vectors as boldface small letters
We will tend to write matrices using boldface capital letters
We will tend to write vectors as boldface small letters
Image Registration Lecture 2 24
Square MatricesSquare Matrices
Much of the remaining discussion will focus only on square matrices: Trace Determinant Inverse Eigenvalues Orthogonal / orthonormal matrices
When we discuss the singular value decomposition we will be back to non-square matrices
Much of the remaining discussion will focus only on square matrices: Trace Determinant Inverse Eigenvalues Orthogonal / orthonormal matrices
When we discuss the singular value decomposition we will be back to non-square matrices
Image Registration Lecture 2 25
Trace of a MatrixTrace of a Matrix
Sum of the terms on the main diagonal of a square matrix:
The trace equals the sum of the eigenvalues of the matrix.
Sum of the terms on the main diagonal of a square matrix:
The trace equals the sum of the eigenvalues of the matrix.
Image Registration Lecture 2 26
DeterminantDeterminant
Notation:
Recursive definition: When n=1,
When n=2
Notation:
Recursive definition: When n=1,
When n=2
Image Registration Lecture 2 27
Determinant (continued)Determinant (continued)
For n>2, choose any row i of A, and define Mi,j be the (n-1)x(n-1) matrix formed by deleting row i and column j of A, then
We get the same formula by choosing any column j of A and summing over the rows.
For n>2, choose any row i of A, and define Mi,j be the (n-1)x(n-1) matrix formed by deleting row i and column j of A, then
We get the same formula by choosing any column j of A and summing over the rows.
Image Registration Lecture 2 28
Some Properties of the DeterminantSome Properties of the Determinant
If any two rows or any two columns are equal, the determinant is 0
Interchanging two rows or interchanging two columns reverses the sign of the determinant
The determinant of A equals the product of the eigenvalues of A
For square matrices
If any two rows or any two columns are equal, the determinant is 0
Interchanging two rows or interchanging two columns reverses the sign of the determinant
The determinant of A equals the product of the eigenvalues of A
For square matrices
Image Registration Lecture 2 29
Matrix InverseMatrix Inverse
The inverse of a square matrix A is the unique matrix A-1 such that
Matrices that do not have an inverse are said to be non-invertible or singular
A matrix is invertible if and only if its determinant is non-zero
We will not worry about the mechanism of calculating inverses, except using the singular value decomposition
The inverse of a square matrix A is the unique matrix A-1 such that
Matrices that do not have an inverse are said to be non-invertible or singular
A matrix is invertible if and only if its determinant is non-zero
We will not worry about the mechanism of calculating inverses, except using the singular value decomposition
Image Registration Lecture 2 30
Eigenvalues and EigenvectorsEigenvalues and Eigenvectors
A scalar and a vector v are, respectively, an eigenvalue and an associated (unit) eigenvector of square matrix A if
For example, if we think of a A as a transformation and if then Av=v implies v is a “fixed-point” of the transformation.
Eigenvalues are found by solving the equation
Once eigenvalues are known, eigenvectors are found,, by finding the nullspace (we will not discuss this) of
A scalar and a vector v are, respectively, an eigenvalue and an associated (unit) eigenvector of square matrix A if
For example, if we think of a A as a transformation and if then Av=v implies v is a “fixed-point” of the transformation.
Eigenvalues are found by solving the equation
Once eigenvalues are known, eigenvectors are found,, by finding the nullspace (we will not discuss this) of
Image Registration Lecture 2 31
Eigenvalues of Symmetric MatricesEigenvalues of Symmetric Matrices
They are all real (as opposed to imaginary), which can be seen by studying the following (and remembering properties of vector magnitudes)
We can also show that eigenvectors associated with distinct eigenvalues of a symmetric matrix are orthogonal
We can therefore write a symmetric matrix (I don’t expect you to derive this) as
They are all real (as opposed to imaginary), which can be seen by studying the following (and remembering properties of vector magnitudes)
We can also show that eigenvectors associated with distinct eigenvalues of a symmetric matrix are orthogonal
We can therefore write a symmetric matrix (I don’t expect you to derive this) as
Image Registration Lecture 2 32
Orthonormal MatricesOrthonormal Matrices
A square matrix is orthonormal (sometimes called orthogonal) iff
In other word AT is the right inverse. Based on properties of inverses this immediately
implies
This means for vectors formed by any two rows or any two columns
A square matrix is orthonormal (sometimes called orthogonal) iff
In other word AT is the right inverse. Based on properties of inverses this immediately
implies
This means for vectors formed by any two rows or any two columns
Kronecker delta, which is 1 if i=j and 0 otherwise
Image Registration Lecture 2 33
Orthonormal Matrices - PropertiesOrthonormal Matrices - Properties
The determinant of an orthonormal matrix is either 1 or -1 because
Multiplying a vector by an orthonormal matrix does not change the vector’s length:
An orthonormal matrix whose determinant is 1 (-1) is called a rotation (reflection).
Of course, as discussed on the previous slide
The determinant of an orthonormal matrix is either 1 or -1 because
Multiplying a vector by an orthonormal matrix does not change the vector’s length:
An orthonormal matrix whose determinant is 1 (-1) is called a rotation (reflection).
Of course, as discussed on the previous slide
Image Registration Lecture 2 34
Singular Value Decomposition (SVD)Singular Value Decomposition (SVD)
Consider an mxn matrix, A, and assume m≥n. A can be “decomposed” into the product of 3
matrices:
Where: U is mxn with orthonormal columns W is a nxn diagonal matrix of “singular
values”, and V is nxn orthonormal matrix
If m=n then U is an orthonormal matrix
Consider an mxn matrix, A, and assume m≥n. A can be “decomposed” into the product of 3
matrices:
Where: U is mxn with orthonormal columns W is a nxn diagonal matrix of “singular
values”, and V is nxn orthonormal matrix
If m=n then U is an orthonormal matrix
Image Registration Lecture 2 35
Properties of the Singular ValuesProperties of the Singular Values
with
and the number of non-zero singular values is
equal to the rank of A
with
and the number of non-zero singular values is
equal to the rank of A
Image Registration Lecture 2 36
SVD and Matrix InversionSVD and Matrix Inversion
For a non-singular, square matrix, with
The inverse of A is
You should confirm this for yourself! Note, however, this isn’t always the best way
to compute the inverse
For a non-singular, square matrix, with
The inverse of A is
You should confirm this for yourself! Note, however, this isn’t always the best way
to compute the inverse
Image Registration Lecture 2 37
SVD and Solving Linear SystemsSVD and Solving Linear Systems
Many times problems reduce to finding the vector x that minimizes
Taking the derivative (I don’t necessarily expect that you can do this, but it isn’t hard) with respect to x, setting the result to 0 and solving implies
Computing the SVD of A (assuming it is full-rank) results in
Many times problems reduce to finding the vector x that minimizes
Taking the derivative (I don’t necessarily expect that you can do this, but it isn’t hard) with respect to x, setting the result to 0 and solving implies
Computing the SVD of A (assuming it is full-rank) results in
Image Registration Lecture 2 38
SummarySummary
Vectors Definition, addition, dot (scalar / inner)
product, length, etc. Matrices
Definition, addition, multiplication Square matrices: trace, determinant,
inverse, eigenvalues Orthonormal matrices SVD
Vectors Definition, addition, dot (scalar / inner)
product, length, etc. Matrices
Definition, addition, multiplication Square matrices: trace, determinant,
inverse, eigenvalues Orthonormal matrices SVD
Image Registration Lecture 2 39
Looking Ahead to Lecture 3Looking Ahead to Lecture 3
Images and image coordinate systems Transformations
Similarity Affine Projective
Images and image coordinate systems Transformations
Similarity Affine Projective
Image Registration Lecture 2 40
Practice ProblemsPractice Problems
A handout will be given with Lecture 3. A handout will be given with Lecture 3.
Top Related