Chapter 5

Orthogonality

1 The scalar product in Rn

The product xTy is called the scalar product of x and y.

In particular, if x=(x1, …, xn)T and y=(y1, …,yn)T, then

xTy=x1y1+x2y2+ +‥‥ xnyn

The Scalar Product in R2 and R3

Definition

Let x and y be vectors in either R2 or R3. The distance bet

ween x and y is defined to be the number ‖x-y‖.

Example If x=(3, 4)T and y=(-1, 7)T, then the distance

between x and y is given by

‖y-x‖= 5

Theorem 5.1.1 If x and y are two nonzero vectors in either

R2 or R3 and θ is the angle between them, then

(1) xTy=‖x‖‖y‖cosθ

Corollary 5.1.2 ( Cauchy-Schwarz Inequality)

If x and y are vectors in either R2 or R3 , then

(2) ︱ xTy︱≤‖x‖‖y‖

with equality holding if and only if one of the vectors is 0 or one

vector is a multiple of the other.

Definition

The vector x and y in R2 (or R3) are said to be orthogonal if

xTy=0.

Example

(a) The vector 0 is orthogonal to every vector in R2.

(b) The vectors and are orthogonal in R2.

(c) The vectors and are orthogonal in R3.

2

3

6

4

1

3

2

1

1

1

Scalar and Vector Projections

x z=x-py

u

p=αuθ

y

yx

y

cosyxcosx

T

The scalar is called the scalar projection of x and y, and

the vector p is called the vector projection of x and y.

Scalar projection of x onto y:

y

yxT

Vector projection of x onto y:

yyy

yxy

y

1up

T

T

Example The point Q is the point on the line that is

closet to the point (1, 4). Determine the coordinates of Q.

xy3

1

xy3

1

(1, 4)

v

Qw

Orthogonality in Rn

The vectors x and y are said to be orthogonal if xTy=0.

2 Orthogonal Subspaces

Definition

Two subspaces X and Y of Rn are said to be orthogonal if

xTy=0 for every x∈X and every y∈Y. If X and Y are orthogon

al, we write X⊥Y.

Example Let X be the subspace of R3 spanned by e1, and

let Y be the subspace spanned by e2.

Example Let X be the subspace of R3 spanned by e1 and e2,

and let Y be the subspace spanned by e3.

Definition

Let Y be a subspace of Rn . The set of all vectors in Rn that a

re orthogonal to every vector in Y will be denoted Y⊥. Thus

Y⊥={ x∈Rn︱ xTy=0 for every y∈Y }

The set Y⊥ is called the orthogonal complement of Y.

Remarks

1. If X and Y are orthogonal subspaces of Rn, then X∩Y={0}.

2. If Y is a subspace of Rn, then Y⊥ is also a subspace of Rn.

Fundamental Subspaces

Theorem 5.2.1 ( Fundamental Subspaces Theorem)

If A is an m×n matrix, then N(A)=R(AT) ⊥ and N(AT)=R(A) ⊥.

Theorem 5.2.2 If S is a subspace of Rn, then

dim S+dim S⊥=n. Furthermore, if {x1, …, xr} is a basis for S and

{xr+1, …, xn} is a basis for S⊥, then {x1, …, xr, xr+1, …, xn}

is a basis for Rn.

Definition

If U and V are subspaces of a vector space W and each w∈W can be written uniquely as a sum u+v, where u∈U and v

∈V, then we say that W is a direct sum of U and V, and we w

rite W=U V.

Theorem 5.2.3 If S is a subspace of Rn, then Rn=S S⊥.

Theorem 5.2.4 If S is a subspace of Rn, then (S⊥) ⊥=S.

Theorem 5.2.5 If A is an m×n matrix and b∈Rm, then

either there is a vector x∈Rn such that Ax=b or there is a

vector y∈Rm such that ATy=0 and yTb≠0.

Example Let

431

110

211

A

Find the bases for N(A), R(AT), N(AT), and R(A).

4 Inner Product Spaces

Definition

An inner product on a vector space V is an operation on V th

at assigns to each pair of vectors x and y in V a real number

<x, y> satisfying the following conditions:

Ⅰ. <x, x>≥0 with equality if and only if x=0.

Ⅱ. <x, y>=<y, x> for all x and y in V.

Ⅲ. <αx+βy, z>=α<x, z>+β<y, z> for all x, y, z in V and all sc

alars α and β.

The Vector Space Rm×n

Given A and B in Rm×n, we can define an inner product by

m

i

n

jijijbaBA

1 1

,

Basic Properties of Inner product Spaces

Theorem 5.4.1 ( The Pythagorean Law )

If u and v are orthogonal vectors in an inner product space V,

then

222vuvu

If v is a vector in an inner product space V, the length or norm

of v is given byvv,v

Example If

33

21

11

A and

43

03

11

B

then 6, BA

5A

6B

Definition

If u and v are vectors in an inner product space V and v≠0, th

en the scalar projection of u onto v is given by

v

vu，

and the vector projection of u onto v is given by

vvv,

vu,v

v

1p

Theorem 5.4.2 ( The Cauchy- Schwarz Inequality)

If u and v are any two vectors in an inner product space V, then

vuvu,

Equality holds if and only if u and v are linearly dependent.

5 Orthonormal Sets

Definition

Let v1, v2, …, vn be nonzero vectors in an inner product space

V. If <vi, vj>=0 whenever i≠j, then { v1, v2, …, vn} is said to be

an orthogonal set of vectors.

Example The set {(1, 1, 1)T, (2, 1, -3)T, (4, -5, 1)T} is an

orthogonal set in R3.

Theorem 5.5.1 If { v1, v2, …, vn} is an orthogonal set of

nonzero vectors in an inner product space V, then v1, v2, …,vn

are linearly independent.

Definition

An orthonormal set of vectors is an orthogonal set of unit vect

ors.

i

n

iic uv

1

Theorem 5.5.2 Let { u1, u2, …, un} be an orthonoemal basis

for an inner product space V. If , then ci=<v, ui>.

The set {u1, u2, …, un} will be orthonormal if and only if

ijji u,u

where

jiif

jiifij 0

1

Corollary 5.5.3 Let { u1, u2, …, un} be an orthonoemal basis

for an inner product space V. If and , theni

n

iia uu

1

i

n

iib uv

1

i

n

iiba

1

vu,

Corollary 5.5.4 If { u1, u2, …, un} is an orthonoemal basis

for an inner product space V and , theni

n

iic uv

1

n

iic

1

22v

Orthogonal MatricesDefinition

An n×n matrix Q is said to be an orthogonal matrix if the colu

mn vectors of Q form an orthonormal set in Rn.

Theorem 5.5.5 An n×n matrix Q is orthogonal if and only if

QTQ=I.

Example For any fixed , the matrix

cossin

sincosQ

is orthogonal.

Properties of Orthogonal Matrices

If Q is an n×n orthogonal matrix, then

(a) The column vectors of Q form an orthonormal basis for Rn.

(b) QTQ=I

(c) QT=Q-1

(d) det(Q)=1 or -1

(e) The thanspose of an orthogonal matrix is an orthogonal

matrix.

(f) The product of two orthogonal matrices is also an orthogonal

matrix.

6 The Gram-Schmidt Orthogonalization Process

Theorem 5.6.1 ( The Gram-Schmidt Process)

Let {x1, x2, …, xn} be a basis for the inner product space V. Let

1

11 x

x

1u

and define u2, …, un recursively by

)px(px

1u 1

11 kk

kkk

for k=1, …, n-1

where

pk=<xk+1, u1>u1+<xk+1, u2>+ <‥‥ xk+1, uk>uk

is the projection of xk+1 onto Span(u1, u2, …, uk). The set

{u1, u2, …, un}

is an orthonormal basis for V.

Example Let

011

241

241

411

A

Find an orthonormal basis for the column space of A.