Session3:Therank · 2020. 12. 14. · = 1123 * ImCA' ' 1=1123. Therank-nullityTheorem 6/20...

Session3: The rankOptimization and Computational Linear Algebra for Data Science

Léo Miolane

Contents

1. The rank2. The rank-nullity Theorem3. More on the inverse of a matrix4. Transpose of a matrix5. Why do we care about all these things ?

Is the rank useful in practice?

-

The rank 1/20

The rank

Recap of the videos

The rank 2/20

DefinitionWe define the rank of a family x1, . . . , xk of vectors ofRn as thedimension of its span:

rank(x1, . . . , xk) def= dim(Span(x1, . . . , xk)).

DefinitionLetM œ Rn◊m. Let c1, . . . , cm œ Rn be its columns. We define

rank(M) def= rank(c1, . . . , cm) = dim(Im(M)).

PropositionLetM œ Rn◊m. Let r1, . . . , rn œ Rm be the rows ofM andc1, . . . , cm œ Rn be its columns. Then we have

rank(r1, . . . , rn) = rank(c1, . . . , cm) = rank(M).

co s

-

Howdowe compute the rank ?

The rank 3/20

For v1, . . . , vk œ Rn, and – œ R \ {0}, — œ Rwe have

rank(v1, . . . , vk) =

Y__]

__[

rank(v1, . . . , vi≠1, –vi , vi+1, . . . , vk)

rank(v1, . . . , vi≠1, vi + —vj , vi+1, . . . , vk)

As a consequence, the Gaussian elimination method keeps the rankof a matrix unchanged!

O

>multTpyFe vector by ato

--

> replace oi by vi t poj for somejtI@

Example

The rank 4/20

Let’s compute the rank of A =

Q

ca1 ≠1 0 12 0 1 ≠1

≠1 5 2 0

R

db← r

.

ay⇐ is

to E 's ) Es -IB here :

° k$ 2 1 Rst Rsranka)=rauhlA')

A"

-

- f ftp.z)rauhta'' ) -

-ranket)

-308800 ⑦ - Rz - 2KO O O Eg %tmImA"=dimImA

Example

The rank 5/20

Cr Cz Cy

claim : rank =3 A

"=(1qO-§Efz )claim : G , ez, Ca are o@linearly independent " leading coefficients

"

Let a,b , c ER such that a-Gtbcztcq =o

-0=0ya ÷÷÷⇐Y÷:-

Hence a,cz, ca are lin indep .Spencer , ez, cu) = 1123 *

ImCA'' 1=1123

The rank-nullity Theorem 6/20

The rank-nullityTheorem

Rank-nullity Theorem


TheoremLetL : Rm æ Rn be a linear transformation. Then

rank(L) + dim(Ker(L)) = m.

Oy -

O

dimitrios) ( dim( Rm)

Very usefall to get dim keel fromrankled and vice-versa !

Intuition


Let us solve the linear systemAx = 0.Q

ca1 ≠1 0 12 0 1 ≠1

≠1 5 2 0

-------

000

R

db

Q

ca1 ≠1 0 10 2 1 ≠30 0 0 7

-------

000

R

db(R1)(R2)(R3) ≠ 2(R2)

Q

ca1 ≠1 0 10 2 1 ≠30 4 2 1

-------

000

R

db(R1)(R2) ≠ 2(R1)(R3) + (R1)

→

.

ten-

e::

spent E)

In:÷÷÷÷÷÷÷÷÷÷÷÷¥¥¥E.

Proof of the rank-nullity Theorem


• Let h = dim Keech) and 61, .. . va) lose a basis

of Kau) .

• I can add vectors Nate , . - van to it to obtain

a basis (y, - . - um) of Rm this follows from P. 1.3of AWI

Can:3) "23

y

Spanked = Keech



Claim : ( Levens) - - - Llvm)) is a basis of Imu)-

-

Given the claim,the theorem follows .

dim Imu) = m -k = me - dim Kall)

⑦ S~a.im/=ImCL) .

• S C ImCL) because levees) - . - Kvm) E ImbB - -

and Im CL) is a subspace .



• ImCL) C Span( Kuan) - Uvm))-s

•Let y EIMCL) ,

there exist at RM

such that y=LCa) .

• Let Ca. - am) be the word. of n in Cy-um)

y =Lcn) =L( diet - - t 2.mum)

= de LCH) t . . - t da that,Kuan) t - - -thanLlvm)to Iso Eo¥

|YC③ henceImu) = Span( Kuan) - - - ileum))



② Let 's show thatILCve.D-kvmllin.indep.TWdate - - - dm EIR such that aah Huat.) t. - + am Llvm)-0

✓ = 2101 t - - tLava -

>I = LCIVatt.tv#c-LCul=o -3

u

Hence u E Kall) therefore there exists as .. - da s. t.

s#we get : auattaaraN=oCh- Vm) lin

. cindep hence are = - - - = am = O.

Invertible matrices 10/20

Invertiblematrices

Invertiblematrices


Definition (Matrix inverse)A squarematrixM œ Rn◊n is called invertible if there exists amatrixM≠1 œ Rn◊n such that

MM≠1 = M≠1M = Idn.

SuchmatrixM≠1 is unique and is called the inverse ofM .

Exercise: LetA, B œ Rn◊n. Show that ifAB = Idn thenBA = Idn.

- =

To - Z -

'

Invertiblematrices


TheoremLetM œ Rn◊n. The following points are equivalent:

1. M is invertible.2. rank(M) = n.

3. Ker(M) = {0}.4. For all y œ Rn, there exists a unique x œ Rn such thatMx = y.

-

I ⇒ dimKeith =

0--0• The rank nullity theorem gives G) ⇐s Cs)

•

as I a ⇒FET

Proof


(1) ⇒ (3) Assume M invertible.

Let NE Keim : Ma -- O

→ N'Mn = Mito =O-

Idn

x. =Idnn=0 12=07this gives /ka①=h

Proof


1%4) ⇒ cu) Let's assume that franklinkaCM)=ho#

IMI) is a subspace of RI dimer) ondimIm =n

Hence /ImCM)=Rt€•From what we have seen last week

, forevery y ER

"

,there exists ( because ImRn)

a unique ( because K¥014 such that they

Proof


(4) ⇒ ( t) Forevery #fR

" there exist

nz§ EIR" such that Macy) = y .

Z

Let's define : L : Rn → RnMOn②→g③

112"

K R"

claim : L is linear.

←

Proof


By construction of L , we have for all

y E RN M LITT = y Idn : Rn→ Rn

- y ↳y

Molly) = Ida Cy )This gives that the linear transformationsMo L and Idn are the same : Mol = Idn-

Hence their matrices are equal :"

matpifoduy.li#M=2Ifdn:M is invertible

Transpose of a matrix 14/20

Transposeof amatrix

Transpose of amatrix


DefinitionLetM œ Rn◊m. We define its transposeMT œ Rm◊n by

(MT)i,j = Mj,i

for all i œ {1, . . . , m} and j œ {1, . . . , n}.

Remark:We have (MT)T = M .The mappingM ‘æ MT is linear.

-

example : M --

f} ! ) M¥123810" the cot . of M became

the rows of MT "

- / (AtBJ = Att BT(aAf = 2 AT

Properties of the transpose


PropositionFor allA œ Rn◊m, rank(A) = rank(AT).

PropositionLetA œ Rn◊m andB œ Rm◊k. Then

(AB)T = BTAT.

Proof.

⇤

because"rank of rows"

Tp→n recife.'m' rank of ooh .

"

E I e mmLet's compute ②

(CABj= CAB)⑧,④ = Ez Aggie Bee,@= E. Cath

, ;=i.¥,

= ( BTA):. so

Symmetricmatrices


DefinitionA square matrixA œ Rn◊n is said to be symmetric if

’i, j œ {1, . . . , n}, Ai,j = Aj,i

or, equivalently ifA = AT.

Remark: For allM œ Rn◊m the matrixMMT is symmetric.

Ep(Mt = MMT : MNT is syen .

Ee : # 29%60) is symmetric

Is the rank useful in practice? 18/20

Is the rankuseful inpractice?

Back to themovies ratings example


Assume that you are given the matrix of movies ratings:Q

ccccca

1 1 5 5 52 2 2 0 01 1.001 5 5 52 2 2 0.0001 0

2.0001 2 2 0 0

R

dddddb

Goal: howmany di�erent « user profiles » do we have ?

→

uses Is

pg

rank (M) = S

Conclusion


The rank is not «robust» !

We need to have a way to check if a matrix has «approximatelya small rank».

Equivalentely, givenm vectors, one would like to be able to seeif there exists a subspace of dimension k π m fromwhich thevectors are « close ».

The singular value decomposition (lecture 6-7) will solves ourproblems !

t

Questions?

21/20

I

Y n V -- hole

Ch-na)

(Cve- Vm)

( un - - - ha ve - - - Van) tin uindep .

At Unt - - - thankt Bak t - - - t Pmvm = O

→ di = pi are all zero

Questions?

21/20

MNT is always symmetricLet's apply this to D= AT

ATIA'T is albs symmetric¥

:* ÷: ¥MTM = Iz

Questions?

21/20

MMT is 40×10 when Mu's 10×2.

MNT is not invertible ! B

rank ( MM g⑧m.in/rankM,raukMDf2⇐ 4 rank CMMD franker)

-

rank (MNT ) E ranking

Session3:Therank · 2020. 12. 14. · = 1123 * ImCA' ' 1=1123. Therank-nullityTheorem 6/20...

Documents

Transcript of Session3:Therank · 2020. 12. 14. · = 1123 * ImCA' ' 1=1123. Therank-nullityTheorem 6/20...