Linear Algebra Primer - Artificial...

Linear Algebra ReviewStanford University

LinearAlgebraPrimer

JuanCarlosNiebles andRanjayKrishnaStanfordVisionandLearningLab

10/2/171

Another,veryin-depthlinearalgebrareviewfromCS229isavailablehere:http://cs229.stanford.edu/section/cs229-linalg.pdfAndavideodiscussionoflinearalgebrafromEE263ishere(lectures3and4):https://see.stanford.edu/Course/EE263

Outline• Vectorsandmatrices– BasicMatrixOperations– Determinants,norms,trace– SpecialMatrices

• TransformationMatrices– Homogeneouscoordinates– Translation

• Matrixinverse• Matrixrank• EigenvaluesandEigenvectors• MatrixCalculus

10/2/172

10/2/173

Vectorsandmatricesarejustcollectionsoforderednumbersthatrepresentsomething:movementsinspace,scalingfactors,pixelbrightness,etc.We’lldefinesomecommonusesandstandardoperationsonthem.

Vector

• Acolumnvectorwhere

• Arowvectorwhere

denotesthetransposeoperation

10/2/174

Vector• We’lldefaulttocolumnvectorsinthisclass

• You’llwanttokeeptrackoftheorientationofyourvectorswhenprogramminginpython

• YoucantransposeavectorVinpythonbywritingV.t.(Butinclassmaterials,wewillalways useVT toindicatetranspose,andwewilluseV’tomean“Vprime”)

10/2/175

Vectorshavetwomainuses

• Vectorscanrepresentanoffsetin2Dor3Dspace

• Pointsarejustvectorsfromtheorigin

10/2/176

• Data(pixels,gradientsatanimagekeypoint,etc)canalsobetreatedasavector

• Suchvectorsdon’thaveageometricinterpretation,butcalculationslike“distance”canstillhavevalue

Matrix

• Amatrixisanarrayofnumberswithsizeby,i.e.mrowsandncolumns.

• If,wesaythatissquare.

10/2/177

Images

10/2/178

• Pythonrepresentsanimageasamatrixofpixelbrightnesses

• Notethattheupperleftcorneris[y,x]=(0,0)

ColorImages• Grayscaleimageshaveonenumberperpixel,andarestoredasanm× nmatrix.

• Colorimageshave3numbersperpixel– red,green,andbluebrightnesses (RGB)

• Storedasanm× n× 3matrix

10/2/179

BasicMatrixOperations• Wewilldiscuss:– Addition– Scaling– Dotproduct–Multiplication– Transpose– Inverse/pseudoinverse– Determinant/trace

10/2/1710

MatrixOperations• Addition

– Canonlyaddamatrixwithmatchingdimensions,orascalar.

• Scaling

10/2/1711

• Norm• Moreformally,anormisanyfunctionthatsatisfies4properties:

• Non-negativity: Forall• Definiteness: f(x)=0ifandonlyifx=0.• Homogeneity: Forall• Triangleinequality: Forall

10/2/1712

Vectors

• ExampleNorms

• Generalnorms:

10/2/1713

MatrixOperations

MatrixOperations• Innerproduct(dotproduct)ofvectors–Multiplycorrespondingentriesoftwovectorsandadduptheresult

– x·y isalso|x||y|Cos(theanglebetweenxandy)

10/2/1714

MatrixOperations

• Innerproduct(dotproduct)ofvectors– IfBisaunitvector,thenA·BgivesthelengthofAwhichliesinthedirectionofB

10/2/1715

MatrixOperations• Theproductoftwomatrices

10/2/1716

MatrixOperations• Multiplication

• TheproductABis:

• Eachentryintheresultis(thatrowofA)dotproductwith(thatcolumnofB)

• Manyuses,whichwillbecoveredlater

10/2/1717

MatrixOperations• Multiplicationexample:

10/2/1718

– Eachentryofthematrixproductismadebytakingthedotproductofthecorrespondingrowintheleftmatrix,withthecorrespondingcolumnintherightone.

MatrixOperations• Theproductoftwomatrices

10/2/1719

MatrixOperations

• Powers– Byconvention,wecanrefertothematrixproductAAasA2,andAAAasA3,etc.

– Obviouslyonlysquarematricescanbemultipliedthatway

10/2/1720

MatrixOperations• Transpose– flipmatrix,sorow1becomescolumn1

• Ausefulidentity:

10/2/1721

• Determinant– returnsascalar– Representsarea(orvolume)oftheparallelogramdescribedbythevectorsintherowsofthematrix

– For,– Properties:

10/2/1722

MatrixOperations

• Trace

– Invarianttoalotoftransformations,soit’susedsometimesinproofs.(Rarelyinthisclassthough.)

– Properties:

10/2/1723

MatrixOperations

• VectorNorms

• Matrixnorms:Normscanalsobedefinedformatrices,suchastheFrobenius norm:

10/2/1724

MatrixOperations

SpecialMatrices• IdentitymatrixI– Squarematrix,1’salongdiagonal,0’selsewhere

– I · [anothermatrix]=[thatmatrix]

• Diagonalmatrix– Squarematrixwithnumbersalongdiagonal,0’selsewhere

– Adiagonal· [anothermatrix]scalestherowsofthatmatrix

10/2/1725

SpecialMatrices

• Symmetricmatrix

• Skew-symmetricmatrix

10/2/1726

40 �2 �52 0 �75 7 0

LinearAlgebraPrimer

JuanCarlosNiebles andRanjayKrishnaStanfordVisionandLearningLab

10/2/1727

Another,veryin-depthlinearalgebrareviewfromCS229isavailablehere:http://cs229.stanford.edu/section/cs229-linalg.pdfAndavideodiscussionoflinearalgebrafromEE263ishere(lectures3and4):https://see.stanford.edu/Course/EE263

Announcements– part1

• HW0submittedlastnight• HW1isduenextMonday• HW2willbereleasedtonight• ClassnotesfromlastThursdayduebeforeclassinexactly48hours

10/2/1728

Announcements– part2

• Futurehomeworkassignmentswillbereleasedviagithub–WillallowyoutokeeptrackofchangesIFtheyhappen.

• SubmissionsforHW1onwardswillbedoneallthroughgradescope.– NOMORECORNSUBMISSIONS– Youwillhaveseparatesubmissionsfortheipythonpdfandthepythoncode.

10/2/1729

Recap- Vector

• Acolumnvectorwhere

• Arowvectorwhere

denotesthetransposeoperation

10/2/1730

Recap- Matrix

• Amatrixisanarrayofnumberswithsizeby,i.e.mrowsandncolumns.

• If,wesaythatissquare.

10/2/1731

Recap- ColorImages• Grayscaleimageshaveonenumberperpixel,andarestoredasanm× nmatrix.

• Colorimageshave3numbersperpixel– red,green,andbluebrightnesses (RGB)

• Storedasanm× n× 3matrix

10/2/1732

• Norm• Moreformally,anormisanyfunctionthatsatisfies4properties:

• Non-negativity: Forall• Definiteness: f(x)=0ifandonlyifx=0.• Homogeneity: Forall• Triangleinequality: Forall

10/2/1733

Recap- Vectors

Recap– projection

• Innerproduct(dotproduct)ofvectors– IfBisaunitvector,thenA·BgivesthelengthofAwhichliesinthedirectionofB

10/2/1734

10/2/1735

Matrixmultiplicationcanbeusedtotransformvectors.Amatrixusedinthiswayiscalledatransformationmatrix.

Transformation

• Matricescanbeusedtotransformvectorsinusefulways,throughmultiplication:x’=Ax

• Simplestisscaling:

(Verifytoyourselfthatthematrixmultiplicationworksoutthisway)

10/2/1736

Rotation• Howcanyouconvertavectorrepresentedinframe“0”toanew,rotatedcoordinateframe“1”?

10/2/1737

Rotation• Howcanyouconvertavectorrepresentedinframe“0”toanew,rotatedcoordinateframe“1”?

• Rememberwhatavectoris:[componentindirectionoftheframe’sxaxis,componentindirectionofyaxis]

10/2/1738

Rotation• Sotorotateitwemustproducethisvector:[componentindirectionofnew xaxis,componentindirectionofnew yaxis]

• Wecandothiseasilywithdotproducts!• Newxcoordinateis[originalvector]dot [thenewxaxis]• Newycoordinateis[originalvector]dot [thenewyaxis]

10/2/1739

Rotation• Insight:thisiswhathappensinamatrix*vectormultiplication– Resultxcoordinateis:[originalvector]dot [matrixrow1]

– Somatrixmultiplicationcanrotateavectorp:

10/2/1740

Rotation• Supposeweexpressapointinthenewcoordinatesystemwhichisrotatedleft

• Ifweplottheresultintheoriginal coordinatesystem,wehaverotatedthepointright

10/2/1741

– Thus,rotationmatricescanbeusedtorotatevectors.We’llusuallythinkoftheminthatsense-- asoperatorstorotatevectors

2DRotationMatrixFormulaCounter-clockwise rotation by an angle q

ùêë

éúû

ùêë

é -=ú

ùêë

cossinsincos

y’P’

yθsinxθcos'x -=xθsinyθcos'y +=

10/2/1742

TransformationMatrices• Multipletransformationmatricescanbeusedtotransformapoint:p’=R2R1Sp

10/2/1743

• Theeffectofthisistoapplytheirtransformationsoneaftertheother,fromrighttoleft.

• Intheexampleabove,theresultis(R2(R1(Sp)))

10/2/1744

• Theeffectofthisistoapplytheirtransformationsoneaftertheother,fromrighttoleft.

• Intheexampleabove,theresultis(R2(R1(Sp)))

• Theresultisexactlythesameifwemultiplythematricesfirst,toformasingletransformationmatrix:p’=(R2R1S)p

10/2/1745

Homogeneoussystem

• Ingeneral,amatrixmultiplicationletsuslinearlycombinecomponentsofavector

– Thisissufficientforscale,rotate,skewtransformations.

– Butnotice,wecan’taddaconstant!L

10/2/1746

Homogeneoussystem

– The(somewhathacky)solution?Sticka“1”attheendofeveryvector:

– Nowwecanrotate,scale,andskewlikebefore,ANDtranslate (notehowthemultiplicationworksout,above)

– Thisiscalled“homogeneouscoordinates”

10/2/1747

Homogeneoussystem– Inhomogeneouscoordinates,themultiplicationworksoutsotherightmostcolumnofthematrixisavectorthatgetsadded.

– Generally,ahomogeneoustransformationmatrixwillhaveabottomrowof[001],sothattheresulthasa“1”atthebottomtoo.

10/2/1748

Homogeneoussystem• Onemorethingwemightwant:todividetheresultbysomething– Forexample,wemaywanttodividebyacoordinate,tomakethingsscaledownastheygetfartherawayinacameraimage

– Matrixmultiplicationcan’tactuallydivide– So,byconvention,inhomogeneouscoordinates,we’lldividetheresultbyitslastcoordinateafterdoingamatrixmultiplication

10/2/1749

2DTranslation

10/2/1750

Linear Algebra ReviewStanford University 10/2/1751

2DTranslationusingHomogeneousCoordinates

úúú

êêê

é×úúú

êêê

úúú

êêê

®1100

)1,,(),()1,,(),(

yxyx ttttyxyx

®=®=

úúú

êêê

é×úúú

êêê

úúú

êêê

®1100

)1,,(),()1,,(),(

yxyx ttttyxyx

®=®=

úúú

êêê

é×úúú

êêê

úúú

êêê

®1100

)1,,(),()1,,(),(

yxyx ttttyxyx

®=®=

úúú

êêê

é×úúú

êêê

úúú

êêê

®1100

)1,,(),()1,,(),(

yxyx ttttyxyx

®=®=

úúú

êêê

é×úúú

êêê

úúú

êêê

®1100

)1,,(),()1,,(),(

yxyx ttttyxyx

®=®=

×=×úû

ùêë

Scaling

10/2/1756

ScalingEquation

P’sy y

)1,,(),(')1,,(),(

ysxsysxsyxyx

yxyx ®=®=

)ys,xs(')y,x( yx=®= PP

10/2/1757

ScalingEquation

P’sy y

úúú

êêê

é×úúú

êêê

úúú

êêê

11000000

)1,,(),(')1,,(),(

ysxsysxsyxyx

yxyx ®=®=

10/2/1758

ScalingEquation

P’sy y

úúú

êêê

é×úúú

êêê

úúú

êêê

11000000

)1,,(),(')1,,(),(

ysxsysxsyxyx

yxyx ®=®=

PSP100S

×=×úû

ùêë

10/2/1759

P’=S∙PP’’=T∙P’

P’’=T·P’=T·(S·P)=T·S·P

Scaling&Translating

P’’

10/2/1760

Scaling&Translating

P '' = T ⋅S ⋅P =1 0 tx0 1 ty0 0 1

sx 0 00 sy 0

sx 0 tx0 sy ty0 0 1

sxx + txsyy+ ty1

= S t0 1

A10/2/1761

Scaling&Translating

P '' = T ⋅S ⋅P =1 0 tx0 1 ty0 0 1

sx 0 00 sy 0

sx 0 tx0 sy ty0 0 1

sxx + txsyy+ ty1

= S t0 1

10/2/1762

Translating&Scalingversus Scaling&Translating

úúú

êêê

=úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

é=××=

1tystxs

100ts0t0s

1000s000s

100t10t01

''' yy

10/2/1763

Translating&Scaling!=Scaling&Translating

úúú

êêê

=úúú

êêê

úúú

êêê

=úúú

êêê

úúú

êêê

úúú

êêê

é=××=

1tsystsxs

100tss0ts0s

100t10t01

1000s000s

úúú

êêê

=úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

é=××=

1tystxs

100ts0t0s

1000s000s

100t10t01

''' yy

10/2/1764

Translating&Scaling!=Scaling&Translating

úúú

êêê

=úúú

êêê

úúú

êêê

=úúú

êêê

úúú

êêê

úúú

êêê

é=××=

1tsystsxs

100tss0ts0s

100t10t01

1000s000s

úúú

êêê

=úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

é=××=

1tystxs

100ts0t0s

1000s000s

100t10t01

''' yy

10/2/1765

Rotation

10/2/1766

RotationEquationsCounter-clockwise rotation by an angle q

ùêë

éúû

ùêë

é -=ú

ùêë

cossinsincos

y’P’

yθsinxθcos'x -=xθsinyθcos'y +=

10/2/1767

RotationMatrixProperties

A2Drotationmatrixis2x2

1)det( ==×=×

RIRRRR TT

ùêë

éúû

ùêë

é -=ú

ùêë

cossinsincos

Note: R belongs to the category of normal matrices and satisfies many interesting properties:

10/2/1768

RotationMatrixProperties• Transposeofarotationmatrixproducesarotationintheoppositedirection

• Therowsofarotationmatrixarealwaysmutuallyperpendicular(a.k.a.orthogonal)unitvectors– (andsoareitscolumns)

10/2/1769

1)det( ==×=×

RIRRRR TT

Scaling+Rotation+TranslationP’=(TRS)P

=úúú

êêê

úúú

êêê

úúú

êêê

úúú

êêê

é=×××=

1000s000s

1000θcossinθ0inθsθcos

100t10t01

=úúú

êêê

úúú

êêê

úúú

êêê

1000s000s

100tθcossinθtinθsθcos

úúú

êêê

ùêë

úúú

êêê

ùêë

éúû

ùêë

10/2/1770

Thisistheformofthegeneral-purposetransformationmatrix

• Matrixinverse• Matrixrank• EigenvaluesandEigenvectors• MatrixCalculate

10/2/1771

Theinverseofatransformationmatrixreversesitseffect

• GivenamatrixA,itsinverseA-1 isamatrixsuchthatAA-1=A-1A=I

• E.g.

• Inversedoesnotalwaysexist.IfA-1 exists,A isinvertible ornon-singular.Otherwise,it’ssingular.

• Usefulidentities,formatricesthatareinvertible:

10/2/1772

Inverse

• Pseudoinverse– SayyouhavethematrixequationAX=B,whereAandBareknown,andyouwanttosolveforX

10/2/1773

MatrixOperations

– Youcouldcalculatetheinverseandpre-multiplybyit:A-1AX=A-1B→X=A-1B

10/2/1774

MatrixOperations

– Youcouldcalculatetheinverseandpre-multiplybyit:A-1AX=A-1B→X=A-1B

– Pythoncommandwouldbenp.linalg.inv(A)*B– Butcalculatingtheinverseforlargematricesoftenbringsproblemswithcomputerfloating-pointresolution(becauseitinvolvesworkingwithverysmallandverylargenumberstogether).

– Or,yourmatrixmightnotevenhaveaninverse.

10/2/1775

MatrixOperations

• Pseudoinverse– Fortunately,thereareworkaroundstosolveAX=Binthesesituations.Andpythoncandothem!

– Insteadoftakinganinverse,directlyaskpythontosolveforXinAX=B,bytypingnp.linalg.solve(A,B)

– Pythonwilltryseveralappropriatenumericalmethods(includingthepseudoinverseiftheinversedoesn’texist)

– PythonwillreturnthevalueofXwhichsolvestheequation• Ifthereisnoexactsolution,itwillreturntheclosestone• Iftherearemanysolutions,itwillreturnthesmallestone

10/2/1776

MatrixOperations

• Pythonexample:

10/2/1777

MatrixOperations

>> import numpy as np>> x = np.linalg.solve(A,B)x =

1.0000-0.5000

10/2/1778

Therankofatransformationmatrixtellsyouhowmanydimensionsittransformsavectorto.

Linearindependence• Supposewehaveasetofvectorsv1, …, vn• Ifwecanexpressv1 asalinearcombinationoftheothervectorsv2…vn,thenv1 islinearlydependent ontheothervectors.– Thedirectionv1 canbeexpressedasacombinationofthedirectionsv2…vn. (E.g.v1 =.7 v2-.7 v4)

10/2/1779

Linearindependence• Supposewehaveasetofvectorsv1, …, vn• Ifwecanexpressv1 asalinearcombinationoftheothervectorsv2…vn,thenv1 islinearlydependent ontheothervectors.– Thedirectionv1 canbeexpressedasacombinationofthedirectionsv2…vn.(E.g.v1 =.7 v2-.7 v4)

• Ifnovectorislinearlydependentontherestoftheset,thesetislinearlyindependent.– Commoncase:asetofvectorsv1, …, vn isalwayslinearlyindependentifeachvectorisperpendiculartoeveryothervector(andnon-zero)

10/2/1780

LinearindependenceNotlinearlyindependent

10/2/1781

Linearlyindependentset

Matrixrank

• Column/rowrank

– Columnrankalwaysequalsrowrank

• Matrixrank

10/2/1782

Matrixrank• Fortransformationmatrices,theranktellsyouthedimensionsoftheoutput

• E.g.ifrankofA is1,thenthetransformation

p’=Apmapspointsontoaline.

• Here’samatrixwithrank1:

10/2/1783

Allpointsgetmappedtotheliney=2x

Matrixrank• Ifanm xmmatrixisrankm,wesayit’s“fullrank”– Mapsanm x1vectoruniquelytoanotherm x1vector– Aninversematrixcanbefound

• Ifrank<m,wesayit’s“singular”– Atleastonedimensionisgettingcollapsed.Nowaytolookattheresultandtellwhattheinputwas

– Inversedoesnotexist

• Inversealsodoesn’texistfornon-squarematrices

10/2/1784

• Matrixinverse• Matrixrank• EigenvaluesandEigenvectors(SVD)• MatrixCalculus

10/2/1785

EigenvectorandEigenvalue

• Aneigenvector x ofalineartransformation A isanon-zerovectorthat,when A isappliedtoit,doesnotchangedirection.

10/2/1786

• Aneigenvector x ofalineartransformation A isanon-zerovectorthat,when A isappliedtoit,doesnotchangedirection.

• Applying A totheeigenvectoronlyscalestheeigenvectorbythescalarvalue λ,calledaneigenvalue.

10/2/1787

• WewanttofindalltheeigenvaluesofA:

• Whichcanwewrittenas:

• Therefore:

10/2/1788

• Wecansolveforeigenvaluesbysolving:

• Sincewearelookingfornon-zerox,wecaninsteadsolvetheaboveequationas:

10/2/1789

Properties

• ThetraceofaAisequaltothesumofitseigenvalues:

10/2/1790

Properties• ThetraceofaAisequaltothesumofitseigenvalues:

• ThedeterminantofAisequaltotheproductofitseigenvalues

10/2/1791

• TherankofAisequaltothenumberofnon-zeroeigenvaluesofA.

10/2/1792

• TherankofAisequaltothenumberofnon-zeroeigenvaluesofA.

• TheeigenvaluesofadiagonalmatrixD=diag(d1,...dn)arejustthediagonalentriesd1,...dn

10/2/1793

Spectraltheory

• Wecallaneigenvalueλ andanassociatedeigenvectoraneigenpair.

• Thespaceofvectorswhere(A−λI)=0isoftencalledtheeigenspace ofAassociatedwiththeeigenvalueλ.

• ThesetofalleigenvaluesofAiscalleditsspectrum:

10/2/1794

Spectraltheory

• Themagnitudeofthelargesteigenvalue(inmagnitude)iscalledthespectralradius

–WhereCisthespaceofalleigenvaluesofA

10/2/1795

Spectraltheory

• Thespectralradiusisboundedbyinfinitynormofamatrix:

• Proof:Turntoapartnerandprovethis!

10/2/1796

Spectraltheory

• Thespectralradiusisboundedbyinfinitynormofamatrix:

• Proof:Letλ andvbeaneigenpair ofA:

10/2/1797

Diagonalization

• Ann× nmatrixAisdiagonalizableifithasnlinearlyindependenteigenvectors.

• Mostsquarematrices(inasensethatcanbemademathematicallyrigorous)arediagonalizable:– Normalmatricesarediagonalizable– Matriceswithndistincteigenvaluesarediagonalizable

Lemma:Eigenvectorsassociatedwithdistincteigenvaluesarelinearlyindependent.

10/2/1798

Diagonalization

• Ann× nmatrixAisdiagonalizableifithasnlinearlyindependenteigenvectors.

• Mostsquarematricesarediagonalizable:– Normalmatricesarediagonalizable–Matriceswithndistincteigenvaluesarediagonalizable

Lemma:Eigenvectorsassociatedwithdistincteigenvaluesarelinearlyindependent.

10/2/1799

Diagonalization

• Eigenvalueequation:

–WhereDisadiagonalmatrixoftheeigenvalues

10/2/17100

Diagonalization

• Eigenvalueequation:

• Assumingallλi’s areunique:

• Rememberthattheinverseofanorthogonalmatrixisjustitstransposeandtheeigenvectorsareorthogonal

10/2/17101

Symmetricmatrices

• Properties:– ForasymmetricmatrixA,alltheeigenvaluesarereal.

– TheeigenvectorsofAareorthonormal.

10/2/17102

Symmetricmatrices

• Therefore:

– where• So,ifwewantedtofindthevectorxthat:

10/2/17103

Symmetricmatrices

• Therefore:

– where• So,ifwewantedtofindthevectorxthat:

– Isthesameasfindingtheeigenvectorthatcorrespondstothelargesteigenvalue.

10/2/17104

SomeapplicationsofEigenvalues

• PageRank• Schrodinger’sequation• PCA

10/2/17105

• Matrixinverse• Matrixrank• EigenvaluesandEigenvectors(SVD)• MatrixCalculus

10/2/17106

MatrixCalculus– TheGradient

• LetafunctiontakeasinputamatrixAofsizem× nandreturnsarealvalue.

• Thenthegradient off:

10/2/17107

MatrixCalculus – TheGradient

• Everyentryinthematrixis:• thesizeof∇Af(A)isalwaysthesameasthesizeofA.SoifAisjustavectorx:

10/2/17108

Exercise

• Example:

• Find:

10/2/17109

Exercise

• Example:

• Fromthiswecanconcludethat:

10/2/17110

MatrixCalculus – TheGradient

• Properties

10/2/17111

MatrixCalculus – TheHessian

• TheHessianmatrixwithrespecttox,writtenorsimplyasHisthen× nmatrixof

partialderivatives

10/2/17112

• Eachentrycanbewrittenas:

• Exercise:WhyistheHessianalwayssymmetric?

10/2/17113

• Eachentrycanbewrittenas:

• TheHessianisalwayssymmetric,because

• ThisisknownasSchwarz'stheorem:Theorderofpartialderivativesdon’tmatteraslongasthesecondderivativeexistsandiscontinuous.

10/2/17114

MatrixCalculus– TheHessian

• Notethatthehessianisnotthegradientofwholegradientofavector(thisisnotdefined).Itisactuallythegradientofeveryentry ofthegradientofthevector.

10/2/17115

MatrixCalculus– TheHessian

• Eg,thefirstcolumnisthegradientof

10/2/17116

Exercise

• Example:

10/2/17117

Exercise

10/2/17118

Exercise

10/2/17119

Dividethesummationinto3partsdependingonwhether:• i ==kor• j==k

Exercise

10/2/17120

Exercise

10/2/17121

Exercise

10/2/17122

Exercise

10/2/17123

Exercise

10/2/17124

Exercise

10/2/17125

Exercise

10/2/17126

Exercise

10/2/17127

Exercise

10/2/17128

Whatwehavelearned• Vectorsandmatrices– BasicMatrixOperations– SpecialMatrices

10/2/17129

Linear Algebra Primer - Artificial...

Documents

Transcript of Linear Algebra Primer - Artificial...

Linear Algebra - Exercisesmapmf.pmfst.unist.hr/~vlasta/mathematics/Unit 1/Linear algebra... · Linear Algebra - Exercises 9 Linear Systems. Method of Elimination 1. Determine whether

L11: Sparse Linear Algebra on GPUs CS6235. 2 Sparse Linear Algebra 1 L11: Sparse Linear Algebra CS6235 .

Linear Algebra Tutorial - NPTELnptel.ac.in/courses/106106139/LinAlTutorial.pdfWhat is Linear Algebra Linear Algebra Linear algebra is the branch of mathematics concerning vector spaces

Linear Algebra Tutorial - NPTEL · What is Linear Algebra Linear Algebra Linear algebra is the branch of mathematics concerning vector spaces and linear mappings between such spaces.

Dynamical Systems and Linear Algebra · Dynamical Systems and Linear Algebra. Dynamical Systems and Linear Algebra. Dynamical Systems and Linear Algebra ... wouldliketomentionthebook

Sage Primer for Linear Algebra A First Course in Linear ...linear.ups.edu/download/fcla-3.11-sage-5.12-primer.pdfSage Primer for Linear Algebra A First Course in Linear Algebra Robert

Linear Algebra: Matrices, Vectors, Determinants. Linear Systems€¦ · Linear Algebra: Matrices, Vectors, Determinants. Linear Systems Linear algebra is a fairly extensive subject

Linear Algebra Meets Lie Algebra

Primer on Linear Algebra - Imperial College LondonPrimer on Linear Algebra ... • Solving systems ... • Interpretation: the linear transformation implied by cannot change the direction

Www.soran.edu.iq Linear Algebra Diyako Ghaderyan 1 Contents: Linear Equations in Linear Algebra Matrix Algebra Determinants Vector Spaces Eigenvalues.

QUANTUM ALGORITHMS VIA LINEAR ALGEBRAmmrc.amss.cas.cn/tlb/201702/W020170224608149911380.pdf · Quantum algorithms via linear algebra: a primer / Richard J. Lipton and Kenneth W. Regan.

Linear Algebra

Linear Algebra and Smarandache Linear Algebra

Linear Algebra Prerequisites Jana Kosecka kosecka@gmukosecka/cs687/linear-algebra-review.pdf · Linear Algebra Prerequisites Jana Kosecka kosecka@gmu.edu Recommended txt: Linear Algebra

Linear Algebra and Applications: Numerical Linear Algebra · · 2008-07-06Linear Algebra and Applications: Numerical Linear Algebra ... Linear Algebra, SIAM, 1997. James W. Demmel,

Primer for Geometric Algebra

Geometric Algebra Primer

Algebra Linear

Linear Algebra Primer - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1819/files/02_linalg_review.pdf · a ew Stanford University 27-p-2018 1 Linear Algebra Primer

Numerical Linear Algebra Primer - CMU Statisticsryantibs/convexopt/lectures/num-lin... · 2019. 10. 28. · Numerical Linear Algebra Primer Ryan Tibshirani Convex Optimization 10-725.