Matrix Algebra

Methods for DummiesFIL

November 17 2004

Mikkel Wallentinmikkel@pet.auh.dk

Sources

• www.sosmath.com• www.mathworld.wolfram.com• www.wikipedia.org• Maria Fernandez’ slides (thanks!) from

previous MFD course: http://www.fil.ion.ucl.ac.uk/spm/doc/mfd-2004.html

• Slides from SPM courses: http://www.fil.ion.ucl.ac.uk/spm/course/

Design matrix …

=

+

= +Y X

data ve

ctor

design

matr

ix

param

eters

error

vecto

r

= the b

etas (

here : 1

to 9)

Scalars, vectors and matrices • Scalar: Variable described by a single

number – e.g. Image intensity (pixel value)

e

n

vv

v

• Vector: Variable described by magnitude and direction

476145321

A

Square (3 x 3) Rectangular (3 x 2) d i j : ith row, jth column

837241

C 3

2• Matrix: Rectangular array of scalars

333231

232221

131211

ddddddddd

D

z

y

x

v

vv

v

vczbyax vbyax

Matrices

• A matrix is defined by the number of Rows and the number of Columns (eg. a (mxn) matrix has m rows and n columns).

• A square matrix of order n, is a (nxn) matrix.

• Addition (matrix of same size)

– Commutative: A+B=B+A– Associative: (A+B)+C=A+(B+C)

• Eg.

Matrix addition

3333

1111

2222

BA

Matrix multiplication

Rule: In order to perform the multiplication AB, where A is a (mxn) matrix and B a (kxl) matrix, then we must have n=k. The result will be a (mxl) matrix.

Multiplication of a matrixand a constant:

…Each parameter (the betas) assigns a weight to a single column in the design matrix …

=

+

= +Y X

data ve

ctor

design

matr

ix

param

eters

error

vecto

r

= the b

etas (

here : 1

to 9)

Transposition

943

Td

211

b 211Tb 943d

column → row row → column

476145321

A

413742651

TA

332313

322212

312111

321

3

2

1

yxyxyxyxyxyxyxyxyx

yyyxxx

Txy

Outer product = matrix

ii

iT yxyxyxyx

yyy

xxx

3

1332211

3

2

1

321yx

Inner product = scalar

Two vectors:

3

2

1

xxx

x

3

2

1

yyy

y

Example

Note: (1xn)(nx1) -> (1X1)

Note: (nx1)(1xn) -> (nXn)

…A contrast estimate is obtained by multiplying the parameter estimates by a transposed contrast vector …

=

+

= +Y X

data ve

ctor

design

matr

ix

param

eters

error

vecto

r

contr

ast v

ector

c

SPM{t}

A contrast = a linear combination of parameters: cT

cT = 1 0 0 0 0 0 0 0

divide by estimated standard deviation

T test - one dimensional contrasts - SPM{t}

T =

contrast ofestimated

parameters

varianceestimate

T =

ss22ccT (X(XTX)X)++cc

ccTbb

box-car amplitude > 0 ?=

> 0 ? =>

Compute 1xb + 0xb + 0xb + 0xb + 0xb + . . . and

b b b b b ....

Identity matrices• Is there a matrix which plays a similar role as the number 1 in

number multiplication? Consider the nxn matrix:

• For any nxn matrix A, we have A In = In A = A • For any nxm matrix A, we have In A = A, and A Im = A

H0: 3-9 = (0 0 0 0 ...)

cT =

SPM{F}

tests multiple linear hypotheses. Ex : does DCT set model anything?

F-test (SPM{F}) : a reduced model or ...multi-dimensional contrasts ?

test H0 : cT b = 0 ?

X1 (3-9)X0

This model ? Or this one ?

H0: True model is X0

X0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 00 0 0 0 1 0 0 00 0 0 0 0 1 0 00 0 0 0 0 0 1 00 0 0 0 0 0 0 1

Inverse matrices• Definition. A matrix A is called nonsingular or invertible if there

exists a matrix B such that:

• Notation. A common notation for the inverse of a matrix A is A-1. So:

• The inverse matrix is unique when it exists. So if A is invertible, then A-1 is also invertible and

Determinants

Recall that for 2x2 matrices:

•Determinants are mathematical objects that are very useful in the analysis and solution of systems of linear equations (i.e. GLMs).•The determinant is a function that associates a scalar det(A) to every square matrix A.•The fundamental geometric meaning of the determinant is as the scale factor for volume when A is regarded as a linear transformation. • A matrix A has an inverse matrix A-1 if and only if det(A)≠0.• Determinants can only be found for square matrices.•For a 2x2 matrix A, det(A) = ad-bc. Lets have at closer look at that:

And generally :

Matrix Inverse - Calculations

dcba

A IAA

1001

43

211

dcba

xxxx

43

211

xxxx

A

acbd

bcad )(11A

A general matrix can be inverted using methods such as the Gauss-Jordan elimination, Gaussian elimination or LU decomposition

1001

43

43

21

21

dxbxcxax

dxbxcxax

bbcadadbc

bxdxacxb

acxx

)(1

)(01

1

222

21

acbd

A)det(11Ai.e. Note: det(A)≠0

System of linear equationsImagine a drink made of egg, milk and orange juice. Some of the propertiesof these ingredients are described in this table:

If we now want to make a drink with 540 calories and 25 gof protein, the problem of finding the right amount of the ingredientscan be formulated like this:

zyx

69211016080

25540

or

A similar problem …

=

+

= +Y X

data ve

ctor

design

matr

ix

param

eters

error

vecto

r

= the b

etas (

here : 1

to 9)

Cramer’s rule• Consider the linear system (in matrix form)

• A X = B

• where A is the matrix coefficient, B the nonhomogeneous term, and X the unknown column-matrix. We have: Theorem. The linear system AX = B has a unique solution if and only if A is invertible. In this case, the solution is given by the so-called Cramer's formulas:

• •

• where xi are the unknowns of the system or the entries of X, and the matrix Ai is obtained from A by replacing the ith column by the column B. In other words, we have

• •

• where the bi are the entries of B. Thank you Bent Kramer!