(2d) Matrices

CS101 2012.1

Chakrabarti

Declaration and access

int imat[rows][cols];double dmat[rows][cols]; rows*cols cells allocated of the given type Access just like 1d vectors, example:

• imat[rx][cx] = rvalue;• lvalue = 5 * imat[rx][cx] – 3;

Make sure 0 rx < rows and 0 cx < cols Violations may lead to silent garbage

Chakrabarti

Storage: row major Internally, no distinction between [3][5] and [15] cellNumber(rx, cx) = rx * cols + cx Base address of imat[rx][cx] is base address of imat

plus 4 * cellNumber(rx, cx)

0 1 2 3 4

0 Cell 0, Byte 0

1, 4 2, 8 3, 12 4, 16

1 5, 20 6, 24 7, 28 8, 32 9, 36

2 10, 40 11, 44 12, 48 13, 52 14, 56…59

Demo

Chakrabarti

Basic operations Initialize square matrix to identityfor (int rx = 0; rx < rows; ++rx) { for (int cx = 0; cx < cols; ++cx) { dmat[rx][cx] = (rx == cx)? 1 : 0; } }

Better code uses two loops (why?)• In first rx,cx loop set all elements to zerofor (int rx=0; rx<rows; ++rx) {for (int cx=0; cx<cols; ++cx) { dmat[rx][cx]=0; } }

• In second loop set diagonal elements to onefor (int dx=0; dx<rows; ++dx) { dmat[dx][dx]=1; }

Chakrabarti

Basic operations Initialize to upper triangular matrix with 1s

for (int rx = 0; rx < rows; ++rx) {

for (int cx = 0; cx < cols; ++cx) {

dmat[rx][cx] = (rx <= cx)? 1 : 0;

}}

cx

r

x rx <= cx

Chakrabarti

Basic operations Transpose a square matrix Don’t double transpose back to square one!

for (int rx = 0; rx < rows; ++rx) { for (int cx = rx+1; cx < cols; ++cx) {

float tmp = fmat[rx][cx]; fmat[rx][cx] = fmat[cx][rx]; fmat[cx][rx] = tmp; }}

No need to transpose diagonal or below

Swap [rx][cx] with [cx][rx]

Chakrabarti

Transpose demo

1 2 3

4 5 6

7 8 9

1 4 3

2 5 6

7 8 9

1 4 3

2 5 6

7 8 9

1 4 7

2 5 6

3 8 9

1 4 7

2 5 6

3 8 9

1 4 7

2 5 8

3 6 9

Chakrabarti

Matrix vector multiplication code

float A[rows][cols];float x[cols], y[rows];// fill up A and xfor (int rx=0; rx < rows; ++rx) { y[rx] = 0; for (int cx=0; cx < cols; ++cx) {

y[rx] += A[rx][cx] * x[cx]; }}

Chakrabarti

Matrix vector multiplication demo

for (int rx=0; rx < rows; ++rx) { y[rx] = 0; for (int cx=0; cx < cols; ++cx) { y[rx] += A[rx][cx] * x[cx]; }}

a00 a01 a02

a10 a11 a12

x0

x1

x2

y0

y1=

a00*x0+a01*x1+a02*x2x0

x1

x2

a10*x0+a11*x1+a12*x2

Chakrabarti

Matrix as transformation

Consider all x’s whose arrow tips lie on the unit circle (i.e., norm=1)

After transformation by multiplying by A, what is the locus of the arrow tips of resulting y’s?

preimagex

yimage

Demo

Chakrabarti

Mapping a circle to…

Preimages onunit circle

Images y aftertransformation,on an ellipse

Two directionsto rule them all

Chakrabarti

Stretched but not turned The two directions that do not

turn when scaled by A are eigen vectors u0, u1

Length of semi-major and semi-minor axes are eigen values 0 > 1

u0 and u1 are perpendicular to each other, have unit norm

Can be used as new coordinate axes

Consider a vector x with coordinates (1, c) along new axes u0 and u1

Chakrabarti

Power iterations Find largest eigenvalue and corresponding

eigenvector u of matrix A Solves Au = u Initialize x arbitrarily Repeat

• y = Ax

• x = y / ||y||2 Why does this work? Demo using Scilab

Call the sequence ofvalues of x and y asx(0), y(0), x(1), y(1), etc.

Chakrabarti

Big crunchUnitnorm

Chakrabarti

When to stop When x(k-1) and x(k) are sufficiently close to

each other ||x(k-1)-x(k)||2 is smaller than some small For you to explore: how to find u1, 1?

Let’s write Boost code for power iterations Building blocks we need

• Matrix, vector• Vector difference, norm• Matrix vector multiplication

All provided by Boost UBLAS library

Chakrabarti

Boost code

for (;;) { vector<double> y = prod(A, x); vector<double> newX = y/norm_2(y);

if (norm_2(newX - x) < eps) { break; } x = newX;}

Already implementedIn Boost UBLAS library

Chakrabarti

Matrix-matrix multiplicationfloat amat[lsize][msize], bmat[msize][nsize], cmat[lsize][nsize];

for (int crx=0; crx<lsize; ++crx) { for (int ccx=0; ccx<nsize; ++ccx) { cmat[crx][ccx] = 0; for (int abx=0; abx<msize; ++abx) { cmat[crx][ccx] += amat[crx][abx] * bmat[abx][ccx];

} }}

Like matrix-vector multiplicationBut x has many columnsTriple nested loopTime required is lsize*msize*nsize

Chakrabarti

Least squares fit

Chakrabarti

Least square objective

But we don’t know how to invert matrices yet

Chakrabarti

Iterative solution to find a zero of g(w) Use an iterative scheme instead Fix an arbitrary step size>0 Start with some w(0)

Repeat for t = 1, 2, …• Set w(t) = w(t-1) - g(w(t-1))

Until w(t) and w(t-1) are very close

In this specific problem, guaranteed to converge

Demo

Chakrabarti

Determinant Adding a multiple of one row to another

leaves the determinant unchanged Exchanging a pair of rows or columns

causes the determinant to be multiplied by minus one

Determinant of a upper triangular matrix (all zeros below diagonal) is the product of diagonal elements

Chakrabarti

Determinant: approach

Chakrabarti

Determinant code sketch// matrix<double> A filled w/ valuesfor (int dx=0; dx<A.size1(); ++dx) { // make A(dx,0) through A(dx,dx-1) zeros for (int sx=0; sx<dx; ++sx) { double fac = A(dx,sx)/A(sx,sx); for (int cx=0; cx<A.size2(); ++cx) { A(dx,cx) -= fac * A(sx,cx); } //cx } //sx} //dx// finally multiply together diagonals

Chakrabarti

Near-zero diagonal elements If a diagonal element (d, d) becomes too

small, we look for a lower row z where dth column is not small

Then we swap rows d and z And remember to adjust the sign later Called a “pivot” step What if there are no such rows below d? More advanced pivoting needed Sometimes the determinant may be zero

• All pivoting attempts will fail

Chakrabarti

Toward better pivoting

Goal is to zero out first column below diagonal

for (int rx = 1; rx < A.size1(); ++rx) { double fac = A(rx,0) / A(0,0); for (int cx = 0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(0, cx); }}

0

0

0

0

0

0 0

0 0

0 0

0

0 0

0 0 0

0 0 0

Chakrabarti

Nested loopsfor (int dx=0; dx < A.size1(); ++dx) {

// zero A(dx+1,dx) through A(n-1,dx) for (int rx=dx+1; rx < A.size1(); ++rx) {

double fac = A(rx,dx) / A(dx,dx); for (int cx=0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(dx, cx); }//cx }//rx}//dx

If |A(dx,dx)| < do a pivot step

Chakrabarti

Modularity Thus far we have written most of our code in

the main function Invoking library functions like sqrt or log as

needed Increasingly important to modularize our own

code• Implement once, reuse from many programs• Debug part by part, maintain in one place• Read code and think more clearly• Separate specification from implementation

Chakrabarti

Detour: swap function Swap values in two variables of any type “Polymorphic” function like +, *, /

template <class T>

void swap(T& a, T& b) { T tmp = a; a = b; b = tmp;}

Works for any type T

Don’t forget the &

Chakrabarti

swapRowstemplate <class T>

void swapRows(matrix<T>& mat, int r1, int r2){ for (int cx=0; cx<mat.size2(); ++cx) {

swap(mat(r1,cx), mat(r2,cx)); }}

Chakrabarti

pivotIfNeededbool pivotIfNeeded(matrix<double>& A, int dx, double& sign){ if (abs(A(dx,dx)) > eps) return true; for (int rx = dx+1; rx < A.size1(); ++rx) { if (abs(A(rx,dx)) > eps) {

swapRows(A, dx, rx); sign = -sign; return true; } } return false; // failed to find a pivot}

In/OutIn/Out

In

Return true iff pivoting not needed,

or successful

Chakrabarti

Solving a system of equations Given Ax = y where A is nn; x, y are n1 Goal is to solve for x Operations we have done so far on A do not

change the constraints on x A turned into an upper triangular matrix Code to manipulate rows of y in sync with A Can solve backward: find xn1, then xn2, …

Chakrabarti

Pivoting y in sync with A Earlier, pivot function had inputs A, dx, sign New inputs will be A, y, dx No need to keep track of signbool pivotIfNeeded(matrix<double>& A, vector<double>& y, int dx) { if (abs(A(dx,dx)) > eps) return true; for (int rx = dx+1; rx < A.size1(); ++rx) { if (abs(A(rx,dx)) > eps) { swapRows(A, dx, rx); swap(y(dx), y(rx)); return true; } } return false; }

Chakrabarti

Nested loops including yfor (int dx=0; dx < A.size1(); ++dx) { pivotIfNeeded(A, y, dx); // zero A(dx+1,dx) through A(n-1,dx) for (int rx=dx+1; rx < A.size1(); ++rx) {

double fac = A(rx,dx) / A(dx,dx); for (int cx=0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(dx, cx); }//cx y(rx) -= fac * y(dx); }//rx}//dx

Chakrabarti

Last step// A is already upper triangular// y available, x allocatedfor (int ix=x.size()-1; ix>=0; --ix) { x(ix) = y(ix); for (int jx=ix+1; jx<x.size(); ++jx) {

x(ix) -= A(ix, jx) * x(jx); } x(ix) /= A(ix, ix);}

Chakrabarti

Inverse Once we can solve Ax = y, inverting a matrix

is no big deal (Conceptually) invoke solver n times§

First with y =(1, 0, …, 0)T

Then with y=(0, 1, …, 0)T

And so on up to y=(0, 0, …, 1)T

I.e., columns of the identity matrix Collect x’s to form A-1

§Obviously, don’t triangularize A n times over! Pass in all different y’s in one shot

Chakrabarti

for (int dx=0; dx < A.size1(); ++dx) { pivotIfNeeded(A, R, dx); // zero A(dx+1,dx) through A(n-1,dx) for (int rx=dx+1; rx < A.size1(); ++rx) {

double fac = A(rx,dx) / A(dx,dx); for (int cx=0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(dx, cx); R(rx, cx) -= fac * R(dx, cx); }//cx}//rx}//dx

Nested loops, version 3

R initialized to the identity matrix;its columns are our initial y’s

Chakrabarti

Image processing applications If you look at any “bitmap”

image up close, you see it is a 2d array of pixels

Each pixel has a red, green blue intensity

Mix RGB primary colors An intensity is an integer

between 0 and 255 (one unsigned byte)

Called “24 bit color”

Chakrabarti

Object identification Babies identify hundreds of objects by age 1 They get shapes before they get colors Need to perceive boundaries of objects Significant change of color among neighboring

pixels If you are very

different from theaverage of yourneighbors, you may be an edge pixel

Chakrabarti

Edge detection Find (weighted) average of neighbor pixels Compare with my value If difference large, (likely to be) edge

i,j

Chakrabarti

Contrast enhancement An image may have been captured under

less than ideal circumstances• Scene lighting, camera calibration• Tissue density and water content

Use statisticaltechniques to“correct” theimage

Critical in medicalimaging and diagnosis

Chakrabarti

Code details EasyBMP.h publishes function signatures for

reading and writing .bmp image files EasyBMP.cpp implements the functions Effectively turns them into three in-memory

pixel matrices with R, G, B intensities Compile EasyBMP.cpp to EasyBMP.o ahead

of time, once Compile Edgy.cpp and link with EasyBMP.o

to produce Edgy.exe Compile Enhance.cpp … Enhance.exe