Algorithm for Gauss elimination

1) first eliminate

for each eq. j, j=1 to n-1

for all eq.s k greater than j

a) multiply eq. j by akj/ajj

b) subtract the result from eq. K

This leads to upper triangular

2) now backsubstitute

a) determine xn from

b) put xn into n-1 eq.

c) solve for xn-1

d) repeat from b), moving back to n-2, n-3, etc. until all equations are solved

n

n

nn

n a

bx

1

1

Matlab code

Operation counting

Important as matrix gets large

For Gauss elimination

• elimination routine uses on the order ofoperations

• backsubstitution uses

3

3n

2

2nO

Problems with Gauss elimination (as done last time)

1) division by zero

2) round off errors

3) ill conditioned systems

Division by zero

1522

457

523

321

321

32

xxx

xxx

xx

Using first eq. to eliminate x1 in second eq. means dividing by 0

1522

42*0

153*

0

170*

0

11

523

321

321

32

xxx

xxx

xx

Pivoting developed to avoid this

find row with largest absolute value under pivot element

switch rows

More later

Round off errors

With more than n3/3 operations, get a lot of chopping.

More important - error is propagted

More than 100 equations - round can be very important - system dependent

Ill conditioned systems - small changes in coefficients lead to large changes in solution

Round-off errors especially important in ill-conditioned systems

Recall ill conditioned system from graphical methods

7.07.1

15.15.2

21

21

xx

xx

-2

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6

x1

x2

2222121

1212111

bxaxa

bxaxa

Can write as

22

21

22

212

12

11

12

112

a

bx

a

ax

a

bx

a

ax

Since slopes are almost equal

22

21

12

11

a

a

a

a

22

21

12

11

a

a

a

a

21122211 aaaa becomes

021122211 aaaa

0det2221

1211

aa

aa

05.055.25.217.1

5.15.2det

Determinant close to zero indicates ill-conditioned set of equations.

How close?

No clear answer

Problem of scale

Multiply our set of equations by 100

70100170

100150250

21

21

xx

xx

500500,25000,25100170

150250det

-2

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6

x1

x2

However, graphically

No change

Scaling

Make maximum element in any row =1

4118.05882.0

4.06.0

21

21

xx

xx70100170

100150250

21

21

xx

xx

0118.05882.01

6.01det

Another problem - singular systems

Two equations in the set are the same

Determinant is 0.

Calculating determinant using Gauss elimination

Given

nn

n

n

n

n

a

aa

aaa

aaaa

A

1

333

22322

1131211

''''

'''

nn

naaaaA 1332211 '''det

then

How to avoid pitfalls

•higher precision

•pivoting

•scaling

Partial Pivoting

Determine the largest coefficient in the column below pivot element

Then switch rows

(Compete pivot switches columns also, but is rarely used.)

Example:

4

11

3

7

653

3210

451

1273

Exchange rows 2 and 3

4

3

11

7

653

451

3210

1273

And now eliminate

Algorithm for Gauss elimination using improvements

1) first eliminate

for each eq. j, j=1 to n-1

first scale each equation k greater than jthen pivotnow

a) multiply eq. j by akj/ajj

b) subtract the result from eq. k

The Gauss Jordan method

Major difference - eliminate unknowns from all rows, not just subsequent ones

Normalize matrix so all entries are 1

Leads to identity matrix instead of upper triangular

Backsubstitution is easy

3

2

1

3

2

1

'

'

'

1

1

1

b

b

b

x

x

x