Gaussian-, Jordan Elimination and Matrix Inversesb_novak/BANKI/gauss_ang.pdf · Gaussian-, Jordan...

© Bércesné dr. Novák Ágnes 2015. február 13.

Gaussian-, Jordan Elimination and Matrix Inverses

GAUSSIAN ELIMINATION

A system of linear equations consists of several linear equation :

Example: in general: for instance:

Here a11 is a coefficient – a number. The first unknown variable is x1. aik is a coefficient – a number. The kth unknown variable is xk.

Consider the following system of linear equations in three variables.

x1 + x2 + 2x3 = 8

x2 - 5x3 = -9 x3 = 2

It is easy to solve, since x3 = 2 is given in the third equation. We just substitute this value into

the 2nd equation, in order to get x2, and finally, with x2 and x3 being substituted into the first

we get the solution.

How nice would it be, if each system had the same echelon form!

The good news is, that even if they usually do not look like the one above - but using

eqvivalent elementary raw operations, we are able to achieve this form.

Elementary row operations are steps for solving the linear system of equations.

The insight of the method of elementary row operations is that in some sense they do not

change the system of equations. For example, the solution to a set of linear equations does

not depend on what equation is written first; thus swapping rows should not change the

solution.

There are three elementary row operations:

I. Interchange two equations

II. Multiply an equation with non zero real number

III. Add a multiple of one equation to another equation

a11x1+a12x2+ a13x3…a1nxn =b1

a21x1+a22x2+ a23x3…a2nxn =b2

a31x1+a32x2+ a33x3…a3nxn =b3

…

am1x1+am2x2+ am3x3…amnxn= bm

x1+ 2x2+ 3x3+ 4x4 = 5

6x1+ 7x2+ 8x3+ 9x4 = 10

11x1+12x2+ 13x3+14x4 =15

16x1+17x2+ 18x3+19x4= 20


Look at this system of equation.

One possible algebraic solution is this. First we divided the 3rd equation by 3, getting x+y+ z=6. This one is moved into the very first row. The other operations are denoted in the

boxes below. → →

Here is the echelon form, and from there, with back substitution we get z=3, y=2, and x=1.

So, this equation has ONE solution.

4x-2y+ 2z=6

4x+ y- z=3

3x+ 3y+3z=18

x+ y+ z=6

4x-2y+ 2z=6 /+(-4)*I.e

4x+ y- z=3 /+(-4)*I.e

x+y+z=6

-6y-2z=-18

-3y-5z=-21/+(-1/2)*2*II.e

e

x+ y+ z=6

-6y-2z=-18

-4z=-12

→ → →

4x-2y+ 2z=6

4x+ y- z=3

3x+ 3y+3z=18

Each equation can be interpreted as an equation of a plane, as you can see

in the figure below. These planes have exactly on common point: (1,2,3)

so, this is the only solution of this system.

Now we will follow how to get an algebraic solution,.


Methods for solving System of Linear equations

1. Gaussian Elimination Method

2. Gauss – Jordan Elimination Method

Gaussian Elimination Method

STEP 1. by using elementary row operations we produce an echelon form, being

eqvivalent to the original system of equation.

a11x1+a12x2+ a13x3…a1nxn =b1

a21x1+a22x2+ a23x3…a2nxn =b2

a31x1+a32x2+ a33x3…a3nxn =b3

…


α11x1+ α 12x2+ α13x3+… α 1nxn =1

α 22x2+ α23x3+… α 2nxn =2

α33x3+… α 3nxn =3

…

α m’nxn =m

= =


How can we do this?

Suppose we are at the first step, that is, with a11 x1 we want to eliminate x1 from the ith

equation. In a.) we produce a 1 coefficient in equation 1. for x1 . In b.) the new 1st equation is

multiplied by -ai1 so have the opposit coefficient in the 1st and in the ith equatoion. Adding

these two, in c.) we get 0 x1 in the ith equation. . Similarly we eliminate x1 from all the

equations below except for the first equation.

First and ith

equation from

the system,

a.)

c.)

When we finished the elimination of x1, the process continues with the elimination of only

x2 , that are standing below the main diagonal. For this, we use the x2 in the second equation.

In general, xk is eliminated always by the help of the kth element of the kth equation, except

when we have 0 at the position akk. In this case we change the equations in order to have a

nonzero element in akk. We are ready when we have only zeros below the main diagonal.

1

1111

13

11

132

11

121

1b

ax

a

ax

a

ax

a

ax n

n

...

332211 ininiii bxaxaxaxa

ininiii bxaxaxaxa 332211

11313212111 bxaxaxaxa nn

...

332211 ininiii bxaxaxaxa

/ : a11

1

11

1

11

113

11

13132

11

1212

1

1111

13

11

132

11

121

10

1

ba

abxa

aaax

a

aaax

a

aaa

ba

xa

ax

a

ax

a

ax

iinn

iiniiii

nn

/ (-ai1)

+


STEP 2. Find solution by back – substitutions.

Since at the end we have an echelon form, the last unknown is calculated by a simple

division by α m’n. . The previous one can be got from the previous equation by simple

substitution of xn and so on…

So, a Gaussian elimination is a method by which we achieve an echelon form, and by back-

substituting we get all the values for the unknowns.

Example

1. x1+ 2x2 + 3x3 + 4x4 = 5

2. 6x1+ 7x2+ 8x3+ 9x4 = 10

3. 11x1+12x2+ 13x3+14x4 = 15

4. 16x1+ 17x2+ 18x3+19x4 = 20

Solution:

:

1. x1+ 2x2+ 3x3+ 4x4 = 5

2. 6x1+ 7x2+ 8x3+ 9x4 = 10 /+ (-6)* 1. eq. 3. 11x1+12x2+ 13x3+14x4 = 15/+(-11) * 1. eq. 4. 16x1+17x2+ 18x3+19x4 = 20/+(-16) * 1. eq.

Result:

1. x1+ 2x2+ 3x3+ 4x4 = 5

2. 0 -5x2 -10x3 -15x4 = -20

3. 0-10x2 -20x3 -30x4 = -40

4. 0-15x2-30x3-45x4= -60

1st ELIMINATION step is done. Now we are working on the elimination of x2

from equations 3 and 4.

1. x1+2x2+ 3x3 +4x4 = 5

2. 0- 5x2-10x3 - 15x4 = -20/:(-5) → x2 +2x3+3x4 = 4 new 2.eq. 3. 0-10x2-20x3 - 30x4 = -40/+10* new 2.eq. → 0 + 0+ 0 = 0

4. 0-15x2-30x3 - 45x4= -60/+15*new 2.eq. → 0 + 0+ 0 = 0

α11x1+ α 12x2+ α13x3+… α 1nxn =1

α 22x2+ α23x3+… α 2nxn =2

α33x3+… α 3nxn =3

…

α m’nxn =m


We kept the rules, but because the equations were not independent - one can be expressed by the

combination of the others – we got two rows with full of zeros. So it is not really an echelon form.

But still we can figure out the solution.

Beginning with the new 2nd equation:

x2+2x3+3x4 = 4→ x2= 4-2x3-3x4 substituted into the 1. equation:

x1+ 2x2+ 3x3+4x4 = 5 → x1= 5-2x2-3x3-4x4= 5- 2(4-2x3-3x4)-3x3-4x4=

= 5 -8+4x3+6x4-3x3- 4x4=-3+x3+2x4

So: x1= 5 -8+4x3+6x4-3x3- 4x4=-3+x3+2x4

x2= 4-2x3-3x4

This is the solution explicitely expressed each unknowns:

x1= -3+ p+2q

x2= 4-2p-3q

x3= p, pR

x4=q, qR

This simpler system of linear equation is the solution!

So there are infinitely many solutions.The meaning is, that each quartett x1, x2 x3 x4 satisfying this

new system, is a solution. x1 and x2 can be calculated with the freely chosen x3 and x4,

It is important, which unknown can be chosen freely. Sometimes it is not unique.

Generally if there IS a number in the main diagonal in the echelon (or last) form, then

that unknown SHOULD NOT BE CHOSEN FREELY, it will be expressed by the other

unknowns.

Gaussian elimination is matrix form

We may simplify the writing. As we learnt in high-school algebra, the variables x, y, and z

are just “place holders” in that the real structure of the problem is contained in the numbers.

So, there is no need to write them down.

One use of matrices is to focus on this structure without attending to the variables.

There are two matrices associated with this problem, the matrix of coefficients and the

augmented matrix. We show these two matrices just below.


STEP 1. by using elementary row operations

3

2

1

23

1312

3

2

1

333231

232221

131211

100

10

1

B

B

B

A

AA

b

b

b

aaa

aaa

aaa

STEP 2. Write back to a system of equation, and find solution by back – substitutions.

In other words:

System of linear equations:

33133232131

23123222121

13113212111

bxaxaxa

bxaxaxa

bxaxaxa

can be written in the form of matrices product

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

or we may write it in the form AX=b,

where A=

333231

232221

131211

aaa

aaa

aaa

, X =

3

2

1

x

x

x

, b =

3

2

1

b

b

b

a11x1+a12x2+ a13x3…a1nxn =b1

a21x1+a22x2+ a23x3…a2nxn =b2

a31x1+a32x2+ a33x3…a3nxn =b3

…


System of a linear equations:

mnmmm

n

n

n

…aaaa

…aaaa

…aaaa

…aaaa

321

3333231

2232221

1131211

Matrix of coefficients Augmented matrix:

mmnmmm

n

n

n

b…aaaa

b…aaaa

b…aaaa

b…aaaa

321

3333231

22232221

11131211

3


Augmented matrix is

3

2

1

333231

232221

131211

:

b

b

b

aaa

aaa

aaa

bA

Example:

Solution by Gaussian elimination in matrix form:

4x- 2y+ 2z=6

4x+ y -z=3

3x+ 3y+ 3z=18

4x- 2y+ 2z=6

4x+ y -z=3

3x+ 3y+ 3z=18

augment

ed →

matrix

→

/:3 and swap

rows:

/+(-4)* 1.R

/+(-4)* 1.R

↓

/+(-1/2)* 2.R

←

correspond-

ing

←

system

x + y + z=6

-6y -2z=-18

-4z=-12

/: (-6)

→

/: (-4)

x + y + z=6 x=1

y + (1/3)z=3, y=2

z=3


Example: Solve the system of linear equations by Gaussion- elimination method

x1 + x2 + 2x3 = 8

- x1 - 2x2 + 3x3 = 1

3x1 - 7 x2 + 4x3 = 10

Solution: Augmented matrix is

10 473

1 321

8 211

STEP 1.

14- 2100

9 510

8 211

R1+R2, -3R1+R3

104- 5200

9- 510

8 211

-R2, 10R2+R3

2100

9510

8211

-R3/52

Equivalent system of equations form is:

x1 + x2 + 2x3 = 8

x2 - 5x3 = -9 x3 = 2

STEP 2. Back Substitution x3 = 2

x2 = 5x3 -9 =10 – 9 =1

x1 = - x2 - 2x3 + 8 = -1 – 4 +8 = 3

Solution is x1 = 3, x2 = 1, x3 = 2.


GAUSS – JORDAN ELIMINATION METHOD

3

2

1

3

2

1

333231

232221

131211

100

010

001

B

B

B

b

b

b

aaa

aaa

aaa

Example.4. Solve the system of linear equations by Gauss - Jorden elimination method

x1 + x2 + 2x3 = 8

- x1 - 2x2 + 3x3 = 1

3x1 - 7 x2 + 4x3 = 10

Solution: Augmented matrix is

10 473

1 321

8 211

14- 2100

9 510

8 211

R1+R2, -3R1+R3

104- 5200

9- 510

8 211

-R2, 10R2+R3

2100

9510

8211

-R3/52

2100

1010

4011

-2R3+R1, 5R3+R2


2100

1010

3001

-R2+R1

Equivalent system of equations form is:

x1 = 3

x2 = 1

x3 = 2 is the solution of the system.

SYSTEM WITH NO SOLUTION

Example: 6 . Solve the system of linear equations

5161112

22 7 3

14 2

uzyx

uzyx

uzyx

Solution:

Augmented matrix is:

51611121

22731

14121

Reducing it to row echelon form (using Gaussian - elimination method)

41212100

16650

14121

R2- R1, R3-R1

30000

16650

14121

-R3+2R2

Last equation is

-30 but

30000

uzyx

hence there is no solution for the given system of linear equations.


An importamt special case - homogenous system: constants are 0.

Example:

There is always one solution, the so called trivial solution: x=y=z=0

Now we prove that in this case there is no other solution (follow the arrows):

However, in other cases there might be non-trivial solutions as well. IN this case, there are

infinitely many solutions.

Example: Full zero rows are neglected belw, since they do not hold information. Follow

the arrows.

.

x=p , y=-2p z=p, pR.

4x- 2y+2z=0

4x+ y- z=0

3x+ 3y+3z=0

→

/:3 and

change the

rows

/+(-1)* 3.R

/+2*3.R

↓

/+(-1/2)* 2.R

← ←

/:(-4)

↓*

→

/:(-6) →

/+(-1)* 2.R

↓

x =0 6y

y=0

z=0

↓

-x + z=0

2x+3y+4z=0

5x+6y+7z=0

/+2* 1.R

/+5*1.R /+(-2)* 2.R

/* (1/3)

-x + z=0

y+2z=0

→

←

210

001

000

630

101

630

101

1260

630

101

765

432

101


Conditions on Solutions

Example:For which values of ‘a’ will be following system

2)14(4

2 53

4 32

2

azayx

zyx

zyx

(i) infinitely many solutions?

(ii) No solution?

(iii) Exactly one solution?

Solution:

Augmented matrix is

21414

2513

4321

2 aa

Reducing it to reduced row echelon form

14270

101470

4321

2 aa

R2-3R1, R3-4R1

41600

210

4321

2

710

aa

71 R2, R3-R2

writing in the equation form,

4)4( 4)(

as written becan 3equation or

3 4)16(

2 2

1 4 32

2

710

azaa

aza

zy

zyx


CASE I .

00 4 za

2

4 32

710

zy

zyx

as number of equations are less than number of unknowns, hence the system has

infinite many solutions,

let z = t

78

720

710

434

2

tttx

ty

where ‘t’ is any real number.

CASE II

, -80but , -80z 4 a hence, there is no solution.

CASE III

1let ,4,4 aaa

5

1z

-315z-

41)41)(41( .3. Equatins

z

35

47

35

64

5

3

35

64

5

2

7

10

)(24

x

y

the system will have unique solution when a 4 and a -4

and for a=1 the solution is

. and ,51

3564

3547 zyx

NOTE: (i) a=-4, no solution,

(ii) a=4, infinite many solutions and

(iii)a 4, a -4, exactly one solution .


Example:8. What conditions must a, b, and c satisfy in order for the system of

equations

czyx

bzx

azyx

32

2

to be consistent.

Solution: The augmented matrix is

c

b

a

312

101

211

reducing it to reduced row echelon form

ac

ab

a

2110

110

211

R2-R1, R3-2R1

bac

ab

a

000

110

211

R3-R1

The system will be consistent if only if c – a - b = 0

or c = a + b

Thus the required condition for system to be consistent is

c = a + b.


INVERSE MATRIX

We next consider a method of solutions based on the inverse of the matrix of coefficients.

Matrix multiplication is a well defined operation, and the definition of a matrix inverse

follows from this operation. Suppose that matrix A has an inverse, denoted A-1.

Definition: AA-1 = A-1A = I, where I is the identity matrix.

For a 3-by-3 matrix I =

100

010

001

.

The following theorem is important to know: if you find one inverse – you find all of them!

Theorem: For each associative operation the inverse is unique.

Proof: Indirectly, suppose, that there are 2 inverses, A* and A-1.

A*=A*I=A*(A A-1)=(A*A) A-1 =I A-1= A-1

Calculating inverse matrix

Suppose we have:

531

532

211

A

333231

232221

131211

1

xxx

xxx

xxx

A

Substituting into the definition:

3

333231

232221

131211

1

100

010

001

531

532

211

E

xxx

xxx

xxx

AA


So, we need to solve 3 system of linear equations, having the same

coefficient matrix:

0

0

1

531

532

211

31

21

11

31

21

11

x

x

x

x

x

x

A ,

0

1

0

531

532

211

32

22

12

32

22

12

x

x

x

x

x

x

A ,

1

0

0

531

532

211

33

23

13

33

23

13

x

x

x

x

x

x

A

These identical coefficient matrices mean, that the steps during the Gauss-Jordan elimination

must be exactly the same for each solution, no matter, that the positions in the augmented

matrix for the constants are different:

1

0

0

,

0

1

0

,

0

0

1

0

0

1

5

5

2

3

3

1

1

2

1

,

0

1

0

5

5

2

3

3

1

1

2

1

,

1

0

0

5

5

2

3

3

1

1

2

1

.

1

0

1

3

5

2

2

3

1

0

2

1

,

0

1

0

3

5

2

2

3

1

0

2

1

,

1

0

0

3

5

2

2

3

1

0

2

1

.

SAME NEW COEFFICIENTS


1

0

0

0

1

0

0

0

1

5

5

2

3

3

1

1

2

1

)1()3()3(.

Matrix A IDENTITY Matrix I

1

0

0

0

1

0

1

0

1

3

5

2

2

3

1

0

2

1

)1(*2)2()2()1()3()3(

1

0

0

0

1

0

1

2

1

3

1

2

2

1

1

0

0

1

)2(*1)2(

1

0

0

0

1

0

1

2

1

3

1

2

2

1

1

0

0

1

)2(*2)3()3()2()1()1(

1

0

0

2

1

1

3

2

3

1

1

1

0

1

0

0

0

1

)3()2()2()3()1()1(

1

1

1

2

3

1

3

5

0

1

0

0

0

1

0

0

0

1


The inverse to A is:

123

135

110

Please check by multiplication.

Example (calculated in the lecture, here is a simplier calculation):

Find the inverse of matrix

521

213

132

The augmented matrix:

1

0

0

0

1

0

0

0

1

5-21

213

1-32

Changing 1st and last row:

001-132

010213

100-521

1st elimination step, zeros in the 1st column:

-2019-10

-31017-50

100-521


Changing 2nd and 3rd rows:

-31017-50

20-1-910

100-521

2nd elimination step: zeros in 2nd column (up and down):

71-5-2800

20-1-910

-3021301

Dividing last row by -28:

-1/4-1/285/28100

20-1-910

-3021301

3rd elimination step: zeros in 3rd column:

-1/4-1/285/28100

-1/4-9/2817/28010

1/413/28-9/28001

so the inverse is:

715

7917

7139

28

1

Check it by multiplication!


MATRIX INVERSE SUMMARY

Finding an inverse means to solve n systems of n linear equations,

respectively.

So. we have the nn 2 augmented matrix:

1. nn 2 kibővített mátrix:

1

0

0

0

1

0

0

0

1

2

1

2

22

12

1

21

11

nn

n

n

nn

n

a

a

a

a

a

a

a

a

a

EA

from this, applying Gauss-Jordan elimination we get:

nn

n

n

nn

nn

a

a

a

a

a

a

a

a

a

AI

2

1

2

22

12

1

21

11

1

1

0

0

0

1

0

0

0

1

Gaussian-, Jordan Elimination and Matrix Inversesb_novak/BANKI/gauss_ang.pdf · Gaussian-, Jordan...

Documents

Transcript of Gaussian-, Jordan Elimination and Matrix Inversesb_novak/BANKI/gauss_ang.pdf · Gaussian-, Jordan...