Gaussian Elimination

Carl Friedrich Gauss (1777-1855)

German mathematician and scientist,

contributed to number theory, statistics, algebra, analysis,differential geometry, geophysics, electrostatics, astronomy,

optics

1/33

Gaussian Elimination

Carl Friedrich Gauss (1777-1855)

German mathematician and scientist,contributed to number theory, statistics, algebra, analysis,

differential geometry, geophysics, electrostatics, astronomy,optics

1/33

Gaussian Elimination Method: This is a GEM of a methodto solve a system of linear equations.

Recall that a system of m linear equations in n unknownsx1, . . . , xn is of the form

a11x1 + a12x2 + . . . + a1nxn = b1

a21x1 + a22x2 + . . . + a2nxn = b2

. . . . . .

am1x1 + am2x2 + . . . + amnxn = bm

where aij (1 ≤ i ≤ m, 1 ≤ j ≤ n) and bi (1 ≤ i ≤ m)are known scalars.

Basic observation: Operations of three types on theseequations do not alter the solutions:

1. Interchanging two equations.2. Multiplying all the terms of an equation by a nonzero scalar.3. Adding to one equation a multiple of another equation.

2/33

Gaussian Elimination Method: This is a GEM of a methodto solve a system of linear equations.Recall that a system of m linear equations in n unknownsx1, . . . , xn is of the form

a11x1 + a12x2 + . . . + a1nxn = b1

a21x1 + a22x2 + . . . + a2nxn = b2

. . . . . .

am1x1 + am2x2 + . . . + amnxn = bm

where aij (1 ≤ i ≤ m, 1 ≤ j ≤ n) and bi (1 ≤ i ≤ m)are known scalars.

Basic observation: Operations of three types on theseequations do not alter the solutions:

1. Interchanging two equations.2. Multiplying all the terms of an equation by a nonzero scalar.3. Adding to one equation a multiple of another equation.

2/33

Gaussian Elimination Method: This is a GEM of a methodto solve a system of linear equations.Recall that a system of m linear equations in n unknownsx1, . . . , xn is of the form

a11x1 + a12x2 + . . . + a1nxn = b1

a21x1 + a22x2 + . . . + a2nxn = b2

. . . . . .

am1x1 + am2x2 + . . . + amnxn = bm

where aij (1 ≤ i ≤ m, 1 ≤ j ≤ n) and bi (1 ≤ i ≤ m)are known scalars.Basic observation: Operations of three types on theseequations do not alter the solutions:

1. Interchanging two equations.2. Multiplying all the terms of an equation by a nonzero scalar.3. Adding to one equation a multiple of another equation.

2/33

Gaussian Elimination Method: This is a GEM of a methodto solve a system of linear equations.Recall that a system of m linear equations in n unknownsx1, . . . , xn is of the form

a11x1 + a12x2 + . . . + a1nxn = b1

a21x1 + a22x2 + . . . + a2nxn = b2

. . . . . .

am1x1 + am2x2 + . . . + amnxn = bm

where aij (1 ≤ i ≤ m, 1 ≤ j ≤ n) and bi (1 ≤ i ≤ m)are known scalars.Basic observation: Operations of three types on theseequations do not alter the solutions:

1. Interchanging two equations.

2. Multiplying all the terms of an equation by a nonzero scalar.3. Adding to one equation a multiple of another equation.

2/33

Gaussian Elimination Method: This is a GEM of a methodto solve a system of linear equations.Recall that a system of m linear equations in n unknownsx1, . . . , xn is of the form

a11x1 + a12x2 + . . . + a1nxn = b1

a21x1 + a22x2 + . . . + a2nxn = b2

. . . . . .

am1x1 + am2x2 + . . . + amnxn = bm

where aij (1 ≤ i ≤ m, 1 ≤ j ≤ n) and bi (1 ≤ i ≤ m)are known scalars.Basic observation: Operations of three types on theseequations do not alter the solutions:

1. Interchanging two equations.2. Multiplying all the terms of an equation by a nonzero scalar.

3. Adding to one equation a multiple of another equation.

2/33

Gaussian Elimination Method: This is a GEM of a methodto solve a system of linear equations.Recall that a system of m linear equations in n unknownsx1, . . . , xn is of the form

a11x1 + a12x2 + . . . + a1nxn = b1

a21x1 + a22x2 + . . . + a2nxn = b2

. . . . . .

am1x1 + am2x2 + . . . + amnxn = bm

where aij (1 ≤ i ≤ m, 1 ≤ j ≤ n) and bi (1 ≤ i ≤ m)are known scalars.Basic observation: Operations of three types on theseequations do not alter the solutions:

1. Interchanging two equations.2. Multiplying all the terms of an equation by a nonzero scalar.3. Adding to one equation a multiple of another equation.

2/33

The above system of linear equations can be written inmatrix form as follows. a11 . . . a1n

......

am1 . . . amn

x1

...xn

=

b1...

bm

(∗)

or in short as Ax = b, where

A =

a11 . . . a1n...

...am1 . . . amn

, x =

x1...

xn

, b =

b1...

bm

.

The m× n matrix A = (aij) is called the coefficient matrix ofthe system. By a solution of (∗) we mean any choice ofx1, x2, . . . , xn which satisfies all the equations in thesystem.If each bi = 0, then the system is said to be homogeneous.Otherwise it is called an inhomogeneous system.

3/33

The above system of linear equations can be written inmatrix form as follows. a11 . . . a1n

......

am1 . . . amn

x1

...xn

=

b1...

bm

(∗)

or in short as Ax = b, where

A =

a11 . . . a1n...

...am1 . . . amn

, x =

x1...

xn

, b =

b1...

bm

.

The m× n matrix A = (aij) is called the coefficient matrix ofthe system. By a solution of (∗) we mean any choice ofx1, x2, . . . , xn which satisfies all the equations in thesystem.

If each bi = 0, then the system is said to be homogeneous.Otherwise it is called an inhomogeneous system.

3/33

The above system of linear equations can be written inmatrix form as follows. a11 . . . a1n

......

am1 . . . amn

x1

...xn

=

b1...

bm

(∗)

or in short as Ax = b, where

A =

a11 . . . a1n...

...am1 . . . amn

, x =

x1...

xn

, b =

b1...

bm

.

The m× n matrix A = (aij) is called the coefficient matrix ofthe system. By a solution of (∗) we mean any choice ofx1, x2, . . . , xn which satisfies all the equations in thesystem.If each bi = 0, then the system is said to be homogeneous.Otherwise it is called an inhomogeneous system.

3/33

All the known data in the system (∗) is captured in them × (n + 1) matrix

(A|b) :=

a11 . . . a1n...

...am1 . . . amn

∣∣∣∣∣∣∣b1...

bm

.

This is called the augmented matrix for the system.

Now the above three operations on the equations in thelinear system correspond to the following operations on therows of the augmented matrix:

(i) interchanging two rows,(ii) multiply a row by a nonzero scalar,(iii) adding a multiple of one row to another.

These are called elementary row operations.

Gaussian Elimination Method consists of reducing theaugmented matrix to a simpler matrix from which solutionscan be easily found. This reduction is by means ofelementary row operations.

4/33

All the known data in the system (∗) is captured in them × (n + 1) matrix

(A|b) :=

a11 . . . a1n...

...am1 . . . amn

∣∣∣∣∣∣∣b1...

bm

.

This is called the augmented matrix for the system.Now the above three operations on the equations in thelinear system correspond to the following operations on therows of the augmented matrix:

(i) interchanging two rows,(ii) multiply a row by a nonzero scalar,(iii) adding a multiple of one row to another.

These are called elementary row operations.

Gaussian Elimination Method consists of reducing theaugmented matrix to a simpler matrix from which solutionscan be easily found. This reduction is by means ofelementary row operations.

4/33

All the known data in the system (∗) is captured in them × (n + 1) matrix

(A|b) :=

a11 . . . a1n...

...am1 . . . amn

∣∣∣∣∣∣∣b1...

bm

.

This is called the augmented matrix for the system.Now the above three operations on the equations in thelinear system correspond to the following operations on therows of the augmented matrix:

(i) interchanging two rows,(ii) multiply a row by a nonzero scalar,(iii) adding a multiple of one row to another.

These are called elementary row operations.

Gaussian Elimination Method consists of reducing theaugmented matrix to a simpler matrix from which solutionscan be easily found. This reduction is by means ofelementary row operations.

4/33

Example 1 (A system with a unique solution):

x − 2y + z = 52x − 5y + 4z = −3

x − 4y + 6z = 10.

The augmented matrix for this system is the 3× 4 matrix 1 −2 12 −5 41 −4 6

∣∣∣∣∣∣5−310

The elementary row operations mentioned above will beperformed on the rows of this augmented matrix,

5/33

Example 1 (A system with a unique solution):

x − 2y + z = 52x − 5y + 4z = −3

x − 4y + 6z = 10.

The augmented matrix for this system is the 3× 4 matrix 1 −2 12 −5 41 −4 6

∣∣∣∣∣∣5−310

The elementary row operations mentioned above will beperformed on the rows of this augmented matrix,

5/33

First we add -2 times the first row to the second row. Then wesubtract the first row from the third row.

i.e.,

1 −2 10 −1 20 −2 5

∣∣∣∣∣∣5−13

5

The circled entry is the first nonzero entry in the first rowand all the entries below this are 0. Such a circled entry iscalled a pivot. This next step is called ‘sweeping’ a column.Here we repeat the process for the smaller matrix:

viz.(−1 2−2 5

∣∣∣∣ −135

)⇒

(-1 2

0 1

∣∣∣∣∣ −1331

)Put back the rows and columns that has been cut outearlier:

6/33

First we add -2 times the first row to the second row. Then wesubtract the first row from the third row.

i.e.,

1 −2 10 −1 20 −2 5

∣∣∣∣∣∣5−13

5

The circled entry is the first nonzero entry in the first rowand all the entries below this are 0. Such a circled entry iscalled a pivot. This next step is called ‘sweeping’ a column.Here we repeat the process for the smaller matrix:

viz.(−1 2−2 5

∣∣∣∣ −135

)⇒

(-1 2

0 1

∣∣∣∣∣ −1331

)Put back the rows and columns that has been cut outearlier:

6/33

First we add -2 times the first row to the second row. Then wesubtract the first row from the third row.

i.e.,

1 −2 10 −1 20 −2 5

∣∣∣∣∣∣5−13

5

The circled entry is the first nonzero entry in the first rowand all the entries below this are 0. Such a circled entry iscalled a pivot. This next step is called ‘sweeping’ a column.Here we repeat the process for the smaller matrix:

viz.(−1 2−2 5

∣∣∣∣ −135

)⇒

(-1 2

0 1

∣∣∣∣∣ −1331

)Put back the rows and columns that has been cut outearlier:

6/33

First we add -2 times the first row to the second row. Then wesubtract the first row from the third row.

i.e.,

1 −2 10 −1 20 −2 5

∣∣∣∣∣∣5−13

5

The circled entry is the first nonzero entry in the first rowand all the entries below this are 0. Such a circled entry iscalled a pivot. This next step is called ‘sweeping’ a column.Here we repeat the process for the smaller matrix:

viz.(−1 2−2 5

∣∣∣∣ −135

)

⇒

(-1 2

0 1

∣∣∣∣∣ −1331

)Put back the rows and columns that has been cut outearlier:

6/33

First we add -2 times the first row to the second row. Then wesubtract the first row from the third row.

i.e.,

1 −2 10 −1 20 −2 5

∣∣∣∣∣∣5−13

5

The circled entry is the first nonzero entry in the first rowand all the entries below this are 0. Such a circled entry iscalled a pivot. This next step is called ‘sweeping’ a column.Here we repeat the process for the smaller matrix:

viz.(−1 2−2 5

∣∣∣∣ −135

)⇒

(-1 2

0 1

∣∣∣∣∣ −1331

)

Put back the rows and columns that has been cut outearlier:

6/33

First we add -2 times the first row to the second row. Then wesubtract the first row from the third row.

i.e.,

1 −2 10 −1 20 −2 5

∣∣∣∣∣∣5−13

5

The circled entry is the first nonzero entry in the first rowand all the entries below this are 0. Such a circled entry iscalled a pivot. This next step is called ‘sweeping’ a column.Here we repeat the process for the smaller matrix:

viz.(−1 2−2 5

∣∣∣∣ −135

)⇒

(-1 2

0 1

∣∣∣∣∣ −1331

)Put back the rows and columns that has been cut outearlier:

6/33

1 −2 1

0 -1 2

0 0 1

∣∣∣∣∣∣∣5−1331

(∗)

The matrix represents the linear system:

x − 2y + z = 5−y + 2z = −13

z = 31

These can be solved successively by backward substitutionz = 31;y = 13 + 2z = 75;x = 5 + 2y − z = 124.

7/33

1 −2 1

0 -1 2

0 0 1

∣∣∣∣∣∣∣5−1331

(∗)

The matrix represents the linear system:

x − 2y + z = 5−y + 2z = −13

z = 31

These can be solved successively by backward substitutionz = 31;y = 13 + 2z = 75;x = 5 + 2y − z = 124.

7/33

1 −2 1

0 -1 2

0 0 1

∣∣∣∣∣∣∣5−1331

(∗)

The matrix represents the linear system:

x − 2y + z = 5−y + 2z = −13

z = 31

These can be solved successively by backward substitutionz = 31;

y = 13 + 2z = 75;x = 5 + 2y − z = 124.

7/33

1 −2 1

0 -1 2

0 0 1

∣∣∣∣∣∣∣5−1331

(∗)

The matrix represents the linear system:

x − 2y + z = 5−y + 2z = −13

z = 31

These can be solved successively by backward substitutionz = 31;y = 13 + 2z = 75;

x = 5 + 2y − z = 124.

7/33

1 −2 1

0 -1 2

0 0 1

∣∣∣∣∣∣∣5−1331

(∗)

The matrix represents the linear system:

x − 2y + z = 5−y + 2z = −13

z = 31

These can be solved successively by backward substitutionz = 31;y = 13 + 2z = 75;x = 5 + 2y − z = 124.

7/33

We can continue Gaussian elimination to simplify theaugmented matrix further. This is called the Gauss-JordanProcess.

Here, we ensure that all the pivots are equal to 1and moreover all the other entries in the column containingthe pivot are 0. In other words, we have 0’s not only belowbut also above the pivot. numbers.

Recall

1 −2 10 −1 20 0 1

∣∣∣∣∣∣5−1331

(∗)

(1) multiply the second row throughout by −1,(2) add twice third row to the second(3) then subtract the third row from the first(3) add twice the second row to the first

This gives

8/33

We can continue Gaussian elimination to simplify theaugmented matrix further. This is called the Gauss-JordanProcess. Here, we ensure that all the pivots are equal to 1and moreover all the other entries in the column containingthe pivot are 0. In other words, we have 0’s not only belowbut also above the pivot. numbers.

Recall

1 −2 10 −1 20 0 1

∣∣∣∣∣∣5−1331

(∗)

(1) multiply the second row throughout by −1,(2) add twice third row to the second(3) then subtract the third row from the first(3) add twice the second row to the first

This gives

8/33

We can continue Gaussian elimination to simplify theaugmented matrix further. This is called the Gauss-JordanProcess. Here, we ensure that all the pivots are equal to 1and moreover all the other entries in the column containingthe pivot are 0. In other words, we have 0’s not only belowbut also above the pivot. numbers.

Recall

1 −2 10 −1 20 0 1

∣∣∣∣∣∣5−1331

(∗)

(1) multiply the second row throughout by −1,(2) add twice third row to the second(3) then subtract the third row from the first(3) add twice the second row to the first

This gives

8/33

We can continue Gaussian elimination to simplify theaugmented matrix further. This is called the Gauss-JordanProcess. Here, we ensure that all the pivots are equal to 1and moreover all the other entries in the column containingthe pivot are 0. In other words, we have 0’s not only belowbut also above the pivot. numbers.

Recall

1 −2 10 −1 20 0 1

∣∣∣∣∣∣5−1331

(∗)

(1) multiply the second row throughout by −1,(2) add twice third row to the second(3) then subtract the third row from the first(3) add twice the second row to the first

This gives

8/33

⇒

1 0 00 1 00 0 1

∣∣∣∣∣∣∣1247531

⇒ This simple augmented matrix quickly gives the desiredsolution

x = 124, y = 75, z = 31.

It is useful to have a shorthand notation for the three typesof elementary row operations.Notation: Let Ri denote the i th row of a given matrix.

Operation NotationInterchange Ri and Rj Ri ↔ RjMultiply Ri by a (nonzero) scalar c cRiMultiply Rj by a scalar c and add to Ri Ri + cRj

9/33

⇒

1 0 00 1 00 0 1

∣∣∣∣∣∣∣1247531

⇒ This simple augmented matrix quickly gives the desired

solution

x = 124, y = 75, z = 31.

It is useful to have a shorthand notation for the three typesof elementary row operations.Notation: Let Ri denote the i th row of a given matrix.

Operation NotationInterchange Ri and Rj Ri ↔ RjMultiply Ri by a (nonzero) scalar c cRiMultiply Rj by a scalar c and add to Ri Ri + cRj

9/33

⇒

1 0 00 1 00 0 1

∣∣∣∣∣∣∣1247531

⇒ This simple augmented matrix quickly gives the desired

solution

x = 124, y = 75, z = 31.

It is useful to have a shorthand notation for the three typesof elementary row operations.Notation: Let Ri denote the i th row of a given matrix.

Operation NotationInterchange Ri and Rj Ri ↔ RjMultiply Ri by a (nonzero) scalar c cRiMultiply Rj by a scalar c and add to Ri Ri + cRj

9/33

Example 2 (A system with infinitely many solutions):

x − 2y + z − u + v = 52x − 5y + 4z + u − v = −3x − 4y + 6z + 2u − v = 10

⇒

1 −2 1 −1 12 −5 4 1 −11 −4 6 2 −1

∣∣∣∣∣∣5−310

We shall use the notation introduced above for the rowoperations

.R2 − 2R1R3 − R1−→

1 −2 1 −1 10 −1 2 3 −30 −2 5 3 −2

∣∣∣∣∣∣5−13

5

.

R3 − 2R2−→

1 −2 1 −1 10 −1 2 3 −30 0 1 −3 4

∣∣∣∣∣∣5−1331

10/33

Example 2 (A system with infinitely many solutions):

x − 2y + z − u + v = 52x − 5y + 4z + u − v = −3x − 4y + 6z + 2u − v = 10

⇒

1 −2 1 −1 12 −5 4 1 −11 −4 6 2 −1

∣∣∣∣∣∣5−310

We shall use the notation introduced above for the rowoperations

.R2 − 2R1R3 − R1−→

1 −2 1 −1 10 −1 2 3 −30 −2 5 3 −2

∣∣∣∣∣∣5−13

5

.

R3 − 2R2−→

1 −2 1 −1 10 −1 2 3 −30 0 1 −3 4

∣∣∣∣∣∣5−1331

10/33

Example 2 (A system with infinitely many solutions):

x − 2y + z − u + v = 52x − 5y + 4z + u − v = −3x − 4y + 6z + 2u − v = 10

⇒

1 −2 1 −1 12 −5 4 1 −11 −4 6 2 −1

∣∣∣∣∣∣5−310

We shall use the notation introduced above for the rowoperations

.R2 − 2R1R3 − R1−→

1 −2 1 −1 10 −1 2 3 −30 −2 5 3 −2

∣∣∣∣∣∣5−13

5

.

R3 − 2R2−→

1 −2 1 −1 10 −1 2 3 −30 0 1 −3 4

∣∣∣∣∣∣5−1331

10/33

Example 2 (A system with infinitely many solutions):

x − 2y + z − u + v = 52x − 5y + 4z + u − v = −3x − 4y + 6z + 2u − v = 10

⇒

1 −2 1 −1 12 −5 4 1 −11 −4 6 2 −1

∣∣∣∣∣∣5−310

We shall use the notation introduced above for the rowoperations

.R2 − 2R1R3 − R1−→

1 −2 1 −1 10 −1 2 3 −30 −2 5 3 −2

∣∣∣∣∣∣5−13

5

.R3 − 2R2−→

1 −2 1 −1 10 −1 2 3 −30 0 1 −3 4

∣∣∣∣∣∣5−1331

10/33

Example 2 (A system with infinitely many solutions):

x − 2y + z − u + v = 52x − 5y + 4z + u − v = −3x − 4y + 6z + 2u − v = 10

⇒

1 −2 1 −1 12 −5 4 1 −11 −4 6 2 −1

∣∣∣∣∣∣5−310

We shall use the notation introduced above for the rowoperations

.R2 − 2R1R3 − R1−→

1 −2 1 −1 10 −1 2 3 −30 −2 5 3 −2

∣∣∣∣∣∣5−13

5

.

R3 − 2R2−→

1 −2 1 −1 10 −1 2 3 −30 0 1 −3 4

∣∣∣∣∣∣5−1331

10/33

.−R2−→

1 −2 1 −1 10 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣51331

.R1 + 2R2−→

1 0 −3 −7 70 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣311331

.

R2 + 2R3R1 + 3R3−→

1 0 0 −16 190 1 0 −9 110 0 1 −3 4

∣∣∣∣∣∣1247531

The system of linear equations corresponding to the lastaugmented matrix is:

x = 124 + 16u − 19vy = 75 + 9u − 11vz = 31 + 3u − 4v .

11/33

.−R2−→

1 −2 1 −1 10 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣51331

.

R1 + 2R2−→

1 0 −3 −7 70 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣311331

.R2 + 2R3R1 + 3R3−→

1 0 0 −16 190 1 0 −9 110 0 1 −3 4

∣∣∣∣∣∣1247531

The system of linear equations corresponding to the lastaugmented matrix is:

x = 124 + 16u − 19vy = 75 + 9u − 11vz = 31 + 3u − 4v .

11/33

.−R2−→

1 −2 1 −1 10 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣51331

.

R1 + 2R2−→

1 0 −3 −7 70 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣311331

.

R2 + 2R3R1 + 3R3−→

1 0 0 −16 190 1 0 −9 110 0 1 −3 4

∣∣∣∣∣∣1247531

The system of linear equations corresponding to the lastaugmented matrix is:

x = 124 + 16u − 19vy = 75 + 9u − 11vz = 31 + 3u − 4v .

11/33

.−R2−→

1 −2 1 −1 10 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣51331

.

R1 + 2R2−→

1 0 −3 −7 70 1 −2 −3 30 0 1 −3 4

∣∣∣∣∣∣311331

.

R2 + 2R3R1 + 3R3−→

1 0 0 −16 190 1 0 −9 110 0 1 −3 4

∣∣∣∣∣∣1247531

The system of linear equations corresponding to the lastaugmented matrix is:

x = 124 + 16u − 19vy = 75 + 9u − 11vz = 31 + 3u − 4v .

11/33

We say that u and v are independent (or free) variablesand x , y , z are dependent (or basic) variables. The generalsolution is given by

(x , y , z,u, v)T

= (124 + 16t1 − 19t2,75 + 9t1 − 11t2,31 + 3t1 − 4t2, t1, t2)T

= (124,75,31,0,0)T + t1(16,9,3,1,0)T

+ t2(−19,−11,−4,0,1)T .

Note that (124,75,31,0,0) is a particular solution of theinhomogeneous system.v1 = (16,9,3,1,0) and v2 = (−19,−11,−4,0,1) aresolutions of the corresponding homogeneous system.(These two solutions are “linearly independent” and everyother solution of the homogeneous system is a linearcombination of these two solutions.)

12/33

We say that u and v are independent (or free) variablesand x , y , z are dependent (or basic) variables. The generalsolution is given by

(x , y , z,u, v)T

= (124 + 16t1 − 19t2,75 + 9t1 − 11t2,31 + 3t1 − 4t2, t1, t2)T

= (124,75,31,0,0)T + t1(16,9,3,1,0)T

+ t2(−19,−11,−4,0,1)T .

Note that (124,75,31,0,0) is a particular solution of theinhomogeneous system.v1 = (16,9,3,1,0) and v2 = (−19,−11,−4,0,1) aresolutions of the corresponding homogeneous system.(These two solutions are “linearly independent” and everyother solution of the homogeneous system is a linearcombination of these two solutions.)

12/33

We say that u and v are independent (or free) variablesand x , y , z are dependent (or basic) variables. The generalsolution is given by

(x , y , z,u, v)T

= (124 + 16t1 − 19t2,75 + 9t1 − 11t2,31 + 3t1 − 4t2, t1, t2)T

= (124,75,31,0,0)T + t1(16,9,3,1,0)T

+ t2(−19,−11,−4,0,1)T .

Note that (124,75,31,0,0) is a particular solution of theinhomogeneous system.

v1 = (16,9,3,1,0) and v2 = (−19,−11,−4,0,1) aresolutions of the corresponding homogeneous system.(These two solutions are “linearly independent” and everyother solution of the homogeneous system is a linearcombination of these two solutions.)

12/33

We say that u and v are independent (or free) variablesand x , y , z are dependent (or basic) variables. The generalsolution is given by

(x , y , z,u, v)T

= (124 + 16t1 − 19t2,75 + 9t1 − 11t2,31 + 3t1 − 4t2, t1, t2)T

= (124,75,31,0,0)T + t1(16,9,3,1,0)T

+ t2(−19,−11,−4,0,1)T .

Note that (124,75,31,0,0) is a particular solution of theinhomogeneous system.v1 = (16,9,3,1,0) and v2 = (−19,−11,−4,0,1) aresolutions of the corresponding homogeneous system.

(These two solutions are “linearly independent” and everyother solution of the homogeneous system is a linearcombination of these two solutions.)

12/33

We say that u and v are independent (or free) variablesand x , y , z are dependent (or basic) variables. The generalsolution is given by

(x , y , z,u, v)T

= (124 + 16t1 − 19t2,75 + 9t1 − 11t2,31 + 3t1 − 4t2, t1, t2)T

= (124,75,31,0,0)T + t1(16,9,3,1,0)T

+ t2(−19,−11,−4,0,1)T .

Note that (124,75,31,0,0) is a particular solution of theinhomogeneous system.v1 = (16,9,3,1,0) and v2 = (−19,−11,−4,0,1) aresolutions of the corresponding homogeneous system.(These two solutions are “linearly independent” and everyother solution of the homogeneous system is a linearcombination of these two solutions.)

12/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T .

Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0.

Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S.

Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

TheoremSuppose Ax = b is a system of linear equations whereA = ((aij)) is a m × n matrix and x = (x1, x2, . . . , xn)

T ,b = (b1,b2, . . . ,bm)

T . Suppose c = (c1, c2, . . . , cn)T is a

solution of Ax = b and S is the set of all solutions to theassociated homogeneous system Ax = 0. Then the set of allsolutions to Ax = b is c + S := {c + v|v ∈ S}.

Proof: Let r ∈ Rn be a solution of Ax = b. Then

A(r− c) = Ar− Ac = b− b = 0.

Hence r− c ∈ S. Thus r ∈ c + S.

Conversely, let v ∈ S. Then

A(c + v) = Ac + Av = b + 0 = b.

Hence c + v is a solution to Ax = b. 2

13/33

Example 3 (A system with no solution):

x − 5y + 4z = 3x − 5y + 3z = 6

2x − 10y + 13z = 5

⇒

1 −5 41 −5 32 −10 13

∣∣∣∣∣∣365

→ Apply Gauss Elimination Method to get 1 −5 4

0 0 −10 0 0

∣∣∣∣∣∣3314

→ The bottom row corresponds to the equation 0.z = 14.→ Hence the system has no solutions.

14/33

Example 3 (A system with no solution):

x − 5y + 4z = 3x − 5y + 3z = 6

2x − 10y + 13z = 5

⇒

1 −5 41 −5 32 −10 13

∣∣∣∣∣∣365

→ Apply Gauss Elimination Method to get 1 −5 40 0 −10 0 0

∣∣∣∣∣∣3314

→ The bottom row corresponds to the equation 0.z = 14.→ Hence the system has no solutions.

14/33

Example 3 (A system with no solution):

x − 5y + 4z = 3x − 5y + 3z = 6

2x − 10y + 13z = 5

⇒

1 −5 41 −5 32 −10 13

∣∣∣∣∣∣365

→ Apply Gauss Elimination Method to get 1 −5 4

0 0 −10 0 0

∣∣∣∣∣∣3314

→ The bottom row corresponds to the equation 0.z = 14.→ Hence the system has no solutions.

14/33

Example 3 (A system with no solution):

x − 5y + 4z = 3x − 5y + 3z = 6

2x − 10y + 13z = 5

⇒

1 −5 41 −5 32 −10 13

∣∣∣∣∣∣365

→ Apply Gauss Elimination Method to get 1 −5 4

0 0 −10 0 0

∣∣∣∣∣∣3314

→ The bottom row corresponds to the equation 0.z = 14.

→ Hence the system has no solutions.

14/33

Example 3 (A system with no solution):

x − 5y + 4z = 3x − 5y + 3z = 6

2x − 10y + 13z = 5

⇒

1 −5 41 −5 32 −10 13

∣∣∣∣∣∣365

→ Apply Gauss Elimination Method to get 1 −5 4

0 0 −10 0 0

∣∣∣∣∣∣3314

→ The bottom row corresponds to the equation 0.z = 14.→ Hence the system has no solutions.

14/33

STEPS IN GAUSSIAN ELIMINATIONStage 1: Forward Elimination Phase

The basic idea is to reduce the augmented matrix [A|b] byelementary row operations to [A′|c], where A′ is simple, or moreprecisely, in REF. This can always be achieved by the GaussianElimination Algorithm, which consists of the following steps.1. Search the first column of [A|b] from the top to the bottom for thefirst non-zero entry, and then if necessary, the second column (thecase where all the coefficients corresponding to the first variable arezero), and then the third column, and so on. The entry thus found iscalled the current pivot.2. Interchange, if necessary, the row containing the current pivot withthe first row.3. Keeping the row containing the pivot (that is, the first row) un-touched, subtract appropriate multiples of the first row from all theother rows to obtain all zeroes below the current pivot in its column.4. Repeat the preceding steps on the submatrix consisting of all thoseelements which are below and to the right of the current pivot.5. Stop when no further pivot can be found.

15/33

REF

The m × n coefficient matrix A of the linear system Ax = b isthus reduced to an (m × n) matrix A′ in row echelon form andso the augmented matrix [A|b] becomes [A′|c], which looks like

0 . . . p1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ . . . ∗ c10 . . . 0 . . . p2 ∗ ∗ ∗ ∗ ∗ . . . ∗ c20 . . . 0 0 0 . . . p3 ∗ ∗ ∗ . . . ∗ c3...

......

... 0 0 . . ....

......

...0 . . . 0 0 0 0 0 0 pr ∗ . . . ∗ cr0 . . . 0 0 0 0 0 0 0 0 . . . 0 cr+1...

......

......

......

......

......

0 . . . 0 0 0 0 0 0 0 0 . . . 0 cm

.

The entries denoted by ∗ and the ci ’s are real numbers; they may ormay not be zero. The pi ’s denote the pivots; they are non-zero. Notethat there is exactly one pivot in each of the first r rows of U and thatany column of U has at most one pivot. Hence r ≤ m and r ≤ n.

16/33

REF

The m × n coefficient matrix A of the linear system Ax = b isthus reduced to an (m × n) matrix A′ in row echelon form andso the augmented matrix [A|b] becomes [A′|c], which looks like

0 . . . p1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ . . . ∗ c10 . . . 0 . . . p2 ∗ ∗ ∗ ∗ ∗ . . . ∗ c20 . . . 0 0 0 . . . p3 ∗ ∗ ∗ . . . ∗ c3...

......

... 0 0 . . ....

......

...0 . . . 0 0 0 0 0 0 pr ∗ . . . ∗ cr0 . . . 0 0 0 0 0 0 0 0 . . . 0 cr+1...

......

......

......

......

......

0 . . . 0 0 0 0 0 0 0 0 . . . 0 cm

.

The entries denoted by ∗ and the ci ’s are real numbers; they may ormay not be zero.

The pi ’s denote the pivots; they are non-zero. Notethat there is exactly one pivot in each of the first r rows of U and thatany column of U has at most one pivot. Hence r ≤ m and r ≤ n.

16/33

REF

The m × n coefficient matrix A of the linear system Ax = b isthus reduced to an (m × n) matrix A′ in row echelon form andso the augmented matrix [A|b] becomes [A′|c], which looks like

0 . . . p1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ . . . ∗ c10 . . . 0 . . . p2 ∗ ∗ ∗ ∗ ∗ . . . ∗ c20 . . . 0 0 0 . . . p3 ∗ ∗ ∗ . . . ∗ c3...

......

... 0 0 . . ....

......

...0 . . . 0 0 0 0 0 0 pr ∗ . . . ∗ cr0 . . . 0 0 0 0 0 0 0 0 . . . 0 cr+1...

......

......

......

......

......

0 . . . 0 0 0 0 0 0 0 0 . . . 0 cm

.

The entries denoted by ∗ and the ci ’s are real numbers; they may ormay not be zero. The pi ’s denote the pivots; they are non-zero.

Notethat there is exactly one pivot in each of the first r rows of U and thatany column of U has at most one pivot. Hence r ≤ m and r ≤ n.

16/33

REF

The m × n coefficient matrix A of the linear system Ax = b isthus reduced to an (m × n) matrix A′ in row echelon form andso the augmented matrix [A|b] becomes [A′|c], which looks like

0 . . . p1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ . . . ∗ c10 . . . 0 . . . p2 ∗ ∗ ∗ ∗ ∗ . . . ∗ c20 . . . 0 0 0 . . . p3 ∗ ∗ ∗ . . . ∗ c3...

......

... 0 0 . . ....

......

...0 . . . 0 0 0 0 0 0 pr ∗ . . . ∗ cr0 . . . 0 0 0 0 0 0 0 0 . . . 0 cr+1...

......

......

......

......

......

0 . . . 0 0 0 0 0 0 0 0 . . . 0 cm

.

The entries denoted by ∗ and the ci ’s are real numbers; they may ormay not be zero. The pi ’s denote the pivots; they are non-zero. Notethat there is exactly one pivot in each of the first r rows of U and thatany column of U has at most one pivot.

Hence r ≤ m and r ≤ n.

16/33

REF

The m × n coefficient matrix A of the linear system Ax = b isthus reduced to an (m × n) matrix A′ in row echelon form andso the augmented matrix [A|b] becomes [A′|c], which looks like

0 . . . p1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ . . . ∗ c10 . . . 0 . . . p2 ∗ ∗ ∗ ∗ ∗ . . . ∗ c20 . . . 0 0 0 . . . p3 ∗ ∗ ∗ . . . ∗ c3...

......

... 0 0 . . ....

......

...0 . . . 0 0 0 0 0 0 pr ∗ . . . ∗ cr0 . . . 0 0 0 0 0 0 0 0 . . . 0 cr+1...

......

......

......

......

......

0 . . . 0 0 0 0 0 0 0 0 . . . 0 cm

.

The entries denoted by ∗ and the ci ’s are real numbers; they may ormay not be zero. The pi ’s denote the pivots; they are non-zero. Notethat there is exactly one pivot in each of the first r rows of U and thatany column of U has at most one pivot. Hence r ≤ m and r ≤ n.

16/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations)

and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1,

then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .

If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable;

otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the numberof equations) and cr+k 6= 0 for some k ≥ 1, then the (r + k)throw corresponds to the self-contradictory equation 0 = cr+k

and so the system has no solutions (inconsistent system) .If (i) r = m or (ii) r < m and cr+k = 0 for all k ≥ 1, then thereexists a solution of the system (consistent system) .

(Basic & Free Variables)If the jth column of U contains a pivot, then xj is called a basicvariable; otherwise xj is called a free variable.

In fact, there are n − r free variables , where n is the numberof columns (unknowns) of A (and hence of U).

17/33

Stage 2: (Back Substitution Phase)

In the case of a consistent system, if xj is a free variable, then itcan be set equal to a parameter sj which can assume arbitraryvalues.

If xj is a basic variable, then we solve for xj in terms ofxj+1, . . . , xm, starting from the last basic variable and workingour way up row by row.

Remark: The general solution of a consistent system of mequations in n unknowns will involve n − r free variables or freeparameters (often denoted by t1, t2, . . . or s1, s2, . . . ), where r isthe number of pivots in a REF of the coefficient matrix. Thenumbers r and n − r associated with the matrix A are importantquantities and deserve a name. Thus we definerank(A) = number of non-zero rows in REF of A

nullity(A) = number of free variables in the solution of AX = 0.

Since the homogeneous system Ax = 0 is always consistent,we see that nullity(A) = n−rank(A).

18/33

Stage 2: (Back Substitution Phase)

In the case of a consistent system, if xj is a free variable, then itcan be set equal to a parameter sj which can assume arbitraryvalues.If xj is a basic variable, then we solve for xj in terms ofxj+1, . . . , xm, starting from the last basic variable and workingour way up row by row.

Remark: The general solution of a consistent system of mequations in n unknowns will involve n − r free variables or freeparameters (often denoted by t1, t2, . . . or s1, s2, . . . ), where r isthe number of pivots in a REF of the coefficient matrix. Thenumbers r and n − r associated with the matrix A are importantquantities and deserve a name. Thus we definerank(A) = number of non-zero rows in REF of A

nullity(A) = number of free variables in the solution of AX = 0.

Since the homogeneous system Ax = 0 is always consistent,we see that nullity(A) = n−rank(A).

18/33

Stage 2: (Back Substitution Phase)

In the case of a consistent system, if xj is a free variable, then itcan be set equal to a parameter sj which can assume arbitraryvalues.If xj is a basic variable, then we solve for xj in terms ofxj+1, . . . , xm, starting from the last basic variable and workingour way up row by row.

Remark: The general solution of a consistent system of mequations in n unknowns will involve n − r free variables or freeparameters (often denoted by t1, t2, . . . or s1, s2, . . . ), where r isthe number of pivots in a REF of the coefficient matrix. Thenumbers r and n − r associated with the matrix A are importantquantities and deserve a name. Thus we definerank(A) = number of non-zero rows in REF of A

nullity(A) = number of free variables in the solution of AX = 0.

Since the homogeneous system Ax = 0 is always consistent,we see that nullity(A) = n−rank(A).

18/33