Section 1.1: Systems of Linear Equations

A linear equation:a1x1 + a2x2 + ⋯ + anxn = b

EXAMPLE:

4x1 − 5x2 + 2 = x1 and x2 = 2 6 − x1 + x3

↓ ↓rearranged rearranged

↓ ↓3x1 − 5x2 = −2 2x1 + x2 − x3 = 2 6

Not linear:4x1 − 6x2 = x1x2 and x2 = 2 x1 − 7

A system of linear equations (or a linear system):A collection of one or more linear equations involving the sameset of variables, say, x1,x2, . . . ,xn.

A solution of a linear system:A list s1, s2, . . . , sn of numbers that makes each equation in thesystem true when the values s1, s2, . . . , sn are substituted forx1,x2, . . . ,xn, respectively.

1

EXAMPLE Two equations in two variables:

x1 + x2 = 10

−x1 + x2 = 0

x1 − 2x2 = −3

2x1 − 4x2 = 8

2 4 6 8 10x1

2

4

6

8

10x2

−1 1 2 3 4 5x1

−2

−1

1

2

3

4

x2

one unique solution no solution

x1 + x2 = 3

−2x1 − 2x2 = −6

−1 1 2 3 4 5x1

−2

−1

1

2

3

4

x2

infinitely many solutions

BASIC FACT: A system of linear equations has either

(i) exactly one solution (consistent) or

(ii) infinitely many solutions (consistent) or

(iii) no solution (inconsistent).

2

EXAMPLE: Three equations in three variables. Each equationdetermines a plane in 3-space.

i) The planes intersect in ii) The planes intersect in one

one point. (one solution) line. (infinitely many solutions)

iii) There is not point in common

to all three planes. (no solution)

3

The solution set:● The set of all possible solutions of a linear system.

Equivalent systems:● Two linear systems with the same solution set.

STRATEGY FOR SOLVING A SYSTEM:● Replace one system with an equivalent system that is

easier to solve.

EXAMPLE:x1 − 2x2 = −1

−x1 + 3x2 = 3

x1 − 2x2 = −1

x2 = 2

x1 = 3

x2 = 2

4

−10 −5 5 10x1

−4

−2

2

4

x2

−10 −5 5 10x1

−4

−2

2

4

x2

x1 − 2x2 = −1

−x1 + 3x2 = 3

x1 − 2x2 = −1

x2 = 2

−10 −5 5 10

−4

−2

2

4

x1 = 3

x2 = 2

5

Matrix Notation

x1 − 2x2 = −1

−x1 + 3x2 = 3

1 −2

−1 3

(coefficient matrix)

x1 − 2x2 = −1

−x1 + 3x2 = 3

1 −2 −1

−1 3 3

(augmented matrix)

↓

x1 − 2x2 = −1

x2 = 2

1 −2 −1

0 1 2

↓x1 = 3

x2 = 2

1 0 3

0 1 2

6

Elementary Row Operations:1. (Replacement) Add one row to a multiple of another row.2. (Interchange) Interchange two rows.3. (Scaling) Multiply all entries in a row by a nonzero constant.

Row equivalent matrices: Two matrices where one matrix canbe transformed into the other matrix by a sequence of elementaryrow operations.

Fact about Row Equivalence: If the augmented matrices of twolinear systems are row equivalent, then the two systems have thesame solution set.

7

EXAMPLE:

x1 − 2x2 + x3 = 0

2x2 − 8x3 = 8

−4x1 + 5x2 + 9x3 = −9

1 −2 1 0

0 2 −8 8

−4 5 9 −9

x1 − 2x2 + x3 = 0

2x2 − 8x3 = 8

− 3x2 + 13x3 = −9

1 −2 1 0

0 2 −8 8

0 −3 13 −9

x1 − 2x2 + x3 = 0

x2 − 4x3 = 4

− 3x2 + 13x3 = −9

1 −2 1 0

0 1 −4 4

0 −3 13 −9

x1 − 2x2 + x3 = 0

x2 − 4x3 = 4

x3 = 3

1 −2 1 0

0 1 −4 4

0 0 1 3

x1 − 2x2 = −3

x2 = 16

x3 = 3

1 −2 0 −3

0 1 0 16

0 0 1 3

8

x1 = 29

x2 = 16

x3 = 3

1 0 0 29

0 1 0 16

0 0 1 3

Solution: 29,16,3

Check: Is 29,16,3 a solution of the original system?

x1 − 2x2 + x3 = 0

2x2 − 8x3 = 8

−4x1 + 5x2 + 9x3 = −9

29 − 216 + 3 = 29 − 32 + 3 = 0216 − 83 = 32 − 24 = 8

−429 + 516 + 93 = −116 + 80 + 27 = −9

9

Two Fundamental Questions (Existence and Uniqueness)

1) Is the system consistent; (i.e. does a solution exist?)

2) If a solution exists, is it unique? (i.e. is there one & only onesolution?)

EXAMPLE: Is this system consistent?

x1 − 2x2 + x3 = 0

2x2 − 8x3 = 8

−4x1 + 5x2 + 9x3 = −9

In the last example, this system was reduced to the triangularform:

x1 − 2x2 + x3 = 0

x2 − 4x3 = 4

x3 = 3

1 −2 1 0

0 1 −4 4

0 0 1 3

This is sufficient to see that the system is consistent and unique.Why?

10

EXAMPLE: Is this system consistent?

3x2 − 6x3 = 8

x1 − 2x2 + 3x3 = −1

5x1 − 7x2 + 9x3 = 0

0 3 −6 8

1 −2 3 −1

5 −7 9 0

Solution: Row operations produce:

0 3 −6 8

1 −2 3 −1

5 −7 9 0

→1 −2 3 −1

0 3 −6 8

0 3 −6 5

→1 −2 3 −1

0 3 −6 8

0 0 0 −3

Equation notation of triangular form:

x1 − 2x2 + 3x3 = −1

3x2 − 6x3 = 8

0x3 = −3 ← Never true

The original system is inconsistent!

11

EXAMPLE: For what values of h will the following system beconsistent?

3x1 − 9x2 = 4

−2x1 + 6x2 = h

Solution: Reduce to triangular form.

3 −9 4

−2 6 h→ 1 −3 4

3

−2 6 h→ 1 −3 4

3

0 0 h + 83

The second equation is 0x1 + 0x2 = h + 83 . System is consistent

only if h + 83 = 0 or h = −8

3 .

12

+

−

0 V

σ1

σ2

σ3σ5

σ4

V

V1

V2

Figure 1: A simple resitor network.

Why do we care?

Example 3 (Resistor Networks). Consider a network of resistors; According to Kirkoff’s(conservation of charge) and Ohm’s law (V = IR for I = σV where σ = 1/R) we have,given V, that V1 and V2 satisfy the equations;

σ1 (V1 − 0) + σ2 (V1 − V ) + σ3 (V1 − V2) = 0

σ2 (V2 − 0) + σ3 (V2 − V1) + σ5 (V2 − V ) = 0.

9

This is equivalent to the system of equations;

(σ1 + σ2 + σ3)V1 − σ3V2 = σ2V

−σ3V1 + (σ2 + σ3 + σ5)V2 = σ5V.

We would certainly like to know that these equations have unique solutions!

10

Example 4 (Equilibrium Heat Distribution). A numerical approximation to the heat distributionon as steel plate. The temperature at an interior node is the avergage of its nearest neighbors(Newton’s law of cooling!)For example

t1 =1

4(0 + 20 + t2 + t4)

t5 =1

4(t2 + t4 + t6 + t8)

...

We will get 9 equations in 9 unknowns.

11

Example 5 (Discrete Heat Equation). If we are not at equilibrium in the above example, thenwe have to solve the system of linear differential equations;

d

dstk (s) = Avg Temp. of Neighbors − tk (s)

so for example;

d

dst1 (s) =

1

4(0 + 20 + t2 (s) + t4 (s))− t1 (s)

d

dst5 (s) =

1

4(t2 (s) + t4 (s) + t6 (s) + t8 (s))− t5 (s)

...

12

m m

kk k

x1 x2

Figure 2: A simple spring and mass system.fs

Example 6 (Oscillators = Wave Equations)). Consider the following spring mass system;

mx1 (t) = −kx1 (t) + k (x2 (t)− x1 (t)) = −2kx1 (t) + kx2 (t)

mx2 (t) = −kx2 (t)− k (x2 (t)− x1 (t)) = kx1 − 2kx2.

13

Example 7 (Markov Chains). Let

P =

0 1/2 1/21/3 1/3 1/31/4 0 3/4

where Pij is the probability of jumping from site i to j. We would like to know what are thedynamics and equilibrium properties of this system.

14

1 2

3

1/3

1/3

0

1/2

1/21/4

3/4

1/3

Figure 3: A jump diagram for a Markov chain which we study the long run behavior.

15

1. Differential calculus – derivative is the best linear approximation

2. Integral calculus – determinants are used to compute areas, volumes, etc.

3. Least squares fitting.

4. Principle component analysis (Singular Value Decomposition) (NETFLIX)

5. Optimization problems – linear programming.

6. Google Page Rank – see MatLab assignment.

7. Linear control theory – how to make a plane fly.

8. Filtering theory – finding a signal in the noise

9. Quantum mechanics = matrix mechanics.

10. Computer graphics and rigid body mechanics

11. Image/Data compression

12. Data Encryption algorithms.

13. Tensors; stress, metric, field strengths, etc. – used in almost all physical theories.

16

Bruce K. Driver

Math 20F (Linear Algebra) Fall 2010

September 22, 2010 File:20F.tex

Section 1.2: Row Reduction and Echelon Forms

Echelon form (or row echelon form):1. All nonzero rows are above any rows of all zeros.

2. Each leading entry (i.e. left most nonzero entry) of a row isin a column to the right of the leading entry of the row aboveit.

3. All entries in a column below a leading entry are zero.

EXAMPLE: Echelon forms

(a)

■ ∗ ∗ ∗ ∗

0 ■ ∗ ∗ ∗

0 0 0 0 0

0 0 0 0 0

(b)

■ ∗ ∗

0 ■ ∗

0 0 ■0 0 0

(c)

0 ■ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

0 0 0 ■ ∗ ∗ ∗ ∗ ∗ ∗ ∗

0 0 0 0 ■ ∗ ∗ ∗ ∗ ∗ ∗

0 0 0 0 0 0 0 ■ ∗ ∗ ∗

0 0 0 0 0 0 0 0 ■ ∗ ∗

Reduced echelon form: Add the following conditions toconditions 1, 2, and 3 above:

4. The leading entry in each nonzero row is 1.

5. Each leading 1 is the only nonzero entry in its column.

1

EXAMPLE (continued):

Reduced echelon form :

0 1 ∗ 0 0 ∗ ∗ 0 0 ∗ ∗

0 0 0 1 0 ∗ ∗ 0 0 ∗ ∗

0 0 0 0 1 ∗ ∗ 0 0 ∗ ∗

0 0 0 0 0 0 0 1 0 ∗ ∗

0 0 0 0 0 0 0 0 1 ∗ ∗

Theorem 1 (Uniqueness of The Reduced Echelon Form):

Each matrix is row-equivalent to one and only onereduced echelon matrix.

2

2

09/30/2010

Example 2.1. Find the general solution to the system of equations whose augmented matrix is

A =

0 −3 −6 4

... 9

−1 −2 −1 3... 1

−2 −3 0 3... −1

1 4 5 −9... −7

∼

1 0 −3 0... 5

0 1 2 0... −3

0 0 0 1... 0

0 0 0 0... 0

.

The pivot collumns are at 1, 2, and 4. The free variable is x3. Thus the general solution is (x1, x2, x3, x4) such that

x4 = 0,

x2 = −3− 2x3 where x3 is free.

x1 = 5 + 3x3

8 2 09/30/2010

Example 2.2. Find the general solution to the system of equations whose augmented matrix is

B =

0 3 −6 6 4

... −5

3 −7 8 −5 8... 9

3 −9 12 −9 6... 15

∼

1 0 −2 3 0... −24

0 1 −2 2 0... −7

0 0 0 0 1... 4

.The pivot collumns are at 1, 2, and 5. The free variables are then x3 and x4. Thus the general solution is (x1, x2, x3, x4, x5) such that

x5 = 4,

x2 = −7 + 2x3 − 2x4 where x3 and x4 are free.

x1 = −24 + 2x3 − 3x4

Page: 8 job: 20F_2013 macro: svmonob.cls date/time: 30-Sep-2013/10:10

2 09/30/2010 9

Example 2.3. Suppose now B is an augmented matrix which is row equivalent to1 0 −2 3 0

... −24

0 1 −2 2 0... −7

0 0 0 0 0... 4

in which case the bottom row is shorthand for the equation,

0x1 + 0x2 + 0x3 + 0x4v + 0x5 = 4

which has no solutions. Hence the original system of equaitons is inconsistent.

Page: 9 job: 20F_2013 macro: svmonob.cls date/time: 30-Sep-2013/10:10