[email protected] ENGR-25_Linear_Equations-1.ppt 1 Bruce Mayer, PE Engineering/Math/Physics...

[email protected] • ENGR-25_Linear_Equations-1.ppt1

Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods

Bruce Mayer, PELicensed Electrical & Mechanical Engineer

[email protected]

Engr/Math/Physics 25

Chp8 LinearAlgebraic

Eqns-2



Learning Goals

Define Linear Algebraic Equations Solve Systems of Linear Equation by

Hand using• Gaussian Elimination• Cramer’s Method

Distinguish between Equation System Conditions: Exactly Determined, Overdetermined, Underdetermined

Use MATLAB to Solve Systems of Eqns



Cramer’s Method for Eqn Sys

Solves equations using determinants. Gives insight into the

• existence and uniqueness of solutions– Identifies SINGULAR (a.k.a. Divide by Zero)

Systems

• effects of numerical inaccuracy– Identifies ILL-CONDITIONED

(a.k.a. Stiff) Systems



Cramer’s Method – Illustrated-1

Cramer’s Method can Solve “Square” Systems; • i.e., [No. Eqns] =

[No. Unknowns]

Consider Sq Sys

502248

3263

3312921

zyx

zyx

zyx

Calc Cramer’s Determinant, Dc

• Also Called the “Characteristic” or “Denominator” Determinant

Dc Determinant of the Coefficients

2248

263

12921

cD




Now, to Find The Individual Solns, Sub The Constraint Vector for the Variable Coefficients and Compute the Determinant for Each unknown, Dk

In this Example Find Dx, Dy, Dz as

22450

263

12933

xD

502248

3263

3312921

zyx

zyx

zyx

22508

233

123321

yD

5048

363

33921

zD




Once We’ve Calculated all these Determinants, The Rest is Easy

These Eqns Ilustrate the most UseFul Feature of Cramer’s Method

c

x

D

Dx

c

y

D

Dy

c

z

D

Dz

Dc appears in all THREE

Denominators




Since – However, We can ANTICIPATE Problems if |Dc| << than the SMALLEST Coefficient

Completing the Example

c

k

D

Dk

Can ID “Condition” by Calculating Dc

• SINGULAR Systems → Dc = 0

• ILL-CONDITIONED Systems → Dc = “Small”– Small is technically

relative to the Dk

502248

3263

3312921

zyx

zyx

zyx



Cramer’s Method – Illustrated-6 Calc the

Determinants First Recall

The SIGN pattern for Determinants

48

63112

228

2319

224

26121

cD

Find Dc

1146

7207382604

c

c

D

D

6*84*312

2*822*39

4*222*621

cD

Dc is LARGE → WELL Conditioned System



Cramer’s Method – Illustrated-7 Solve using MATLAB’s det Function

>> EqnSys = [21 -9 -12 -33; -3 6 -2 3; -8 -4 22 50];

>> Dc = det(EqnSys(:,1:3))Dc = 1146

>> Dx = det([EqnSys(:,4),EqnSys(:,2:3)])

Dx = 1146

>> Dy = det([EqnSys(:,1),EqnSys(:,4), EqnSys(:,3)])Dy = 2292

All Row Elements of Cols 1-3

All Row Elements of Cols 4, 2-3

All Row Elements of Cols 1, 4, 3



Cramer’s Method – Illustrated-8 Solve using MATLAB’s det Function

>> Dz = det([EqnSys(:,1:2),EqnSys(:,4)])Dz = 3438

>> x = Dx/Dcx = 1

>> y = Dy/Dcy = 2

>> z = Dz/Dcz = 3

All Row Elements of Cols 1-2, 4



Cramer vs Homogenous: 0bAx

In general, for a set of HOMOGENEOUS linear algebraic equations that contains the same number of equations as unknowns• a nonzero solution exists only if the

set is singular; that is, if Cramer’s determinant is zero

• furthermore, the solution is not unique.• If Cramer’s determinant is not zero, the

homogeneous set has a zero solution; that is, all the unknowns are zero



Cramer’s Rule Summary

Cramer’s determinant gives some insight into ill-conditioned problems, which are close to being singular.

A Cramer’s determinant close to zero indicates an ill-conditioned problem.



UnderDetermined Systems An UNDERdetermined system does not

contain enough information to solve for ALL of the unknown variables • Usually because it has fewer equations

than unknowns. In this case an INFINITE number of

solutions can exist, with one or more of the unknowns dependent on the remaining unknowns.• For such systems the Matrix-Inverse

and Cramer’s methods will NOT work



UnderDetermined Example-1

A simple UnderDetermined system is the equation 63 yx

All we can do is solve for one of the unknowns in terms of the other; for example, x = 6 – 3y OR y = −x/3 + 2 • An INFINITE number of (x,y) solutions

satisfy this equation



UnderDetermined Example-2

When there are more Unknowns than Equations, the LEFT-DIVISION method will give a solution with some of the unknowns set equal to ZERO• For Example

which corresponds to • x = 0 • y = 2

>>A = [1, 3]; b = 6;>>solution = A\bsolution =

02

63163

y

xyx



More UnderDetermined Systems

An infinite number of solutions might exist EVEN when the number of equations EQUALS the number of knowns• Predict by Cramer as: 0det cDA

For such systems the Matrix Inverse method and Cramer’s method will also NOT work• MATLAB’s left-division method generates

an error message warning us that the matrix A is singular



Minimum Norm Solution

When det(A) = 0, We can use the PSEUDOINVERSE method to find ONE Solution, x, such that the Euclidean (or Pythagorean) Length of x is MINIMIZED

In MATLAB:

223

22

21min nxxxx x

MATLAB will return the MINIMUM NORM SOLUTION →

x = pinv(A)*b



Existence and Uniqueness

The set Ax = b with m equations and n unknowns has solutions if and only if

1rankrank AbA Rank[A] is the maximum number of

LINEARLY INDEPENDENT rows of A• Linear Independence → No Row of A

is a SCALAR multiple of ANY OTHER Combinations of Rows



An INconsistent Example

Consider Ax = b Since The Ranks are Unequal → this system of equations is NOT solvable

Graphically

5

4

42

21

2

1

x

x

ERO: Multiply the 1st row by −2 and add to the 2nd row

00

21Rank[A]=1

Rank[Ab]=2

34

0

2

0

1



Existence and Uniqueness

Recall Rank for m-Eqns & n-Unknwns

1rankrank AbA Now Let r = rank[A]

• If condition (1) is satisfied and if r = n, then the solution is unique

• If condition (1) is satisfied but r < n, an infinite number of solutions exists and – r unknown variables can be expressed as linear

combinations of the other n−r unknown variables, whose values are ARBITRARY



Homogenous Case The homogeneous set Ax = 0 is a

special case in which b = 0 For this case rank[A] = rank[Ab] always,

and thus the system in all cases has the trivial solution x = 0

A nonzero solution, in which at least one unknown is nonzero, exists if and only if rank[A] < n (n No. Unknowns)

If m < n, the homogeneous set always has a nonzero solution



ILLconditioned Systems An ill-conditioned set of equations is a

set that is CLOSE to being SINGULAR The ill-conditioned status depends on

the ACCURACY with which the solution calculations are made• When the internal numerical accuracy used

by MATLAB is INsufficient to obtain a solution, then MATLAB prints a message to warn you that the matrix is close to singular and that the results might be INaccurate



OVERdetermined Systems - 1 By Contrast an OVERdetermined

system is a set of equations that has MORE independent equations than unknowns

For such a system the Matrix Inverse method and Cramer’s method will NOT work because the A matrix is not square

However, SOME overdetermined systems have exact solutions, and they can be obtained with the left division method x = A\b



OVERdetermined Systems - 2 For OTHER OverDetermined systems,

NO exact solution exists.• In some of these cases, the left-division

method does not yield an answer, while in other cases the left-division method gives an answer that satisfies the equation set only in a “LEAST SQUARES” sense– When MATLAB gives an answer to an

overdetermined set, it does NOT tell us whether the answer is Exact or Least-Squares in NatureWe need to check the ranks of A and [Ab] to

know whether the answer is the exact solution.



Example Under/Over Determined% Bruce Mayer, PE * 27Feb08% ENGR25 * Under/OverDetermined Linear Systems% file = Over_Under_Lin_Sys_Example_0904.m% Solve Under & Over Determined Systems for [x1; x2; x3; x4]%% An UNDERdetermined case => 3-Eqns in 4-UnknwnsAu = [2 -3 7 4; -9 2 -2 -7; 7 6 -5 -4]bu = [5; -9; 7]disp('Showing UNDERdetermined solution & check - Hit Any Key to continue')xu = Au\bubu_chk = Au*xupause%% An OVERdetermined case => 5-Eqns in 4-UnknwnsAo = [2 -3 7 4; -9 2 -2 -7; 7 6 -5 -4; 1 9 0 -6; 2 -5 -8 1]bo = [5; -9; 7; 2; -6]disp('Showing OVERdetermined solution & check')xo = Ao\bobo_chk= Ao*xo



Uniqueness of solutions

A system has a unique solution If, and Only If (IFF): Rank[A]=Rank[Ab] = n• Where n is the ORDER of the system;

i.e., the Number of UNKNOWNS

Such systems are called full-rank systems



OverDetermined Rank

To interpret MATLAB answers correctly for an OVERdetermined system, first check the ranks of [A] and [Ab] to see whether an exact solution exists r[Ab] = r[A] = n (the no. of Unknowns)

If r[Ab] = r[A] > n then a unique soln does not exist, then you know that the left-division answer is a least squares solution• Can easily Chk: Axsoln = b (exactly)?

– This is at least as easy as the rank calc



Solving Linear Eqns Summary - 1

If the number of equations (m) in the set equals the number of unknown variables (n), the matrix A is square and MATLAB provides two ways of solving the equation set Ax = b:

1. The matrix inverse method; solve for x by typing x = inv(A)*b.

2. The matrix left-division method; solve for x by typing x = A\b.




If A is square and if MATLAB does not generate an error message when you use one of these methods, then the set has a unique solution, which is given by the left-division method.

You can always CHECK the solution for x by typing A*x to see if the result is the same as b.

• Use this to avoid the rank calcs




If when you type A*x you receive an error message, the set is UNDERdetermined, and either it does not have a solution or it has more than one solution

In such a case, if you need more information, you must use the following procedures.



Solving Linear Eqns Summary - 4 For UNDERdetermined and

OVERdetermined sets, MATLAB provides three ways of dealing with the equation set Ax = b. (Note that the matrix inverse method will NEVER work with such sets.)

1. The matrix left-division method; solve for x by typing x = A\b

2. The pseudoinverse method; solve for x by typing x = pinv(A)*b

3. the Reduced Row Echelon Form (RREF) method. This method uses the MATLAB function rref to obtain a solution.




UNDERdetermined Systems• In an underdetermined system NOT

enough information is given to determine the values of all the unknown variables.

– An INFINITE number of solutions might exist in which one or more of the unknowns are dependent on the remaining unknowns

– For such systems Cramer’s method and the matrix inverse method will not work because either A is not square or because |A| = 0.



Solving Linear Eqns Summary - 6• Underdetermined Systems cont

– The left-division method will give a solution with some of the unknowns arbitrarily set equal to zero, but this solution is NOT the general solution; it’s just ONE solution (of Trillions)

– An infinite number of solutions might exist even when the number of equations equals the number of unknowns. The left-division method fails to give a solution in such cases

– In cases that have an infinite number of solutions, some of the unknowns can be expressed in terms of the remaining unknowns, whose values are arbitrary. The rref function can be used to find these relations.




OVERdetermined Systems• An overdetermined system is a set of

equations that has more independent equations than unknowns

– For such a system Cramer’s method and the matrix inverse method will not work because the A matrix is NOT SQUARE

– Some overdetermined systems have exact solutions, which can be obtained with the left-division method A\b.



Solving Linear Eqns Summary - 8• OVERdetermined systems cont.

– For overdetermined systems that have no exact solution, the estimated-answer given by the left-division method satisfies the equation set only in a LEAST SQUARES sense

– When we use MATLAB to solve an overdetermined set, the program does not tell us whether the solution is exact. We must determine this information ourselves. The first step is to check the ranks of A and [A b] to see whether a solution exists; if no solution exists, then we know that the left-division solution is a least squares answer.



FlowChart for LinSys Solution

Can Construct Least-Squares Approximation



Descison Tree for LinSys Solns

if Rank(A)==Rank(Ab) & Rank(A)==NoUnknowns disp('EXACT Solution')

elseif Rank(A)==Rank(Ab) & Rank(A)<NoUnknowns disp('INFINITE Solutions')

elseif Rank(Ab) > NoUnknowns disp('LEAST SQS Approximation')

else disp('NO solution; can least-sqs est.')

end



All Done for Today

GabrielCramer

Born: 31 July 1704 in Geneva, SwitzerlandDied: 4 Jan 1752 in Bagnols-sur-Cèze, France



Bruce Mayer, PELicensed Electrical & Mechanical Engineer

[email protected]

Engr/Math/Physics 25

Appendix 6972 23 xxxxf



MATLAB LinSys Solver Code - 1 % Script file lineq.m% Solves the set Ax = b, given A and b.% Check the ranks of A and [A b].if rank(A) == rank([A b]) % Then The ranks are equal. Size_A = size(A); % Does the rank of A equal the number of

unknowns? if rank(A) == size_A(2) % Then the Rank of A equals the number of unknowns. disp(’There is a unique solution,

which is:’) x = A\b % Solve using left division.



MATLAB LinSys Solver Code - 2 else % Rank of A does not equal the number

of unknowns. disp(’There is an infinite number of

solutions.’) disp(’The augmented matrix of the

reduced system is:’) rref([A b]) % Compute the augmented

matrix. endelse % The ranks of A and [A b] are not equal. disp(’There are no solutions.’)end



MATLAB rank

rank • Rank of matrix

Syntax• k = rank(A)

Description• The rank function provides an estimate of

the number of linearly independent rows or columns of a full matrix.

[email protected] ENGR-25_Linear_Equations-1.ppt 1 Bruce Mayer, PE Engineering/Math/Physics...

Documents

Transcript of [email protected] ENGR-25_Linear_Equations-1.ppt 1 Bruce Mayer, PE Engineering/Math/Physics...