Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... ·...
Transcript of Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... ·...
![Page 1: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/1.jpg)
Solving a System of Equations
Objectives
Understand how to solve a system of equations with:
- Gauss Elimination Method
- LU Decomposition Method
- Gauss-Seidel Method
- Jacobi Method
A system of linear algebraic equations
Linear algebraic equations are of the general form,
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
n n nn n n
a x a x a x b
a x a x a x b
a x a x a x b
(1)
Where the a’s are constant coefficients, the b’s are constants, the x’s are unknowns,
and n is the number of equation. The algebraic equations can be briefly rewritten in
the matrix form as follows:
A x b (2)
![Page 2: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/2.jpg)
Gauss Elimination Method
There are many techniques for solving a system of linear algebraic equations.
One of basic techniques that can be applied to large sets of equation and can be
formalized and programmed for the computer is Gauss Elimination Method.
The procedure consists of two major steps which are
1. Forward Elimination – the equations are manipulated to eliminate the unknowns
from the equations until we have one equation with one unknown.
2. Back substitution – The equation with one unknown can be solved directly. The
result is back-substituted into one of the original equations to solve for the
remaining unknown.
Fig. 1 The steps of Gauss Elimination
![Page 3: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/3.jpg)
Example 1 Use Gauss Elimination to solve
1 2 3
1 2 3
1 2 3
2 5 21
2 2 15
4 18
x x x
x x x
x x x
………………………………………………………………………………………………………………………………………………………………..
………………………………………………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
………………………………………………………………………………………………………………………………………………………………..
………………………………………………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
![Page 4: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/4.jpg)
………………………………………………………………………………………………………………………………………………………………..
………………………………………………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
![Page 5: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/5.jpg)
Example 2 Use Gauss Elimination to solve
2 3
1 2 3
1 2 3
2 8
4 6 7 3
2 3 6 18
x x
x x x
x x x
………………………………………………………………………………………………………………………………………………………………..
………………………………………………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
………………………………………………………………………………………………………………………………………………………………..
………………………………………………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
![Page 6: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/6.jpg)
………………………………………………………………………………………………………………………………………………………………..
………………………………………………………………………………………………………………………………………………………………..
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
……………………………………………………………………………………………………………………………………………………………….
………………………………………………………………………………………………………………………………………………………………
……………………………………………………………………………………………………………………………………………………............
![Page 7: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/7.jpg)
Example 3 Use Gauss Elimination to solve
1 2
1 2
0.0003 3.0000 2.0001
1.0000 1.0000 1.0000
x x
x x
Exact Solutions: 1
2
1/ 3 0.3333
2 / 3 0.6667
x
x
If we rearrange the pivoting equation by setting the largest element as the pivot
element.
1 2
1 2
1.0000 1.0000 1.0000
0.0003 3.0000 2.0001
x x
x x
![Page 8: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/8.jpg)
LU Decomposition Method
The forward elimination of Gauss elimination comprises the bulk of the
computational effort, especially the large system of equations. LU decomposition
method separates the time consuming elimination of the matrix [A] by the following
steps.
1. LU decomposition step – The matrix [A] is decomposed into lower triangular
matrix [L], and upper triangular matrix [U].
2. Substitution step – [L] and [U] are used to determine a solution {x} for a right-
hand side {b}. This step consists of the forward and back substitutions.
2.1 The forward substitution is conducted to determine the intermediate
vector {d} from [L]{d}={b}.
2.2 Then, the result of {d} is substituted into [U]{x}={d}, and solve for {x}
through the back substitution.
![Page 9: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/9.jpg)
Fig.2 The steps in LU decomposition.
1. The LU decomposition/ Factorization step
]][[][ ULA
3233233213313222321231311131
2323132122221221211121
131312121111
auululaululaul
auulauulaul
auauau
[L] and [U] can be obtained by solving the above 9 equations.
![Page 10: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/10.jpg)
2. The substitution step
We start from
[ ]{ } { }A x b (1)
After LU decomposition step, we have
[ ][ ]{ } { }L U x b (2)
Now, let
[ ]{ } { }U x d (3)
Eq. (2) can be rewritten as
[ ]{ } { }L d b (4)
Or
1 1
21 2 2
31 32 3 3
1 0 0
1 0
1
d b
l d b
l l d b
(5)
Then, determine {d} through forward substitution. After we get {d}, we
substitute {d} into eq. (3). We have
11 12 13 1 1
22 23 2 2
33 3 3
0
0 0
u u u x d
u u x d
u x d
(6)
Use back substitution to determine {x}.
![Page 11: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/11.jpg)
Example 4 Use LU decomposition to solve
1 2 3
1 2 3
1 2 3
10 2 27
3 6 2 61.5
5 21.5
x x x
x x x
x x x
LU decomposition/Factorization
10 2 1 1 0 0 10 2 1
[ ] 3 6 2 0.3 1 0 0 5.4 1.7
1 1 5 0.1 0.1481 1 0 0 5.3519
A
Substitution step for determine {d} and {x}
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
![Page 12: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/12.jpg)
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
![Page 13: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/13.jpg)
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
![Page 14: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/14.jpg)
Iterative Methods for Systems of Equations
- Jacobi Method
- Gauss-seidel Method
Jacobi Method
Assume that we have a 3x3 set of equations.
11 12 13 1 1
21 22 23 2 2
31 32 33 3 3
a a a x b
a a a x b
a a a x b
Or
11 1 12 2 13 3 1
21 1 22 2 23 3 2
31 1 32 2 33 3 3
a x a x a x b
a x a x a x b
a x a x a x b
(1)
If the diagonal elements are all nonzero, the first equation can be solved for
x1, the second for x2, the third for x3. Then, we have
1 11 12 132 3
111
j jj b a x a x
xa
(2a)
1 12 21 231 3
222
j jj b a x a x
xa
(2b)
1 13 31 321 2
333
j jj b a x a x
xa
(2c)
![Page 15: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/15.jpg)
where j and j-1 are the present and previous iterations. To solve the solutions, we
need the initial guesses for 1 2 3, ,j j jx x x for the first iteration. Then, the iteration is
continued until our solutions converge closely enough to the true values or the error
of the approximation less than or equal to tolerance.
1
, 100j j
i ia i sj
i
x xε ε
x
(3)
where i = 1, 2, 3 for the x’s.
Example 5 Use Jacobi method to solve
1 2 3
1 2 3
1 2 3
2 5 21
2 2 15
4 18
x x x
x x x
x x x
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
![Page 16: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/16.jpg)
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
![Page 17: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/17.jpg)
Gauss-seidel Method
This method is the most commonly used iterative method for solving linear
algebraic equations. Assume that we have a 3x3 set of equations.
11 1 12 2 13 3 1
21 1 22 2 23 3 2
31 1 32 2 33 3 3
a x a x a x b
a x a x a x b
a x a x a x b
(1)
If the diagonal elements are all nonzero, the first equation can be solved for
x1, the second for x2, the third for x3. Then, we have
1 11 12 132 3
111
j jj b a x a x
xa
(2a)
12 21 231 3
222
j jj b a x a x
xa
(2b)
3 31 321 23
33
j jj b a x a x
xa
(2c)
where j and j-1 are the present and previous iterations. To solve the solutions, we
need the initial guesses for 1 2 3, ,j j jx x x for the first iteration. Then, the iteration is
continued until our solutions converge closely enough to the true values or the error
of the approximation less than or equal to tolerance.
1
, 100j j
i ia i sj
i
x xε ε
x
(3)
where i = 1, 2, 3 for the x’s.
![Page 18: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/18.jpg)
Example 6 Use Gauss-Seidel method to solve
1 2 3
1 2 3
1 2 3
10 2 27
3 6 2 61.5
5 21.5
x x x
x x x
x x x
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………
![Page 19: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/19.jpg)
Fig. 3 The difference procedure between (a) the Gauss-Seidel method and (b) the
Jacobi method.
Exercise
1.
2 1
2 3
1 3 2 1
10 3 7
4 7 30 0
7 40 3 5
x x
x x
x x x x
Solve the above system of equations using Gauss elimination, Jacobi, and
Gauss-Seidel methods. Use three iterations for the Jacobi, and Gauss-Seidel
iterations.
![Page 20: Solving a System of Equationseng.sut.ac.th/me/2014/document/StatNumer/525201 Statistics and... · Gauss Elimination Method There are many techniques for solving a system of linear](https://reader034.fdocuments.us/reader034/viewer/2022050509/5f9a2fad2986145c71633386/html5/thumbnails/20.jpg)
2. The position of three masses suspended vertically by series of identical springs
can be modeled by the following steady-state force balances:
2 1 1 1
3 2 2 2 1
3 3 2
0 ( )
0 ( ) ( )
0 ( )
k x x m g kx
k x x m g k x x
m g k x x
If g = 9.81 m/s2, m1 = 2 kg, m2 = 3 kg, m3 = 2.5 kg, and the k’s = 10 N/m. Use
the Gauss elimination, Jacobi, and Gauss-Seidel methods to solve for position
(the x’s) of masses. Note that three iterations are performed for the Jacobi,
and Gauss-Seidel iterations.
3. Use the Jacobi, and Gauss-Seidel methods to solve the following system until
the percent relative error falls below 5%sε .
0.8 0.4 41
0.4 0.8 0.4 25
0.4 0.8 105
x
y
z