1 Gauss Elimination Small Matrices For small numbers of equations, solvable by hand Graphical ...
-
Upload
emmeline-rich -
Category
Documents
-
view
217 -
download
0
Transcript of 1 Gauss Elimination Small Matrices For small numbers of equations, solvable by hand Graphical ...
1
Gauss EliminationGauss Elimination
Small Matrices
For small numbers of equations, solvable by hand Graphical Cramer's rule Elimination
2
nnnn22n11n
2nn2222121
1nn1212111
bxaxaxa
bxaxaxa
b xaxaxa
3
12
12
21
21
x3x3x2x
rearrange 3xx
3xx2
2x1 – x2 = 3x1 + x2 = 3
One solution
Graphical Method
2x1 – x2 = 3
2x1 – x2 = – 1
No solution
Graphical Method
6x1 – 3x2 = 92x1 – x2 = 3
Infinite many solutions
Graphical Method
2x1 – x2 = 3
2.1x1 – x2 = 3
ill conditioned
Graphical Method 方程式斜率非常接近,視覺上無法判定交點
Cramer’s Rule: Determinant
Compute the determinant D ( 行列式 ) 2 x 2 matrix
3 x 3 matrix
7
211222112221
1211 aaaaaa
aaD
3231
222113
3331
232112
3332
232211
333231
232221
131211
aa
aaa
aa
aaa
aa
aaa
aaa
aaa
aaa
D
Cramer’s Rule
To find xk for the following system
Replace kth column of as with bs (i.e., aik bi )
8
nnnn22n11n
2nn2222121
1nn1212111
bxa...xaxa
bxa...xaxabxa...xaxa
)
)(
ijk D(a
matrix newDx
Cramer’s Rule :例
3x3 matrix
9
333231
232221
131211
aaa
aaa
aaa
D
33231
22221
11211
33
33331
23221
13111
22
33323
23222
13121
11
baa
baa
baa
D
1
D
Dx
aba
aba
aba
D
1
D
Dx
aab
aab
aab
D
1
D
Dx
ILL-Conditioned System
What happen if the determinant D is very small or zero?
Divided by zero (linearly dependent system) Divided by a small number: Round-off error Loss of significant digits
10
0AdetD
Eliminate x2
Subtract to get
2222121
1212111
bxaxa
bxaxa
2122221212112
1222122211122
baxaaxaa
baxaaxaa
aaaa
babax
aaaa
babax
babaxaaxaa
21122211
1212112
21121122
2121221
2121221211211122
Not practical for large number (> 4) of equations
Elimination Method
MATLAB’s Methods
Forward slash ( / ) Back-slash ( \ ) Multiplication by the inverse of the quantity under
the slash
12
b*Ainvx
bAxbAx
bAx1
)(
\
Gauss Elimination
Manipulate equations to remove one of the unknowns Develop algorithm to do this repeatedly The goal is to set up upper triangular matrix
Back substitution to find solution (root)13
nn
n333
n22322
n1131211
a
aa
aaa
aaaa
U
Basic Gauss Elimination
Direct method (no iteration required) Forward elimination Column-by-column elimination of the below-
diagonal elements Reduce to upper triangular matrix Back-substitution
14
Naive Gauss Elimination ( 單純高斯消去法 )
Begin with
Multiply the first equation by a21 / a11 and subtract from second equation
15
nnnn22n11n
2nn2222121
1nn1212111
bxa...xaxa
bxa...xaxa
bxa...xaxa
nnnn22n11n
111
212nn1
11
21n2212
11
2122111
11
2121
1nn212111
bxa...xaxa
ba
abxa
a
aa...xa
a
aaxa
a
aa
bxa...xaxa
Forward Elimination
Reduce to
Repeat the forward elimination to get
16
nnnn22n11n
2nn2222
1nn1212111
bxa...xaxa
bxa...xa
bxa...xaxa
nnnn22n
2nn2222
1nn1212111
bxa...xa
bxa...xa
bxa...xaxa
Forward Elimination
First equation is pivot equation ( 軸元方程式 ) a11 is pivot element ( 軸元元素 ) Now multiply second equation by a'32 /a'22 and
subtract from third equation
17
222
323nn2
22
32n3323
22
3233
2nn2323222
1nn1313212111
ba
abxa
a
aaxa
a
aa
bxaxaxa
bxaxaxaxa
Forward Elimination
Repeat the elimination of ai2 and get
Continue and get
18
nnnn33n
3nn3333
2nn2323222
1nn1313212111
bxaxa
bxaxa
bxaxaxa
bxaxaxaxa
)()( 1nnn
1nnn
3nn3333
2nn2323222
1nn1313212111
bxa
bxaxa
bxaxaxa
bxaxaxaxa
Back Substitution ( 向後代換 )
Now we can perform back substitution to get {x} By simple division
Substitute this into (n-1)th equation
Solve for xn-1
Repeat the process to solve for xn-2 , xn-3 , …. x2, x1
19
)()(,
)(,
2n1nn
2nn1n1n
2n1n1n bxaxa
)(
)(
1nnn
1nn
n a
bx
Back Substitution
Back substitution: starting with xn
Solve for xn1 , xn2 , … , x3, x2, x1
20
a
xab
x
a
bx
1iii
n
1ijj
1iij
1ii
i
1nnn
1nn
n
)(
)()(
)(
)(
for i = n1, n2, …, 1
0a 1iii )(
Naive Gauss EliminationNaive Gauss Elimination
)()(
)()(
)()(
1f4
1f3
1f2
baaa0
baaa0
baaa0
baaaa
41
31
21
4444342
3343332
2242322
114131211
114141
113131
112121
444434241
334333231
224232221
114131211
aaf
aaf
aaf
baaaa
baaaa
baaaa
baaaa
/
/
/
Elimination of First Column
)()(
)()(
/
/
2f4
2f3
baa00
baa00
baaa0
baaaa
aaf
aaf
baaa0
baaa0
baaa0
baaaa
42
32
44443
33433
2242322
114131211
224242
223232
4444342
3343332
2242322
114131211
Elimination of Second Column
Upper triangular matrix
)()(
/
3f4
ba000
baa00
baaa0
baaaa
aaf
aaa00
aaa00
baaa0
baaaa
43444
33433
2242322
114131211
33434344443
33433
2242322
114131211
Elimination of Third Column
1141431321211
2242432322
3343433
4444
a/)xaxaxab(x
a/)xaxab(x
a/)xab(x
a/bx
Upper triangular matrix
0a,a,a,a 44332211
ba000
baa00
baaa0
baaaa
444
33433
2242322
114131211
Back Substitution
41
31
21
41
31
21
f14
f13
f12
5141020
24110
00420
13201
6f
0f
1f
14226
24110
13221
13201
)()(
)()(
)()(
Example
42
32
42
32
f24
f23
5141400
24100
00420
13201
1f
1/2f
5141020
24110
00420
13201
)()(
)()(
Forward Elimination
43
43
f34
3370000
24100
00420
13201
14f
5141420
24110
00420
13201
)()(
Upper Triangular Matrix
13/70x3x21x
8/35x2x
4/352x4x
33/707033/x
431
32
43
4
33/70
4/35
8/35
13/70
x
3370000
24100
00420
13201
Back Substitution
29
function x = GaussNaive(A,b)% x = GaussNaive(A,b):% Gauss elimination without pivoting.% input:% A = coefficient matrix% b = right hand side vector% output:% x = solution vector[m,n] = size(A);if m~=n, error('Matrix A must be square'); endnb = n+1;Aug = [A b];% forward eliminationfor k = 1:n-1 for i = k+1:n factor = Aug(i,k)/Aug(k,k); Aug(i,k:nb) = Aug(i,k:nb)-factor*Aug(k,k:nb); end disp(Aug);end% back substitutionx = zeros(n,1);x(n) = Aug(n,nb)/Aug(n,n);for i = n-1:-1:1 x(i) = (Aug(i,nb)-Aug(i,i+1:n)*x(i+1:n))/Aug(i,i);end
Print all factor and Aug
(do not suppress output)Eliminate first column
Eliminate second column
Eliminate third column Back-substitution
Aug = [A, b]
>> format short>> x = GaussNaive(A,b)m = 4n = 4Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1factor = -1Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 0Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 6Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 0 2 -10 -14 -5
factor = 0.5000Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5factor = 1Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 -14 -14 -5
factor = 14Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 0 -70 -33
x4
x3
x2
x1
x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x = -0.1857 0.2286 -0.1143 0.4714
Gauss Elimination: Algorithm
Forward elimination for each equation j, j = 1 to n-1
for all equations k greater than j
(a) multiply equation j by akj /ajj
(b) subtract the result from equation k This leads to an upper triangular matrix
Back substitution (a) determine xn from
(b) put xn into (n-1)th equation, solve for xn-1
(c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved
31
)1n(nn
)1n(nn a/bx
Operation Count For Gauss Elimination
Elimination routine on the order of O(n3/3) operations
Total operation counts for elimination stage = 2n3/3 + O(n2)
Back-substitution uses O(n2/2)
Total operation counts for back substitution stage = n2 + O(n)
))(())((,
))(())((,
))(())((,
))(())((,
3121nn1n
2knkn1knknn1kk
n2n1n2nn32
1n1nn1nn21
flops
iontion/DivisMultiplica
flops
ubtractionAddition/S
i
Loop Inner
k
Loop Outer
Operation Count For Gauss Elimination
#flops (floating-point operations) for Naive Gauss elimination
Computation time increases rapidly with n Most effort incurs in the elimination step Improve efficiency by reducing the elimination effort
33
%....
%.
%.
859910676106861000000106761000
539866666768155010000671550100
588766780510070510
nEliminatio to
Due Percentage
3
2nFlops
Total
onSbustituti
Back
nElimination
868
3
Partial Pivoting ( 部分軸元法 )
Problems with Gauss elimination division by zero round off errors ill conditioned systems
Use “pivoting” to avoid this Find the row with largest absolute coefficient below the
pivot element Switch rows (“partial pivoting”) Complete pivoting switch columns (rarely used)
34
Round-off Errors ( 捨入誤差 )
A lot of chopping with more than n3/3 operations More important - error is propagated For large systems (more than 100 equations),
round-off error is important (machine dependent) Ill conditioned systems - small changes in
coefficients lead to large changes in solution Round-off errors are especially important for ill-
conditioned systems
35
2x1 – x2 = 3
2.1x1 – x2 = 3
Ill-conditioned System
Ill-conditioned System
Consider
Since slopes are almost equal
22
21
22
212
12
11
12
112
2222121
1212111
a
bx
a
ax
a
bx
a
ax
bxaxa
bxaxa
22
21
12
11
a
a
a
a
0aa
aaD
2221
1211 Divided by small number
Determinant
Calculate determinant using Gauss elimination
38
)1n(nn
n333
n22322
n1131211
a
aa
aaa
aaaa
U
)(detdet 1nnn332211 aaaaUA
Gauss Elimination with Partial Pivoting
Forward elimination for each equation j, j = 1 to n-1
• first scale each equation k greater than j• then pivot (switch rows)• Now perform the elimination
(a) multiply equation j by akj /ajj
(b) subtract the result from equation
39
1x4x2x2x6
2x4xx
1x3x2x2x
1 x3x2x
4321
432
4321
431
14226
24110
13221
13201
bA
Partial (Row) Pivoting
Interchange rows 1 & 4
41
31
21
41
31
21
f14
f13
f12
5/67/35/31/30
24110
5/67/37/37/30
14226
1/6f
0f
1/6f
13201
24110
13221
14226
)()(
)()(
)()(
Forward Elimination
No interchange required
42
32
42
32
f24
f23
5/72200
33/145000
5/67/37/37/30
14226
1/7f
3/7f
5/67/35/31/30
24110
5/67/37/37/30
14226
)()(
)()(
Forward Elimination
Interchange rows 3 & 4
13/70/6x 2x 2x 41x
8/357/3/x 7/3x 7/35/6x
4/35/2x 25/7x
33/70/533/14x
2341
342
43
4
)(
)()(
)(
)(
33/70
4/35
8/35
13/70
x
0f
33/145000
5/72200
5/67/37/37/30
14226
43
Back-Substitution
44
function x = GaussPivot(A,b)% x = GaussPivot(A,b):% Gauss elimination without pivoting.
[m,n]=size(A);if m~=n, error('Matrix A must be square'); endnb=n+1;Aug=[A b];% forward eliminationfor k = 1:n-1 % partial pivoting [big,i]=max(abs(Aug(k:n,k))); ipr=i+k-1; if ipr~=k Aug([k,ipr],:)=Aug([ipr,k],:); end for i = k+1:n factor=Aug(i,k)/Aug(k,k); Aug(i,k:nb)=Aug(i,k:nb)-factor*Aug(k,k:nb); end disp(Aug);end% back substitutionx=zeros(n,1);x(n)=Aug(n,nb)/Aug(n,n);for i = n-1:-1:1 x(i)=(Aug(i,nb)-Aug(i,i+1:n)*x(i+1:n))/Aug(i,i);end
Partial pivoting (switch rows)
[big,i] = max(x)
largest element in {x}
index of the largest element
>> format short>> x=GaussPivot0(A,b)Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1big = 6i = 4ipr = 4Aug = 6 2 2 4 1 -1 2 2 -3 -1 0 1 1 4 2 1 0 2 3 1factor = -0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 0 -0.3333 1.6667 2.3333 0.8333
Aug = [A b]
Interchange rows 1 and 4
Find the first pivot element and its index
Eliminate first columnNo need to interchange
big = 2.3333i = 1ipr = 2factor = 0.4286Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333factor = -0.1429Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 0 2.0000 2.0000 0.7143big = 2i = 2ipr = 4Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571
x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x =
-0.1857 0.2286 -0.1143 0.4714
Back substitutionSecond pivot element and index
Third pivot element and index
Eliminate second column
Eliminate third column
Interchange rows 3 and 4
No need to interchange
Save factors fij for
LU decomposition
Banded Matrix
47
HBW: Half band width
Banded Matrix
48
xw
vut
srq
pzk
nml
kji
hgf
edc
ba
0000000
000000
000000
000000
000000
000000
000000
000000
0000000ai,j= 0if j > i + HBor j < i - HB
HB: Half bandwidthB: Bandwidth
B = 2*HB + 1
In this exampleHB = 1 & B = 3
Tridiagonal Matrix ( 三對角線系統 )
Only three nonzero elements in each equation (3n instead of n2 elements) Subdiagonal, diagonal, superdiagonal
Solve by Gauss elimination
49
Tridiagonal Matrix :例
Special case of banded matrix with bandwidth = 3 Save storage, 3 n instead of n n
50
n
1n
i
3
2
1
n
1n
i
3
2
1
nn
1n1n1n
iii
333
222
11
r
r
r
r
r
r
x
x
x
x
x
x
fe
gfe
gfe
gfe
gfe
gf
Tridiagonal Matrix
Forward elimination
Back substitution
51
n32k
rf
err
gf
eff
1k1k
kkk
1k1k
kkk
,,,
1232n1nk f
xgrx
f
rx
k
1kkkk
n
nn
,,,,,
Use factor = ek / fk1
to eliminate subdiagonal element
Apply the same matrix operations to right hand side
Tridiagonal Matrix: Hand Calculations
52
53
2
5
3
x
x
x
x
2515000
50210
0152
0021
4
3
2
1
...
.
411
5053r
f
err
1501
50251g
f
eff
111
12r
f
err
111
12g
f
eff
131
25r
f
err
121
25g
f
eff
33
444
33
444
11
223
22
333
11
222
11
222
)(.
.
).(.
.
)(
)(
)(
)(
11
223
f
xgrx
21
311
f
xgrx
31
4501
f
xgrx
41
4
f
rx
1
2111
2
3222
3
4333
4
44
))((
))((
))(.(
(a) Forward elimination (b) Back substitution
53
function x = Tridiag(e,f,g,r)% x = Tridiag(e,f,g,r):% Tridiagonal system solver.% input:% e = subdiagonal vector% f = diagonal vector% g = superdiagonal vector% r = right hand side vector% output:% x = solution vectorn=length(f);% forward eliminationfor k = 2:n factor = e(k)/f(k-1); f(k) = f(k) - factor*g(k-1); r(k) = r(k) - factor*r(k-1);end% back substitutionx(n) = r(n)/f(n);for k =n-1:-1:1 x(k) = (r(k)-g(k)*x(k+1))/f(k);end
» [e,f,g,r] = example
e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000
» x = Tridiag (e, f, g, r)x = 1 2 3 -1 -2 -3
function [e,f,g,r] = example
e=[ 0 -2 4 -0.5 1.5 -3];
f=[ 1 6 9 3.25 1.75 13];
g=[-2 4 -0.5 1.5 -3 0];
r=[-3 22 35.5 -7.75 4 -33];
33
4
75.7
5.35
22
3
x
x
x
x
x
x
133
375.15.1
5.125.35.0
5.094
462
21
6
5
4
3
2
1
Note: e(1) = 0 and g(n) = 0
Tridiagonal Matrix: 運算例
55
補充: Big-O
56
Concept of Order of Growth
We say fA(n)=30n+8 is (at most) order n, or O(n)
It is, at most, roughly proportional to n fB(n)=n2+1 is order n2, or O(n2)
It is (at most) roughly proportional to n2
Any function whose exact (tightest) order is O(n2) is faster-growing than any O(n) function Later we will introduce Θ for expressing exact order
57
Definition: O(g) (Asymptotic Upper Bound)
“f is at most order g”, or “f is O(g)”, or “f = O(g)” all just mean that fO(g)
Let g be any function RR. Define “at most order g”, written O(g), to be:
{f:RR | c,k: x>k: f(x) cg(x)} “Beyond some point k, function f is at most a constant c
times g (i.e., proportional to g)”
58
About the Definition O(g)
59
Big-O 範例 (1/3)
60
Big-O 範例 (2/3)
61
Big-O 範例 (3/3)
Show that 30n+8 is O(n) To show c,k: n > k: 30n+8 cn
• Let c = 31, k = 8. Assume n > k = 8. Thencn = 31n = 30n + n > 30n+8, so 30n+8 < cn
Show that n2+1 is O(n2) To show c,k: n > k: n2+1 cn2
• Let c = 2, k = 1. Assume n > 1. Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2
62
Note 30n+8 isn’t less than n anywhere (n>0) It isn’t even less than 31n everywhere But it is less than 31n everywhere to the right of n=8
n>k=8
Big-O Example
Increasing n
Val
ue o
f fu
ncti
on
n
30n+8cn =31n
30n+8 O(n)
63
64
應知的定理
65
66
定理應用範例
67
Summary
For any g:RR, “at most order g”,O(g) {f:RR | c,k x > k |f(x)| |cg(x)| } Often, we deal only with positive functions and can
ignore absolute value symbols
“fO(g)” often written “f is O(g)” or “f = O(g)” The latter form is an instance of a more general
convention...
68
(g) {f:RR | c,k x > k, |f(x)| > |cg(x)|}
Remarks O(.) gives worst-case guarantees (good news), while
gives a lower bound (bad news)
的定義 (Asymptotic Lower Bound)
69
Definition: (g), Exactly Order g (Asymptotic Tight Bound)
If fO(g) and gO(f ), then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g)
Another equivalent definition:(g) {f:RR | c1c2k > 0 x > k: |c1g(x)| |f(x)| |c2g(x)| } “Everywhere beyond some point k, f(x) lies in between
two multiples of g(x)”
70