Systems of Linear EquationsSystems of Linear Equations
A linear equation in n variables has the form
A set of linear equations is called a system. A solution exists for a system iff the solution
satisfies all equations in the system. Many scientific and engineering problems
take this form.
a0x0 + a1x1 + … + an-1xn-1 = b
Solving Systems of EquationsSolving Systems of Equations
Many such systems are large.
– Thousands of equations and unknowns
a0,0x0 + a0,1x1 + … + a0,n-1xn-1 = b0
a1,0x0 + a1,1x1 + … + a1,n-1xn-1 = b1
an-1,0x0 + an-1,1x1 + … + an-1,n-1xn-1 = bn-1
Solving Systems of EquationsSolving Systems of Equations
A linear system of equations can be represented in matrix form
a0,0 a0,1 … a0,n-1 x0 b0
a1,0 a1,1 … a1,n-1 x1 b1
an-1,0 an-1,1 … an-1,n-1 xn-1 bn-1
=
Ax = b
Solving Systems of EquationsSolving Systems of Equations
Solving a system of linear equations is done in two steps:
– Reduce the system to upper-triangular
– Use back-substitution to find solution These steps are performed on the system in
matrix form.
– Gaussian Elimination, etc.
Solving Systems of EquationsSolving Systems of Equations
Reduce the system to upper-triangular form
Use back-substitution
a0,0 a0,1 … a0,n-1 x0 b0
0 a1,1 … a1,n-1 x1 b1
0 0 … an-1,n-1 xn-1 bn-1
=
Reducing the SystemReducing the System
Gaussian elimination systematically eliminates variable x[k] from equations k+1 to n-1.
– Reduces the coefficients to zero This is done by subtracting a appropriate
multiple of the kth equation from each of the equations k+1 to n-1
Procedure GaussianElimination(A, b, y) for k = 0 to n-1
/* Division Step */for j = k + 1 to n - 1 A[k,j] = A[k,j] / A[k,k]y[k] = b[k] / A[k,k]A[k,k] = 1
/* Elimination Step */for i = k + 1 to n - 1 for j = k + 1 to n - 1
A[i,j] = A[i,j] - A[i,k] * A[k,j] b[i] = b[i] - A[i,k] * y[k] A[i,k] = 0endfor
endforend
Parallelizing Gaussian Elim.Parallelizing Gaussian Elim.
Use domain decomposition
– Rowwise striping Division step requires no communication Elimination step requires a one-to-all
broadcast for each equation. No agglomeration Initially map one to to each processor
Communication AnalysisCommunication Analysis
Consider the algorithm step by step Division step requires no communication Elimination step requires one-to-all bcast
– only bcast to other active processors
– only bcast active elements Final computation requires no
communication.
Communication AnalysisCommunication Analysis
One-to-all broadcast
– log2q communications
– q = n - k - 1 active processors Message size
– q active processors
– q elements required
T = (ts + twq)log2q
Computation AnalysisComputation Analysis
Division step
– q divisions Elimination step
– q multiplications and subtractions Assuming equal time --> 3q operations
Computation AnalysisComputation Analysis
In each step, the active processor set is reduced by one resulting in:
2/)1(3
11
0
nnCompTime
knCompTimen
k
Can we do better?Can we do better?
Previous version is synchronous and parallelism is reduced at each step.
Pipeline the algorithm Run the resulting algorithm on a linear
array of processors. Communication is nearest-neighbor Results in O(n) steps of O(n) operations
Pipelined Gaussian Elim.Pipelined Gaussian Elim.
Basic assumption: A processor does not need to wait until all processors have received a value to proceed.
Algorithm– If processor p has data for other processors,
send the data to processor p+1– If processor p can do some computation using
the data it has, do it.– Otherwise, wait to receive data from processor
p-1
ConclusionConclusion
Using a striped partitioning method, it is natural to pipeline the Gaussian elimination algorithm to achieve best performance.
Pipelined algorithms work best on a linear array of processors.
– Or something that can be linearly mapped Would it be better to block partition?
– How would it affect the algorithm?
Top Related