Answering Distance Queries in directed graphs using fast matrix multiplication Seminar in Algorithms...
-
date post
21-Dec-2015 -
Category
Documents
-
view
217 -
download
2
Transcript of Answering Distance Queries in directed graphs using fast matrix multiplication Seminar in Algorithms...
Answering Distance Queries in directed
graphs using fast matrix multiplication
Seminar in AlgorithmsProf. Haim Kaplan
Lecture by Lior Eldar1/07/2007
Structure of Lecture
• Introduction & History
• Alg1 – APSP
• Alg2 – preprocess & query
• Alg3 – Hybrid
• Summary
Problem Definition• Given a weighted directed graph,
we are requested to find:– APSP - All pairs shortest paths – find for any pair– SSSP - Single Source shortest paths – find all
distances from s.
• A hybrid problem comes to mind:– Preprocess the graph faster than APSP– Answer ANY two-node distance query faster than
SSSP.– What’s it good for?
),,( wEVG
Previously known results – APSP
• Undirected graphs– Approximated algorithm by Thorup and Zwick:
• Preprocess undirected weighted graph in
expected time.
• Generate data structure of size
• Answer any query in O(1)
• BUT: answer is approximate with a factor of 2k-1.
– For non-negative integer weights at most M – Shoshan and Zwick developed an algorithm of run time
kmnO /1
knO /11
376.2, MnO
• Directed graphs – Zwick - runs in 58.268.0 nMO
Previously known results - SSSP
• Positive weights:– Directed graphs with positive weights – Dijkstra
with – Undirected graphs with positive integer edge
weights – Thorup with
• Negative weights – much harder:– Bellman-Ford– Goldberg and Tarjan – assumes edge weight
values are at least – N.
nnmO log
mO
mnO
NnmO log
New Algorithm by Yuster / Zwick
• Solves the hybrid pre-processing-query problem for:– Directed graphs
– Integer weights from –M to M
• Achieves the following performance:– Pre-processing
– Query answering – O(n)
• Faster than previously known APSP (Zwick) so long as the number of queries is
• Better than SSSP performance (Goldberg&Tarjan) for dense graphs with small alphabet – gap of
376.2, MnO
158.2 MnO
13.0nO
Beyond the numbers…• An extension of this algorithm allows complete
freedom in optimization of the pre-processing - query problem.
• to optimize an algorithm for an arbitrary number of queries q, we want: preprocessing time + q * query time to be minimal.
• This defines the ratio between query time and pre-processing time - completely controlled by the algorithm inputs.
• Meaning: if we know in advance the number of queries we can fine-tune the algorithm as we wish.
Before we begin - scope
• Assumptions:– No negative cycles
• Inputs:– Directed Weighted Graph G=(V,E,w)– Weights are –M,…0,…,M
• Outputs:– Data structure such that – given any two nodes –
produces the shortest distance between them (and not the path itself) – with high probability.
Matrix Multiplication• The matrix product C=AB, where A is an
matrix, B is , and C is matrix, is defined as follows:
• Define: the minimal number of algebraic operations for computing the matrix product.
• Define as the smallest exponent such that
• Theorem by Coppersmith and Winograd:
m
kjkkiji baC
1,,,
lxmmxn lxn
),,( nmlM
),,( tsr )(),,( nOnnnM tsr
376.2
Distance Products• The distance product , where A is an
matrix, B is , and C is matrix, is defined as follows:
• Recall: if W is an n x n matrix of the edge weights of a graph then is the distance matrix of the graph.
• Lemma by Alon: can be computed almost as fast as “regular” matrix multiplication:
jkkimkji baC ,,1, min
lxmmxn lxn
nW
BAC
BAC
),,( tsrMnO
State-of-the-art APSP• Randomized algorithm by Zwick that runs in
time• Intuition:
– Computation of all short paths is intensive.– BUT: long paths are made up of short paths: once
we pay the initial price we can leverage this work to compute longer paths with less effort.
• Strategy: Giving up on certainty - with a small number of distance updates we can be almost sure that any long-enough path has at least one representative that is updated.
58.268.0 nMO
Basic Operations• Truncation
– Replace any entry larger than t with
• Selection– Extract from D the elements whose row indices are
in A, and column indices are in B.
• Min-Assignment– Assign to each element the smallest between the
two corresponding elements of D and D‘.
tD
],[ BAD
],[],[ 'min BADBAD
Pseudo-code• Simply sample nodes and multiply decimated
matrices…
endfor
VBDBVD
nnVsampleB
ntolfor
WD
nV
W
sMsM
l
nxn
],[*],[D
)2/3/()ln(9,
log 1
,...,2,1
M),shortpath( algorithm
min
2/3
On matrices and nodes…
• Column-decimated matrix ],[ BVD
D
],[ BVD
Distance between any two nodes
Shortest directed path from any node to any
node in B
On matrices and nodes…(2)
• Row-decimated matrix ],[ VBD
],[ VBD
Distance between any two nodes
Shortest directed path from any node in B to
any node
What do we prove?
• Lemma: if there is a shortest path between nodes i and j in G that uses at most edges, then after the -th iteration of the algorithm, with high probability we have
• Meaning: at each iteration we update with high probability all the paths in the graph of a certain length. This serves as a basis for the next iteration.
jiji d ,, l
l2/3
Proof Outline
• By Induction:– Base case: easy – the input W contains all paths of
length – Induction step:
• Suppose that the claim holds for and show that it also holds for
• Take any two nodes that their shortest distance is at least . The -th iteration matrix product will (almost certainly) plug in their shortest distance at location (i,j) of D.
12/3 0
1ll
12/3 l l
Why?
• Set
• The path p from i to j is at least 2s/3.
• This divides p into three subsections:– Left – at most s/3– Right – at most s/3– Middle – exactly s/3
ls 2/3
i i’ k j’ j
3/s3/s 3/s
The Details
• The left and right “thirds” - help attain the induction step.– The path p(i,k) and p(k,j) are short enough – at most 2s/3
good for previous step:
• The middle “third” – ensures the fault probability is low enough.– Prob(no k is selected) =
– Probability still goes to 0 (as n tends to infinity) after computation of
• entries• iterations
1l
3
3/1ln9
1ns
ns
2n nO log
So…• Assuming all previous steps were good enough:
– With high probability each long-enough path has a representative in B
– The update of the D using the product
plugs in the correct result.
• Note that:– Each element is first limited to s*M– This is necessary for the fast-matrix-multiplication
algorithm
],[*],[ VBDBVD
Complexity
• Where does the trick hide?– The matrix alphabet increases linearly with iteration
number
– The product size decreases with iteration number
• For each iteration :– Alphabet size: s*M
– Product complexity: , where
– Total:
• Disregarding the log function, and optimizing between fast and naïve matrix products we get:
lsl 2/3,
1,,1 r snnr /
)( 58.268.0 nMO
1,,111,,1 rrr MnnOsMnO
Fast Product versus Naive
)( 2 rnO
)( )1,,1(1 rrnO
*assuming small M
58.2
Complexity Behavior• For a given matrix alphabet M, we find the
cross-over point between the matrix algorithms.
• For high r (>M-dependent threshold) we use FMM– Complexity dependent on M
• For low r (<threshold) we use naïve multiplication– Complexity not dependent on M
• Q: How does complexity change over the iteration number?
Pre-processing algorithm
• Motivation:– We rarely query all node-pairs
• Strategy:– Replace the costly matrix product
with 2 smaller products:1.
2.
– Generate data structure such that each query costs only
)( 2nO
],[*],[ VBDBVD
],[*],[ BBDBVD
)( 2nO)(nO
],[*],[ VBDBBD
Starting with the query…
• Pseudo-code:
• What is a sufficient trait of D, such that the returned value will be, with high probability
• Answer: with high probability, a node k on the path from i to j should have:
jVDViD
Dnxn,*,return
j)i,,dist( algorithm
),( ji
),(, kid ki
),(, jkd jk
Preprocessing algorithm
endfor
VBDBBD
VBDBVD
nnBsampleB
ntolfor
WDVB
nV
W
sMsM
sMsM
l
nxn
],[*],[V]D[B,
],[*],[B]D[V,
)2/3/()ln(9,
log 1
,
,...,2,1
M),shortpath( algorithm
min
min
2/3
New matrix type• Row&Column-decimated matrix ],[ BBD
D
],[ BBD
Query data structure for any two nodes
Query data-structure for any 2 nodes in B
What do we prove?
• Lemma 4.1: If or , and there is a shortest path from i to j in G that uses at most edges, then after the -th iteration of the preprocessing algorithm, with high probability we have .
• Meaning: D has the necessary trait: for any path p, if we iterate long enough, then with high probability, for at least one node k (in p(i,j)) the entries d(i,k), d(k,j) will contain shortest paths. Hence, “query” will return the correct result.
lBi lBj l2/3
l
jijid ,,
Proof Outline - preprocess
• By Induction:– Base case: easy – B=V, and the input W contains
all paths of length . – Induction step:
• Suppose that the claim holds for and show that it also holds for
• Take any two nodes that their shortest distance is at most . The l-th iteration matrix products (2) will (almost certainly) plug in their shortest distance at location (i,j) of D provided that EITHER or
.
1)2/3( 0
1ll
l2/3
lBilBj
Why?
• Set
• The path p from i to j is at least 2s/3.
• This divides p into three subsections:– Left – at most s/3– Right – at most s/3– Middle – exactly s/3
ls 2/3
i i’ k j’ j
3/s3/s 3/s
The Details• Assume that .• With high probability ( ) there will be k in
p(i,j), such that (remember why?)• Both are also in ,since • We therefore attain the induction step:
– The path p(i,k) and p(k,j) are short enough – at most 2s/3 good for previous step.
– The end-points of these paths (k) are in – Therefore their shortest distance is in D– The second product then updates correctly.
(assumption critical here)
lBi
lBk
1lB VBBB ll 01 ...
)( 3nO
1lB
Where’s the catch?
• In APSP, we assure that:– At every iteration l we compute the shortest path
of length at most .– BUT: we had to update all pairs each time
• In the preprocess algorithm, we assure:– At every iteration l, we compute the shortest path
of length at most only for a selected subset.– BUT: this subset covers all possible subsequent
queries, with high probability.
l)2/3(
l)2/3(
2n
Complexity• Matrix product: instead of operations
we only get
• As before, for each iteration , the alphabet size is s*M.
• Total complexity:
• No matrix-product switch here!
lsl 2/3,
1,,1 r
MnOnO
MnOMnnOsMnOr
rrrrrrr
22
)1111,,11,,
1,,2 rr
Performance
• For small M, as long as the number of queries is less than we get better results than APSP.
• For small M:– The algorithm overtakes Goldberg’s algorithm, if
the graph is dense– For a dense-enough graph , we can run
many SSSP queries and still be faster:
58.1158.2 nn
88.138.2~ nOmnOnmO 88.1nm
nmnnm //
The larger picture• We saw:
– Alg1: heavy pre-processing, light query– Alg2: light pre-processing, heavy query– Alg3: ?
Query-oriented (APSP)
Preprocess- oriented (pre-process)
The Third Way• Suppose we know in advance the we require
no more than queries.• We use the following:
1. Perform iterations of the APSP algorithm
2. Perform iterations of the pre-process algorithm
3. Take the matrix B from the last step of step 1. The product returns in any shortest-distance query.
rnq
nr 5.1log1
nr 5.1log
rnO }{,},{ jBDBiD
Huh?
• After the first stage D holds all the shortest path of all “short” paths, of lengths at most with high probability.
• When the second starts stage it can be sure that the induction holds for all
• The second stage takes care of the “long” paths, with respect to querying. Meaning:– If the path is long it will have a representative in
one of the second-phase iterations– If it is too-short – it will fall under the jurisdiction
of the first stage.
rn 1
nrl 5.1log1
Complexity
• The first stage ( updates) costs at most
• The second stage costs only
• The query costs
• For example – if want to answer a distance query in , we can pre-process in time
2n 1,,11 rrMnO
1,,1 rrrMnO
rnO
4/3nO 5.2MnO
Q&A (I ask - you answer)
• Q: Why couldn’t we sample B in the query step of Alg2 – the one that initially costs O(n)?
• A: Because if the path is too short – we will have no guarantee that it will have a representative in B. Alg3 solves this because short distances are computed rigorously.
• Conclusion: the less we sample out of V when we query, the more steps we need to run APSP to begin with.
Final Procedure
• Given q queries, determine the query complexity using .
• This assumes M is small enough so that we use fast product. Otherwise compare to
• Execute alg3 using steps of APSP and steps of pre-process
• Query all q queries.
1,,11 rrr Mnqn
qnr rn 2
nr 5.1log1 nr 5.1log
Summary
• For the problem we defined: directed graph, with integer weights, whose absolute value is at most M, we have seen:– Alg1: State-of-the-art APSP in– Alg2: State-of-the-art SSSP in– Alg3: A method to calibrate between the two, for a
known number of queries.
58.268.0 nMO
38.2MnO
Thank You!