The Shortest Path Problem - Max Planck Societybast/slides/transit-stefan.pdf · The Shortest Path...
Transcript of The Shortest Path Problem - Max Planck Societybast/slides/transit-stefan.pdf · The Shortest Path...
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
The Shortest Path Problem
◮ showcase problem for the power of algorithmics
◮ for general graphs with non-negative edge weights, exactsolution given by Dijkstra’s algorithm in O(m + n log n) wheren =# nodes, m =# edges
2/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
The Shortest Path Problem
◮ showcase problem for the power of algorithmics
◮ for general graphs with non-negative edge weights, exactsolution given by Dijkstra’s algorithm in O(m + n log n) wheren =# nodes, m =# edges
◮ Example: roadmap of the US, (n = 24 Mio m = 58 Mio)
2/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
The Shortest Path Problem
◮ showcase problem for the power of algorithmics
◮ for general graphs with non-negative edge weights, exactsolution given by Dijkstra’s algorithm in O(m + n log n) wheren =# nodes, m =# edges
◮ Example: roadmap of the US, (n = 24 Mio m = 58 Mio)
◮ Good Dijkstra implementation takes around 10secs to answera random source-target query
◮ ⇒ infeasible for a web-based route planner
2/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
The Shortest Path Problem
◮ showcase problem for the power of algorithmics
◮ for general graphs with non-negative edge weights, exactsolution given by Dijkstra’s algorithm in O(m + n log n) wheren =# nodes, m =# edges
◮ Example: roadmap of the US, (n = 24 Mio m = 58 Mio)
◮ Good Dijkstra implementation takes around 10secs to answera random source-target query
◮ ⇒ infeasible for a web-based route planner
◮ ⇒ need to exploit the special structure of roadmaps
2/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
The Shortest Path Problem
◮ showcase problem for the power of algorithmics
◮ for general graphs with non-negative edge weights, exactsolution given by Dijkstra’s algorithm in O(m + n log n) wheren =# nodes, m =# edges
◮ Example: roadmap of the US, (n = 24 Mio m = 58 Mio)
◮ Good Dijkstra implementation takes around 10secs to answera random source-target query
◮ ⇒ infeasible for a web-based route planner
◮ ⇒ need to exploit the special structure of roadmaps
◮ so far, best solutions allow for a query time in the order ofmilliseconds (with preprocessing)
2/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
The Shortest Path Problem
◮ showcase problem for the power of algorithmics
◮ for general graphs with non-negative edge weights, exactsolution given by Dijkstra’s algorithm in O(m + n log n) wheren =# nodes, m =# edges
◮ Example: roadmap of the US, (n = 24 Mio m = 58 Mio)
◮ Good Dijkstra implementation takes around 10secs to answera random source-target query
◮ ⇒ infeasible for a web-based route planner
◮ ⇒ need to exploit the special structure of roadmaps
◮ so far, best solutions allow for a query time in the order ofmilliseconds (with preprocessing)
◮ Remark: we are only talking about exact queries!
2/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Extremely Useful Insight No.1 (Gutman’04)
Only few roads appear in ”the middle” of long shortest paths!
3/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Extremely Useful Insight No.1 (Gutman’04)
Only few roads appear in ”the middle” of long shortest paths!
◮ used to prune the set of edges Dijkstra has to look at
◮ two metrics: one determines what shortest paths are, theother what in the middle means
3/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Extremely Useful Insight No.1 (Gutman’04)
Only few roads appear in ”the middle” of long shortest paths!
◮ used to prune the set of edges Dijkstra has to look at
◮ two metrics: one determines what shortest paths are, theother what in the middle means
◮ extensively exploited e.g. in papers by Gutman, Goldberg etal. or Sanders/Schultes
3/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Extremely Useful Insight No.1 (Gutman’04)
Only few roads appear in ”the middle” of long shortest paths!
◮ used to prune the set of edges Dijkstra has to look at
◮ two metrics: one determines what shortest paths are, theother what in the middle means
◮ extensively exploited e.g. in papers by Gutman, Goldberg etal. or Sanders/Schultes
◮ yields shortest path queries in the order of milliseconds on theUS roadmap (after preprocessing)
3/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Extremely Useful Insight No.1 (Gutman’04)
Only few roads appear in ”the middle” of long shortest paths!
◮ used to prune the set of edges Dijkstra has to look at
◮ two metrics: one determines what shortest paths are, theother what in the middle means
◮ extensively exploited e.g. in papers by Gutman, Goldberg etal. or Sanders/Schultes
◮ yields shortest path queries in the order of milliseconds on theUS roadmap (after preprocessing)
◮ any sensible reason to aim for faster query times ?⇒YES! Web services, traffic simulations, logistics . . .
3/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Some ways to speed up DijkstraOrdinary Dijkstra
Source Target
4/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Some ways to speed up DijkstraGoal-directed Dijkstra
Source Target
4/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Some ways to speed up DijkstraBidirectional Dijkstra
Source Target
4/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Shortest-Path Queries – State of the Art
Some ways to speed up DijkstraDijkstra with with edge levels/hierarchies
Source Target
4/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
First Contribution: Extremely Useful Insight No.2
Imagine you aim to travel ’far’– let’s say more than 50 miles –
how many different routes would youpotentially use to leave your local
neighborhood ?
5/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
First Contribution: Extremely Useful Insight No.2
Imagine you aim to travel ’far’– let’s say more than 50 miles –
how many different routes would youpotentially use to leave your local
neighborhood ?
Only VERY few!
5/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Copenhagen
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Berlin
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Vienna
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Munich
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Rome
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Paris
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → London
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Brussels
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Copenhagen
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Berlin
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Vienna
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Munich
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Rome
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Paris
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → London
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Example: Karlsruhe → Brussels
6/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Second Contribution: Exploitation of our Insight
Based on our new insight we propose Transit Node Routing, ahighly efficient scheme which allows for
7/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Second Contribution: Exploitation of our Insight
Based on our new insight we propose Transit Node Routing, ahighly efficient scheme which allows for
◮ reduction of shortest path distance queries to a constantnumber of table-lookups
7/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Second Contribution: Exploitation of our Insight
Based on our new insight we propose Transit Node Routing, ahighly efficient scheme which allows for
◮ reduction of shortest path distance queries to a constantnumber of table-lookups
◮ various trade-offs between space and query-/preprocessingtimes:
◮ avg. query times between 5µs and 63µs (on the US road map)◮ preprocessing times between 1h and 20h◮ a per node space overhead of 21 to 244 bytes
7/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Second Contribution: Exploitation of our Insight
Based on our new insight we propose Transit Node Routing, ahighly efficient scheme which allows for
◮ reduction of shortest path distance queries to a constantnumber of table-lookups
◮ various trade-offs between space and query-/preprocessingtimes:
◮ avg. query times between 5µs and 63µs (on the US road map)◮ preprocessing times between 1h and 20h◮ a per node space overhead of 21 to 244 bytes
◮ ⇒ Query times orders of magnitudes better than previouslyreported results
Milliseconds (10−3) vs. Microseconds (10−6)!
7/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Our Contribution
Dissemination
◮ follow-up results partly in collaboration with Peter Sanders(Karlsruhe, former MPI) and Dominik Schultes (Karlsruhe)
◮ published in◮ DIMACS ”Shortest Path” Workshop 2006◮ 9th Workshop on Algorithm Engineering and Experimentation
(ALENEX) 2007◮ Science, Apr 27th 2007
◮ press/TV coverage:◮ die Welt, Saarbrucker Zeitung (German newspapers)◮ connect, c’t (German magazines)◮ Saarlandischer Rundfunk, SaarTV (TV stations)
◮ (US) patent on the way
◮ Contacts with several companies in the navigation systemsmarket
8/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Nodes: Formalization
Consider the set Π of all ’long’ shortest paths within the network.We want to find a set of Transit Nodes T such that
9/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Nodes: Formalization
Consider the set Π of all ’long’ shortest paths within the network.We want to find a set of Transit Nodes T such that
◮ every π ∈ Π contains a node from T (global hitting set)
9/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Nodes: Formalization
Consider the set Π of all ’long’ shortest paths within the network.We want to find a set of Transit Nodes T such that
◮ every π ∈ Π contains a node from T (global hitting set)
◮ |T | = O(√
|V |)
9/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Nodes: Formalization
Consider the set Π of all ’long’ shortest paths within the network.We want to find a set of Transit Nodes T such that
◮ every π ∈ Π contains a node from T (global hitting set)
◮ |T | = O(√
|V |)
That comes as no surprise given the previous results, but with thenew insight we might even want the following property:
9/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Nodes: Formalization
Consider the set Π of all ’long’ shortest paths within the network.We want to find a set of Transit Nodes T such that
◮ every π ∈ Π contains a node from T (global hitting set)
◮ |T | = O(√
|V |)
That comes as no surprise given the previous results, but with thenew insight we might even want the following property:
◮ for every node v there is a set of Access Nodes A(v) ⊂ Twhich hits all ’long’ paths starting at v and |A(v)| is constant
9/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Nodes: Formalization
Consider the set Π of all ’long’ shortest paths within the network.We want to find a set of Transit Nodes T such that
◮ every π ∈ Π contains a node from T (global hitting set)
◮ |T | = O(√
|V |)
That comes as no surprise given the previous results, but with thenew insight we might even want the following property:
◮ for every node v there is a set of Access Nodes A(v) ⊂ Twhich hits all ’long’ paths starting at v and |A(v)| is constant
What to do with the transit/access nodes ?
9/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node RoutingPreprocessing
◮ determine T and A(v)
10/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node RoutingPreprocessing
◮ determine T and A(v)◮ compute and store :
◮ for each node v its distances to A(v) – O(n) space
10/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node RoutingPreprocessing
◮ determine T and A(v)◮ compute and store :
◮ for each node v its distances to A(v) – O(n) space◮ all pair-wise distances for T × T – O(n) space
10/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node RoutingPreprocessing
◮ determine T and A(v)◮ compute and store :
◮ for each node v its distances to A(v) – O(n) space◮ all pair-wise distances for T × T – O(n) space
Query(s,t)
◮ decide whether path from s to t is ’long’/non-local
10/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node RoutingPreprocessing
◮ determine T and A(v)◮ compute and store :
◮ for each node v its distances to A(v) – O(n) space◮ all pair-wise distances for T × T – O(n) space
Query(s,t)
◮ decide whether path from s to t is ’long’/non-local◮ YES → for every (as , at), as ∈ A(s), at ∈ A(t) evaluate
dist = d(s, as)︸ ︷︷ ︸
stored with s
+
in dist.-table T ×T︷ ︸︸ ︷
d(as , at) + d(at , t)︸ ︷︷ ︸
stored with t
10/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node RoutingPreprocessing
◮ determine T and A(v)◮ compute and store :
◮ for each node v its distances to A(v) – O(n) space◮ all pair-wise distances for T × T – O(n) space
Query(s,t)
◮ decide whether path from s to t is ’long’/non-local◮ YES → for every (as , at), as ∈ A(s), at ∈ A(t) evaluate
dist = d(s, as)︸ ︷︷ ︸
stored with s
+
in dist.-table T ×T︷ ︸︸ ︷
d(as , at) + d(at , t)︸ ︷︷ ︸
stored with t
◮ NO → use favourite SP data structure – HH, edge reach . . .
10/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node Routing
Source Target
11/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node Routing
non−local ?
Source Target
11/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node Routing
Source Target
11/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node Routing
Source Target
11/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node Routing
Source Target
11/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Formalization
Transit Node Routing
Source Target
11/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation
First implementation of the ”transit node” concept.
◮ impose a grid, e.g. 128 × 128over the network
C
outer
inner
12/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation
First implementation of the ”transit node” concept.
◮ impose a grid, e.g. 128 × 128over the network
◮ for every cell C determine itsaccess nodes C(A) which
◮ lie on the inner boundary◮ appear on a shortest path from
some node in C to some nodeon the outer boundary
C
outer
inner
12/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation
First implementation of the ”transit node” concept.
◮ impose a grid, e.g. 128 × 128over the network
◮ for every cell C determine itsaccess nodes C(A) which
◮ lie on the inner boundary◮ appear on a shortest path from
some node in C to some nodeon the outer boundary
◮ ∀v ∈ C we set A(v) = A(C)
C
outer
inner
12/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation
First implementation of the ”transit node” concept.
◮ impose a grid, e.g. 128 × 128over the network
◮ for every cell C determine itsaccess nodes C(A) which
◮ lie on the inner boundary◮ appear on a shortest path from
some node in C to some nodeon the outer boundary
◮ ∀v ∈ C we set A(v) = A(C)
◮ T = ∪A(v)
C
outer
inner
12/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation: What are ’long’ paths?
Path/Query between source s and target t ’long’/non-local⇔
s and t at least 4 grid cells (horiz./vert.) apart
13/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation: Remarks
◮ The construction as described would take days to weeks onthe US roadmap ⇒ more efficient construction via sweep
algorithm takes around 10 h
14/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation: Remarks
◮ The construction as described would take days to weeks onthe US roadmap ⇒ more efficient construction via sweep
algorithm takes around 10 h
◮ Queries for pairs (s, t) less than 4 grid cells apart are handledby a variation of reach-based Dijkstra
14/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation: Remarks
◮ The construction as described would take days to weeks onthe US roadmap ⇒ more efficient construction via sweep
algorithm takes around 10 h
◮ Queries for pairs (s, t) less than 4 grid cells apart are handledby a variation of reach-based Dijkstra
◮ multi-layered implementation also possible
14/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Implementation
Grid-based Implementation: Remarks
◮ The construction as described would take days to weeks onthe US roadmap ⇒ more efficient construction via sweep
algorithm takes around 10 h
◮ Queries for pairs (s, t) less than 4 grid cells apart are handledby a variation of reach-based Dijkstra
◮ multi-layered implementation also possible
◮ can be made very space-efficient
14/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Experiments
Experiments: US roadmap (n = 24 Mio, m = 58 Mio)
Grid |T | |T |×|T |
nodeavg. |A| % ’long’ construction of
queries transit nodes
64 × 64 2 042 0.1 11.4 91.7% 498 min128× 128 7 426 1.1 11.4 97.4% 525 min256× 256 24 899 12.8 10.6 99.2% 638 min512 × 512 89 382 164.6 9.7 99.8% 859 min1 024 × 1 024 351 484 2 545.5 9.1 99.9% 964 min
Statistics for several grid sizes.
15/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Experiments
Experiments: US roadmap (n = 24 Mio, m = 58 Mio)
Grid |T | |T |×|T |
nodeavg. |A| % ’long’ construction of
queries transit nodes
64 × 64 2 042 0.1 11.4 91.7% 498 min128× 128 7 426 1.1 11.4 97.4% 525 min256× 256 24 899 12.8 10.6 99.2% 638 min512 × 512 89 382 164.6 9.7 99.8% 859 min1 024 × 1 024 351 484 2 545.5 9.1 99.9% 964 min
Statistics for several grid sizes.
non-local (99%) local (1%) all queries preproc. space/node
12µs 5112µs 63µs 20h 21 bytes
Results for 2-layer data structure.
15/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
HH-based implementation
Some remarks on the HH-based implementation
◮ follow-up implementation based on highway hierarchies
◮ main idea: take ”high level” network as transit node set
◮ yields naturally a multi-level scheme
◮ very efficient backup for non-local routines
◮ average query times below 10µs at the cost of a higher spaceconsumption (> 200 bytes/node)
16/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Conclusions
Conclusions
◮ new, extremely useful insight which improves avg. query timesby orders of magnitude: Microseconds vs. Milliseconds!
17/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Conclusions
Conclusions
◮ new, extremely useful insight which improves avg. query timesby orders of magnitude: Microseconds vs. Milliseconds!
◮ we developed two ways to exploit this insight:◮ simple, grid-based implementation tuned for space efficiency◮ sophisticated, extremely fast implementation based on HH
17/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Conclusions
Conclusions
◮ new, extremely useful insight which improves avg. query timesby orders of magnitude: Microseconds vs. Milliseconds!
◮ we developed two ways to exploit this insight:◮ simple, grid-based implementation tuned for space efficiency◮ sophisticated, extremely fast implementation based on HH
◮ not mentioned: efficient ways to output actual paths
17/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Conclusions
Conclusions
◮ new, extremely useful insight which improves avg. query timesby orders of magnitude: Microseconds vs. Milliseconds!
◮ we developed two ways to exploit this insight:◮ simple, grid-based implementation tuned for space efficiency◮ sophisticated, extremely fast implementation based on HH
◮ not mentioned: efficient ways to output actual paths
◮ room for improvement in both realizations: grid-based interms of query/preprocessing, HH-based in terms of spaceconsumption
17/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
Conclusions
Conclusions
◮ new, extremely useful insight which improves avg. query timesby orders of magnitude: Microseconds vs. Milliseconds!
◮ we developed two ways to exploit this insight:◮ simple, grid-based implementation tuned for space efficiency◮ sophisticated, extremely fast implementation based on HH
◮ not mentioned: efficient ways to output actual paths
◮ room for improvement in both realizations: grid-based interms of query/preprocessing, HH-based in terms of spaceconsumption
◮ One of the main challenges: deal with dynamics of real-worldnetworks
17/18
Introduction Transit Node Routing Grid-based TN Routing Conclusions
End
Thank you for your attention!
18/18