Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of...
Transcript of Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of...
![Page 1: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/1.jpg)
Felix Friedrich
Data Structures and AlgorithmsCourse at D-MATH (CSE) of ETH ZurichSpring 2020
![Page 2: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/2.jpg)
1. Introduction
Overview, Algorithms and Data Structures, Correctness, First Example
1
![Page 3: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/3.jpg)
Goals of the course
Understand the design and analysis of fundamental algorithms anddata structures.An advanced insight into a modern programming model (with C++).Knowledge about chances, problems and limits of the parallel andconcurrent computing.
2
![Page 4: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/4.jpg)
Contents
data structures / algorithmsThe notion invariant, cost model, Landau notation
algorithms design, inductionsearching, selection and sorting
amortized analysisdynamic programming
dictionaries: hashing and search treesFundamental algorithms on graphs,
shortest paths, Max-Flow
van-Emde Boas Trees, Splay-Trees
Minimum Spanning Trees, Fibonacci Heaps
prorgamming with C++RAII, Move Konstruktion, SmartPointers,Templates and generic programming
Exceptions functors and lambdasthreads, mutex and monitors
promises and futures
parallel programmingparallelism vs. concurrency, speedup(Amdahl/Gustavson), races, memoryreordering, atomir registers, RMW(CAS,TAS), deadlock/starvation
4
![Page 5: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/5.jpg)
1.2 Algorithms
[Cormen et al, Kap. 1; Ottman/Widmayer, Kap. 1.1]
5
![Page 6: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/6.jpg)
Algorithm
Algorithm
Well-defined procedure to compute output data from input data
6
![Page 7: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/7.jpg)
Example Problem: SortingInput: A sequence of n numbers (comparable objects) (a1, a2, . . . , an)
Output: Permutation (a′1, a′2, . . . , a′n) of the sequence (ai)1≤i≤n, such thata′1 ≤ a′2 ≤ · · · ≤ a′n
Possible input
(1, 7, 3), (15, 13, 12,−0.5), (999, 998, 997, 996, . . . , 2, 1), (1), () . . .
Every example represents a problem instance
The performance (speed) of an algorithm usually depends on the probleminstance. Often there are “good” and “bad” instances.
Therefore we consider algorithms sometimes ”in the average“ and mostoften in the ”worst case“.
7
![Page 8: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/8.jpg)
Example Problem: SortingInput: A sequence of n numbers (comparable objects) (a1, a2, . . . , an)Output: Permutation (a′1, a′2, . . . , a′n) of the sequence (ai)1≤i≤n, such that
a′1 ≤ a′2 ≤ · · · ≤ a′n
Possible input
(1, 7, 3), (15, 13, 12,−0.5), (999, 998, 997, 996, . . . , 2, 1), (1), () . . .
Every example represents a problem instance
The performance (speed) of an algorithm usually depends on the probleminstance. Often there are “good” and “bad” instances.
Therefore we consider algorithms sometimes ”in the average“ and mostoften in the ”worst case“.
7
![Page 9: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/9.jpg)
Example Problem: SortingInput: A sequence of n numbers (comparable objects) (a1, a2, . . . , an)Output: Permutation (a′1, a′2, . . . , a′n) of the sequence (ai)1≤i≤n, such that
a′1 ≤ a′2 ≤ · · · ≤ a′n
Possible input
(1, 7, 3), (15, 13, 12,−0.5), (999, 998, 997, 996, . . . , 2, 1), (1), () . . .
Every example represents a problem instance
The performance (speed) of an algorithm usually depends on the probleminstance. Often there are “good” and “bad” instances.
Therefore we consider algorithms sometimes ”in the average“ and mostoften in the ”worst case“.
7
![Page 10: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/10.jpg)
Example Problem: SortingInput: A sequence of n numbers (comparable objects) (a1, a2, . . . , an)Output: Permutation (a′1, a′2, . . . , a′n) of the sequence (ai)1≤i≤n, such that
a′1 ≤ a′2 ≤ · · · ≤ a′n
Possible input
(1, 7, 3), (15, 13, 12,−0.5), (999, 998, 997, 996, . . . , 2, 1), (1), () . . .
Every example represents a problem instance
The performance (speed) of an algorithm usually depends on the probleminstance. Often there are “good” and “bad” instances.
Therefore we consider algorithms sometimes ”in the average“ and mostoften in the ”worst case“.
7
![Page 11: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/11.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searching
routing: shortest path algorithm, heap data structureDNA matching: Dynamic Programmingevaluation order: Topological Sortingautocomletion and spell-checking: Dictionaries / TreesFast Lookup : Hash-TablesThe travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 12: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/12.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searchingrouting: shortest path algorithm, heap data structure
DNA matching: Dynamic Programmingevaluation order: Topological Sortingautocomletion and spell-checking: Dictionaries / TreesFast Lookup : Hash-TablesThe travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 13: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/13.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searchingrouting: shortest path algorithm, heap data structureDNA matching: Dynamic Programming
evaluation order: Topological Sortingautocomletion and spell-checking: Dictionaries / TreesFast Lookup : Hash-TablesThe travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 14: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/14.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searchingrouting: shortest path algorithm, heap data structureDNA matching: Dynamic Programmingevaluation order: Topological Sorting
autocomletion and spell-checking: Dictionaries / TreesFast Lookup : Hash-TablesThe travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 15: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/15.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searchingrouting: shortest path algorithm, heap data structureDNA matching: Dynamic Programmingevaluation order: Topological Sortingautocomletion and spell-checking: Dictionaries / Trees
Fast Lookup : Hash-TablesThe travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 16: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/16.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searchingrouting: shortest path algorithm, heap data structureDNA matching: Dynamic Programmingevaluation order: Topological Sortingautocomletion and spell-checking: Dictionaries / TreesFast Lookup : Hash-Tables
The travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 17: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/17.jpg)
Examples for algorithmic problems
Tables and statistis: sorting, selection and searchingrouting: shortest path algorithm, heap data structureDNA matching: Dynamic Programmingevaluation order: Topological Sortingautocomletion and spell-checking: Dictionaries / TreesFast Lookup : Hash-TablesThe travelling Salesman: Dynamic Programming, Minimum SpanningTree, Simulated Annealing
8
![Page 18: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/18.jpg)
Characteristics
Extremely large number of potential solutionsPractical applicability
9
![Page 19: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/19.jpg)
Data Structures
A data structure is a particular way oforganizing data in a computer so thatthey can be used eciently (in thealgorithms operating on them).Programs = algorithms + data structures.
10
![Page 20: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/20.jpg)
Eciency
If computers were infinitely fast and had an infinite amount of memory ...... then we would still need the theory of algorithms (only) forstatements about correctness (and termination).
Reality: resources are bounded and not free:Computing time→ EciencyStorage space→ Eciency
Actually, this course is nearly only about eciency.
11
![Page 21: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/21.jpg)
Eciency
If computers were infinitely fast and had an infinite amount of memory ...... then we would still need the theory of algorithms (only) forstatements about correctness (and termination).
Reality: resources are bounded and not free:Computing time→ EciencyStorage space→ Eciency
Actually, this course is nearly only about eciency.
11
![Page 22: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/22.jpg)
Hard problems.
NP-complete problems: no known ecient solution (the existence ofsuch a solution is very improbable – but it has not yet been proven thatthere is none!)Example: travelling salesman problem
This course is mostly about problems that can be solved eciently (inpolynomial time).
12
![Page 23: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/23.jpg)
2. Eciency of algorithms
Eciency of Algorithms, Random Access Machine Model, Function Growth,Asymptotics [Cormen et al, Kap. 2.2,3,4.2-4.4 | Ottman/Widmayer, Kap. 1.1]
13
![Page 24: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/24.jpg)
Eciency of Algorithms
GoalsQuantify the runtime behavior of an algorithm independent of themachine.Compare eciency of algorithms.Understand dependece on the input size.
14
![Page 25: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/25.jpg)
Programs and Algorithms
program
programming language
computer
algorithm
pseudo-code
computation model
implemented in
specified for
specified in
based on
Technology Abstraction
15
![Page 26: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/26.jpg)
Technology Model
Random Access Machine (RAM) Model
Execution model: instructions are executed one after the other (onone processor core).
Memory model: constant access time (big array)Fundamental operations: computations (+,−,·,...) comparisons,assignment / copy on machine words (registers), flow control(jumps)Unit cost model: fundamental operations provide a cost of 1.Data types: fundamental types like size-limited integer or floatingpoint number.
16
![Page 27: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/27.jpg)
Technology Model
Random Access Machine (RAM) Model
Execution model: instructions are executed one after the other (onone processor core).Memory model: constant access time (big array)
Fundamental operations: computations (+,−,·,...) comparisons,assignment / copy on machine words (registers), flow control(jumps)Unit cost model: fundamental operations provide a cost of 1.Data types: fundamental types like size-limited integer or floatingpoint number.
16
![Page 28: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/28.jpg)
Technology Model
Random Access Machine (RAM) Model
Execution model: instructions are executed one after the other (onone processor core).Memory model: constant access time (big array)Fundamental operations: computations (+,−,·,...) comparisons,assignment / copy on machine words (registers), flow control(jumps)
Unit cost model: fundamental operations provide a cost of 1.Data types: fundamental types like size-limited integer or floatingpoint number.
16
![Page 29: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/29.jpg)
Technology Model
Random Access Machine (RAM) Model
Execution model: instructions are executed one after the other (onone processor core).Memory model: constant access time (big array)Fundamental operations: computations (+,−,·,...) comparisons,assignment / copy on machine words (registers), flow control(jumps)Unit cost model: fundamental operations provide a cost of 1.
Data types: fundamental types like size-limited integer or floatingpoint number.
16
![Page 30: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/30.jpg)
Technology Model
Random Access Machine (RAM) Model
Execution model: instructions are executed one after the other (onone processor core).Memory model: constant access time (big array)Fundamental operations: computations (+,−,·,...) comparisons,assignment / copy on machine words (registers), flow control(jumps)Unit cost model: fundamental operations provide a cost of 1.Data types: fundamental types like size-limited integer or floatingpoint number.
16
![Page 31: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/31.jpg)
Size of the Input Data
Typical: number of input objects (of fundamental type).Sometimes: number bits for a reasonable / cost-eective representationof the data.fundamental types fit into word of size : w ≥ log(sizeof(mem)) bits.
17
![Page 32: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/32.jpg)
For Dynamic Data Strcutures
Pointer Machine ModelObjects bounded in size can be dynamically allocated in constanttimeFields (with word-size) of the objects can be accessed in constanttime 1.
top xn xn−1 x1 null
18
![Page 33: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/33.jpg)
Asymptotic behavior
An exact running time of an algorithm can normally not be predicted evenfor small input data.
We consider the asymptotic behavior of the algorithm.And ignore all constant factors.
An operation with cost 20 is no worse than one with cost 1Linear growth with gradient 5 is as good as linear growth with gradient1.
19
![Page 34: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/34.jpg)
2.2 Function growth
O, Θ, Ω [Cormen et al, Kap. 3; Ottman/Widmayer, Kap. 1.1]
21
![Page 35: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/35.jpg)
Superficially
Use the asymptotic notation to specify the execution time of algorithms.We write Θ(n2) and mean that the algorithm behaves for large n like n2:when the problem size is doubled, the execution time multiplies by four.
22
![Page 36: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/36.jpg)
More precise: asymptotic upper bound
provided: a function g : N→ R.Definition:1
O(g) = f : N→ R|∃ c > 0,∃n0 ∈ N :∀ n ≥ n0 : 0 ≤ f(n) ≤ c · g(n)
Notation:O(g(n)) := O(g(·)) = O(g).
1Ausgesprochen: Set of all functions f : N→ R that satisfy: there is some (realvalued) c > 0 and some n0 ∈ N such that 0 ≤ f(n) ≤ n · g(n) for all n ≥ n0.
23
![Page 37: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/37.jpg)
Graphic
g(n) = n2
f ∈ O(g)
n0 n
24
![Page 38: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/38.jpg)
Graphic
g(n) = n2
f ∈ O(g)
h ∈ O(g)
n0
n
24
![Page 39: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/39.jpg)
Converse: asymptotic lower bound
Given: a function g : N→ R.Definition:
Ω(g) = f : N→ R|∃ c > 0,∃n0 ∈ N :∀ n ≥ n0 : 0 ≤ c · g(n) ≤ f(n)
25
![Page 40: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/40.jpg)
Example
g(n) = n
f ∈ Ω(g)
n0 n
26
![Page 41: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/41.jpg)
Example
g(n) = n
f ∈ Ω(g)h ∈ Ω(g)
n0 n
26
![Page 42: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/42.jpg)
Asymptotic tight bound
Given: function g : N→ R.Definition:
Θ(g) := Ω(g) ∩ O(g).
Simple, closed form: exercise.
27
![Page 43: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/43.jpg)
Example
g(n) = n2
f ∈ Θ(n2)
h(n) = 0.5 · n2
n
28
![Page 44: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/44.jpg)
Notions of Growth
O(1) bounded array accessO(log logn) double logarithmic interpolated binary sorted sortO(logn) logarithmic binary sorted searchO(√n) like the square root naive prime number test
O(n) linear unsorted naive searchO(n logn) superlinear / loglinear good sorting algorithmsO(n2) quadratic simple sort algorithmsO(nc) polynomial matrix multiplyO(2n) exponential Travelling Salesman Dynamic ProgrammingO(n!) factorial Travelling Salesman naively
29
![Page 45: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/45.jpg)
Small n
2 3 4 5 6
20
40
60
lnnn
n2
n4 2n
30
![Page 46: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/46.jpg)
Larger n
5 10 15 20
0.2
0.4
0.6
0.8
1·106
lognnn2
n4
2n
31
![Page 47: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/47.jpg)
“Large” n
20 40 60 80 100
0.2
0.4
0.6
0.8
1·1020
lognnn2n4
2n
32
![Page 48: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/48.jpg)
Logarithms
10 20 30 40 50
200
400
600
800
1,000
n
n2
n3/2
logn
n logn
33
![Page 49: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/49.jpg)
Time Consumption
Assumption 1 Operation = 1µs.
problem size 1 100 10000 106 109
log2 n 1µs
7µs 13µs 20µs 30µs
n 1µs
100µs 1/100s 1s 17 minutes
n log2 n 1µs
700µs 13/100µs 20s 8.5 hours
n2 1µs
1/100s 1.7 minutes 11.5 days 317 centuries
2n 1µs
1014 centuries ≈ ∞ ≈ ∞ ≈ ∞
34
![Page 50: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/50.jpg)
Time Consumption
Assumption 1 Operation = 1µs.
problem size 1 100 10000 106 109
log2 n 1µs
7µs 13µs 20µs 30µs
n 1µs 100µs 1/100s 1s 17 minutes
n log2 n 1µs
700µs 13/100µs 20s 8.5 hours
n2 1µs
1/100s 1.7 minutes 11.5 days 317 centuries
2n 1µs
1014 centuries ≈ ∞ ≈ ∞ ≈ ∞
34
![Page 51: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/51.jpg)
Time Consumption
Assumption 1 Operation = 1µs.
problem size 1 100 10000 106 109
log2 n 1µs
7µs 13µs 20µs 30µs
n 1µs 100µs 1/100s 1s 17 minutes
n log2 n 1µs
700µs 13/100µs 20s 8.5 hours
n2 1µs 1/100s 1.7 minutes 11.5 days 317 centuries
2n 1µs
1014 centuries ≈ ∞ ≈ ∞ ≈ ∞
34
![Page 52: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/52.jpg)
Time Consumption
Assumption 1 Operation = 1µs.
problem size 1 100 10000 106 109
log2 n 1µs 7µs 13µs 20µs 30µs
n 1µs 100µs 1/100s 1s 17 minutes
n log2 n 1µs
700µs 13/100µs 20s 8.5 hours
n2 1µs 1/100s 1.7 minutes 11.5 days 317 centuries
2n 1µs
1014 centuries ≈ ∞ ≈ ∞ ≈ ∞
34
![Page 53: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/53.jpg)
Time Consumption
Assumption 1 Operation = 1µs.
problem size 1 100 10000 106 109
log2 n 1µs 7µs 13µs 20µs 30µs
n 1µs 100µs 1/100s 1s 17 minutes
n log2 n 1µs 700µs 13/100µs 20s 8.5 hours
n2 1µs 1/100s 1.7 minutes 11.5 days 317 centuries
2n 1µs
1014 centuries ≈ ∞ ≈ ∞ ≈ ∞
34
![Page 54: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/54.jpg)
Time Consumption
Assumption 1 Operation = 1µs.
problem size 1 100 10000 106 109
log2 n 1µs 7µs 13µs 20µs 30µs
n 1µs 100µs 1/100s 1s 17 minutes
n log2 n 1µs 700µs 13/100µs 20s 8.5 hours
n2 1µs 1/100s 1.7 minutes 11.5 days 317 centuries
2n 1µs 1014 centuries ≈ ∞ ≈ ∞ ≈ ∞
34
![Page 55: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/55.jpg)
About the Notation
Common casual notationf = O(g)
should be read as f ∈ O(g).Clearly it holds that
f1 = O(g), f2 = O(g) 6⇒ f1 = f2!
n = O(n2), n2 = O(n2) but naturally n 6= n2.
We avoid this notation where it could lead to ambiguities.
36
![Page 56: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/56.jpg)
Reminder: Eciency: Arrays vs. Linked Lists
Memory: our avec requires roughly n ints (vector size n), our llvecroughly 3n ints (a pointer typically requires 8 byte)
Runtime (with avec = std::vector, llvec = std::list):
37
![Page 57: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/57.jpg)
Asymptotic Runtimes
With our new language (Ω,O,Θ), we can now state the behavior of thedata structures and their algorithms more precisely
Typical asymptotic running times (Anticipation!)
Data structure RandomAccess
Insert Next InsertAfterElement
Search
std::vector Θ(1) Θ(1)A Θ(1) Θ(n) Θ(n)std::list Θ(n) Θ(1) Θ(1) Θ(1) Θ(n)std::set – Θ(logn) Θ(logn) – Θ(logn)std::unordered_set – Θ(1)P – – Θ(1)PA = amortized, P=expected, otherwise worst case
38
![Page 58: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/58.jpg)
Complexity
Complexity of a problem P
minimal (asymptotic) costs over all algorithms A that solve P .
Complexity of the single-digit multiplication of two numbers with n dig-its is Ω(n) and O(nlog3 2) (Karatsuba Ofman).
39
![Page 59: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/59.jpg)
Complexity
Complexity of a problem P
minimal (asymptotic) costs over all algorithms A that solve P .
Complexity of the single-digit multiplication of two numbers with n dig-its is Ω(n) and O(nlog3 2) (Karatsuba Ofman).
39
![Page 60: Data Structures and Algorithms - ETH Z · Data Structures and Algorithms Course at D-MATH (CSE) of ETH Zurich Spring 2020. 1. Introduction Overview, Algorithms and Data Structures,](https://reader033.fdocuments.us/reader033/viewer/2022053101/605de5855fb262657e746773/html5/thumbnails/60.jpg)
Complexity
Problem Complexity O(n) O(n) O(n2) Ω(n log n)⇑ ⇑ ⇑ ⇓
Algorithm Costs2 3n− 4 O(n) Θ(n2) Ω(n log n)⇓ m m ⇓
Program Execution time Θ(n) O(n) Θ(n2) Ω(n log n)
2Number fundamental operations40