Big O Notation - Measuring programming progress by lines of … · Big O Notation...
78
Big O Notation “Measuring programming progress by lines of code is like measuring aircraft building progress by weight." --Bill Gates Nabil M. Al-Rousan
Transcript of Big O Notation - Measuring programming progress by lines of … · Big O Notation...
Big O Notation
“Measuring programming progressby lines of code is like measuringaircraft building progress by weight." --Bill Gates
Nabil M. Al-Rousan
Example
int linear_search(int[] arr, int target) {for (int i = 0; i < arr.length; i++) {
if (arr[i] == target)return i;
}return -1;
}
1/17
ExampleHow much time does it take to run this function?
int linear_search(int[] arr, int target) {for (int i = 0; i < arr.length; i++) {
if (arr[i] == target)return i;
}return -1;
}
1/17
Absolute Time vs. Time Growth2
0 20000000 40000000 60000000 80000000 100000000
Input size n
0 100
5 107
1 108
1.5 108
2 108
2.5 108
Runtime in nanoseconds
Linear search runtime
refline
1. How much time does it take to run this function?Answer: .1 seconds for 100× 106 array size
2. How does the runtime1 of this function grow?Answer: Linear
1runtime: time it takes to execute a piece of code2a.k.a Complexity
2/17
Absolute Time vs. Time Growth2
0 20000000 40000000 60000000 80000000 100000000
Input size n
0 100
5 107
1 108
1.5 108
2 108
2.5 108
Runtime in nanoseconds
Linear search runtime
refline
1. How much time does it take to run this function?Answer: .1 seconds for 100× 106 array size
2. How does the runtime1 of this function grow?Answer: Linear
1runtime: time it takes to execute a piece of code2a.k.a Complexity
2/17
Absolute Time vs. Time Growth2
0 20000000 40000000 60000000 80000000 100000000
Input size n
0 100
5 107
1 108
1.5 108
2 108
2.5 108
Runtime in nanoseconds
Linear search runtime
refline
1. How much time does it take to run this function?Answer: .1 seconds for 100× 106 array size
2. How does the runtime1 of this function grow?Answer: Linear
1runtime: time it takes to execute a piece of code2a.k.a Complexity
2/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth Analysis: LinearCan we analyze code to find runtime growth?
int linear_search ( int [] arr , int target ) {
for ( int i = 0; i < arr . length ; i ++) {
if ( arr [ i ] == target )
return i ;}return -1;
}
Linear time
Constant time
T(n) =⏞ ⏟ a× n + b
< a× n
< n
=O (n)
1. add different steps
2. drop non-dominate terms
3. drop constants
3/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Runtime Growth in terms of Big O
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
4/17
Big O Comparison
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
Growth of different Big O notations
5/17
100 ?= 10× 10
How much increasing the input affect the growth rate?
Big O Notation Operations for input size10
Operations for input size100
O (1) 1 1O (log n) 3.3219 6.6439O (n) 10 100O (n log n) 33.219 664.39O (n2) 100 10000O (2n) 1024 1267650600228229
401496703205376O (n!) 3628800 9.332621544394415268
1699238856267e+157
6/17
100 ?= 10× 10
How much increasing the input affect the growth rate?
Big O Notation Operations for input size10
Operations for input size100
O (1) 1 1O (log n) 3.3219 6.6439O (n) 10 100O (n log n) 33.219 664.39O (n2) 100 10000O (2n) 1024 1267650600228229
401496703205376O (n!) 3628800 9.332621544394415268
1699238856267e+157
6/17
100 ?= 10× 10
How much increasing the input affect the growth rate?
Big O Notation Operations for input size10
Operations for input size100
O (1) 1 1O (log n) 3.3219 6.6439O (n) 10 100O (n log n) 33.219 664.39O (n2) 100 10000O (2n) 1024 1267650600228229
401496703205376O (n!) 3628800 9.332621544394415268
1699238856267e+157
6/17
100 ?= 10× 10
How much increasing the input affect the growth rate?
Big O Notation Operations for input size10
Operations for input size100
O (1) 1 1O (log n) 3.3219 6.6439 (Twice the time!)O (n) 10 100 (10 times)O (n log n) 33.219 664.39O (n2) 100 10000O (2n) 1024 1267650600228229
401496703205376O (n!) 3628800 9.332621544394415268
1699238856267e+157
6/17
100 ?= 10× 10
How much increasing the input affect the growth rate?
Big O Notation Operations for input size10
Operations for input size100
O (1) 1 1O (log n) 3.3219 6.6439O (n) 10 100O (n log n) 33.219 664.39O (n2) 100 10000O (2n) 1024 1267650600228229
401496703205376O (n!) 3628800 9.332621544394415268
1699238856267e+157
6/17
�����
������:Time Efficiency
Runtime Growth in terms of Big O
�����
���:Time Efficiency
Runtime Growth Big O Notation
constant O (1)logarithmic O (log n)
linear O (n)loglinear O (n log n)quadratic O (n2)exponential O (2n)factorial O (n!)
7/17
Big O Applications
• compares the performance of different algorithms– searching (linear search vs. binary search)– sorting (insertion sort, bubble sort, merge sort etc.)
• describes the worst-case scenario of an algorithmi n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {i f ( a r r [ i ] == t a r g e t )
r e t u r n i ;}r e t u r n − 1;
}
– What if target was at first index?
8/17
Big O Applications
• compares the performance of different algorithms– searching (linear search vs. binary search)– sorting (insertion sort, bubble sort, merge sort etc.)
• describes the worst-case scenario of an algorithmi n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {i f ( a r r [ i ] == t a r g e t )
r e t u r n i ;}r e t u r n − 1;
}
– What if target was at first index?
8/17
Big O Applications
• compares the performance of different algorithms– searching (linear search vs. binary search)– sorting (insertion sort, bubble sort, merge sort etc.)
• describes the worst-case scenario of an algorithmi n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {i f ( a r r [ i ] == t a r g e t )
r e t u r n i ;}r e t u r n − 1;
}
– What if target was at first index?
8/17
Big O Applications
• compares the performance of different algorithms– searching (linear search vs. binary search)– sorting (insertion sort, bubble sort, merge sort etc.)
• describes the worst-case scenario of an algorithmi n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {i f ( a r r [ i ] == t a r g e t )
r e t u r n i ;}r e t u r n − 1;
}
– What if target was at first index?
8/17
Big O Applications
• compares the performance of different algorithms– searching (linear search vs. binary search)– sorting (insertion sort, bubble sort, merge sort etc.)
• describes the worst-case scenario of an algorithmi n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {i f ( a r r [ i ] == t a r g e t )
r e t u r n i ;}r e t u r n − 1;
}
– What if target was at first index?
8/17
O (1) - Constant time
• Algorithm executes in thesame execution timeregardless of the size ofthe data set.
• Examples:– determining if a
number is even orodd.
– using a constant-sizelookup table or hashtable.
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t r e t u r n _ f i r s t ( i n t [ ] a r r ) {r e t u r n a r r [ 0 ] ;
}
9/17
O (1) - Constant time
• Algorithm executes in thesame execution timeregardless of the size ofthe data set.
• Examples:– determining if a
number is even orodd.
– using a constant-sizelookup table or hashtable.
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t r e t u r n _ f i r s t ( i n t [ ] a r r ) {r e t u r n a r r [ 0 ] ;
}
9/17
O (1) - Constant time
• Algorithm executes in thesame execution timeregardless of the size ofthe data set.
• Examples:– determining if a
number is even orodd.
– using a constant-sizelookup table or hashtable.
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t r e t u r n _ f i r s t ( i n t [ ] a r r ) {r e t u r n a r r [ 0 ] ;
}
9/17
O (1) - Constant time
• Algorithm executes in thesame execution timeregardless of the size ofthe data set.
• Examples:– determining if a
number is even orodd.
– using a constant-sizelookup table or hashtable.
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t r e t u r n _ f i r s t ( i n t [ ] a r r ) {r e t u r n a r r [ 0 ] ;
}
9/17
O (1) - Constant time
• Algorithm executes in thesame execution timeregardless of the size ofthe data set.
• Examples:– determining if a
number is even orodd.
– using a constant-sizelookup table or hashtable.
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t r e t u r n _ f i r s t ( i n t [ ] a r r ) {r e t u r n a r r [ 0 ] ;
}
9/17
O (1) - Constant time
• Algorithm executes in thesame execution timeregardless of the size ofthe data set.
• Examples:– determining if a
number is even orodd.
– using a constant-sizelookup table or hashtable.
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t r e t u r n _ f i r s t ( i n t [ ] a r r ) {r e t u r n a r r [ 0 ] ;
}
9/17
O (n) - Linear time
• Execution time of analgorithm∝ size of thedata n.
• Examples:– Traversing an array,
linked list, .. etc.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n tt a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i++ ) {
i f ( a r r [ i ] == t a r g e t )r e t u r n i ;
}r e t u r n − 1;
}
10/17
O (n) - Linear time
• Execution time of analgorithm∝ size of thedata n.
• Examples:– Traversing an array,
linked list, .. etc.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n tt a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i++ ) {
i f ( a r r [ i ] == t a r g e t )r e t u r n i ;
}r e t u r n − 1;
}
10/17
O (n) - Linear time
• Execution time of analgorithm∝ size of thedata n.
• Examples:– Traversing an array,
linked list, .. etc.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n tt a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i++ ) {
i f ( a r r [ i ] == t a r g e t )r e t u r n i ;
}r e t u r n − 1;
}
10/17
O (n) - Linear time
• Execution time of analgorithm∝ size of thedata n.
• Examples:– Traversing an array,
linked list, .. etc.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n tt a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i++ ) {
i f ( a r r [ i ] == t a r g e t )r e t u r n i ;
}r e t u r n − 1;
}
10/17
O (n) - Linear time
• Execution time of analgorithm∝ size of thedata n.
• Examples:– Traversing an array,
linked list, .. etc.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t l i n e a r _ s e a r c h ( i n t [ ] a r r , i n tt a r g e t ) {
f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i++ ) {
i f ( a r r [ i ] == t a r g e t )r e t u r n i ;
}r e t u r n − 1;
}
10/17
O (log n) - Logarithmic time
• Algorithm cuts theproblem space in half ineach iteration.
• Examples:– Traversing a sorted
array using binarysearch 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t b i n a r y _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t , i n t s t a r t , i n t f i n i s h ) {i n t mid = s t a r t + ( f i n i s h − s t a r t ) / 2 ;i f ( f i n i s h >= s t a r t ) { / / r e c u r s i v e s t ep s
i f ( a r r [ mid ] == t a r g e t )r e t u r n mid ;
e l s e i f ( a r r [ mid ] > t a r g e t )r e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , s t a r t , mid − 1 ) ;
e l s er e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , mid + 1 , f i n i s h ) ;
}r e t u r n − 1; / / base case
} 11/17
O (log n) - Logarithmic time
• Algorithm cuts theproblem space in half ineach iteration.
• Examples:– Traversing a sorted
array using binarysearch 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t b i n a r y _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t , i n t s t a r t , i n t f i n i s h ) {i n t mid = s t a r t + ( f i n i s h − s t a r t ) / 2 ;i f ( f i n i s h >= s t a r t ) { / / r e c u r s i v e s t ep s
i f ( a r r [ mid ] == t a r g e t )r e t u r n mid ;
e l s e i f ( a r r [ mid ] > t a r g e t )r e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , s t a r t , mid − 1 ) ;
e l s er e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , mid + 1 , f i n i s h ) ;
}r e t u r n − 1; / / base case
} 11/17
O (log n) - Logarithmic time
• Algorithm cuts theproblem space in half ineach iteration.
• Examples:– Traversing a sorted
array using binarysearch 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t b i n a r y _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t , i n t s t a r t , i n t f i n i s h ) {i n t mid = s t a r t + ( f i n i s h − s t a r t ) / 2 ;i f ( f i n i s h >= s t a r t ) { / / r e c u r s i v e s t ep s
i f ( a r r [ mid ] == t a r g e t )r e t u r n mid ;
e l s e i f ( a r r [ mid ] > t a r g e t )r e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , s t a r t , mid − 1 ) ;
e l s er e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , mid + 1 , f i n i s h ) ;
}r e t u r n − 1; / / base case
} 11/17
O (log n) - Logarithmic time
• Algorithm cuts theproblem space in half ineach iteration.
• Examples:– Traversing a sorted
array using binarysearch 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t b i n a r y _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t , i n t s t a r t , i n t f i n i s h ) {i n t mid = s t a r t + ( f i n i s h − s t a r t ) / 2 ;i f ( f i n i s h >= s t a r t ) { / / r e c u r s i v e s t ep s
i f ( a r r [ mid ] == t a r g e t )r e t u r n mid ;
e l s e i f ( a r r [ mid ] > t a r g e t )r e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , s t a r t , mid − 1 ) ;
e l s er e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , mid + 1 , f i n i s h ) ;
}r e t u r n − 1; / / base case
} 11/17
O (log n) - Logarithmic time
• Algorithm cuts theproblem space in half ineach iteration.
• Examples:– Traversing a sorted
array using binarysearch 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
i n t b i n a r y _ s e a r c h ( i n t [ ] a r r , i n t t a r g e t , i n t s t a r t , i n t f i n i s h ) {i n t mid = s t a r t + ( f i n i s h − s t a r t ) / 2 ;i f ( f i n i s h >= s t a r t ) { / / r e c u r s i v e s t ep s
i f ( a r r [ mid ] == t a r g e t )r e t u r n mid ;
e l s e i f ( a r r [ mid ] > t a r g e t )r e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , s t a r t , mid − 1 ) ;
e l s er e t u r n b i n a r y _ s e a r c h ( a r r , t a r g e t , mid + 1 , f i n i s h ) ;
}r e t u r n − 1; / / base case
} 11/17
O (log n) vs O (n)Binary Vs. Linear search
0 200 400 600 800 1000 1200
Input size n (x105)
0
0.5
1
1.5
2
2.5
Ru
ntim
e
106 Runtime in nanoseconds
Linear search
Binary search
1559975 vs. 15576 ns at last input size 12/17
O (n2) - Quadratic time
• Execution time of analgorithm∝ square ofthe size of the data n.
• Examples:– Bubble, Selection, and
Insertion sorts.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
boolean d u p l i c a t e s _ e x i s t ( i n t [ ] a r r ) {f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {
f o r ( i n t j = i + 1 ; j < a r r . l e n g t h ; j ++ ) {i f ( a r r [ i ] == a r r [ j ] )
r e t u r n t r ue ;}
}r e t u r n f a l s e ;
}
13/17
O (n2) - Quadratic time
• Execution time of analgorithm∝ square ofthe size of the data n.
• Examples:– Bubble, Selection, and
Insertion sorts.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
boolean d u p l i c a t e s _ e x i s t ( i n t [ ] a r r ) {f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {
f o r ( i n t j = i + 1 ; j < a r r . l e n g t h ; j ++ ) {i f ( a r r [ i ] == a r r [ j ] )
r e t u r n t r ue ;}
}r e t u r n f a l s e ;
}
13/17
O (n2) - Quadratic time
• Execution time of analgorithm∝ square ofthe size of the data n.
• Examples:– Bubble, Selection, and
Insertion sorts.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
boolean d u p l i c a t e s _ e x i s t ( i n t [ ] a r r ) {f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {
f o r ( i n t j = i + 1 ; j < a r r . l e n g t h ; j ++ ) {i f ( a r r [ i ] == a r r [ j ] )
r e t u r n t r ue ;}
}r e t u r n f a l s e ;
}
13/17
O (n2) - Quadratic time
• Execution time of analgorithm∝ square ofthe size of the data n.
• Examples:– Bubble, Selection, and
Insertion sorts.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
boolean d u p l i c a t e s _ e x i s t ( i n t [ ] a r r ) {f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {
f o r ( i n t j = i + 1 ; j < a r r . l e n g t h ; j ++ ) {i f ( a r r [ i ] == a r r [ j ] )
r e t u r n t r ue ;}
}r e t u r n f a l s e ;
}
13/17
O (n2) - Quadratic time
• Execution time of analgorithm∝ square ofthe size of the data n.
• Examples:– Bubble, Selection, and
Insertion sorts.1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
boolean d u p l i c a t e s _ e x i s t ( i n t [ ] a r r ) {f o r ( i n t i = 0 ; i < a r r . l e n g t h ; i ++ ) {
f o r ( i n t j = i + 1 ; j < a r r . l e n g t h ; j ++ ) {i f ( a r r [ i ] == a r r [ j ] )
r e t u r n t r ue ;}
}r e t u r n f a l s e ;
}
13/17
O (n log n) - Loglinear Time• For every element in a
collection of size n,– log n operations are
performed.
• Examples:– Merge sort: log n levels with
linear work n for each level 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
[1,4,3,6,2,8,-1,7]
[2,8,-1,7]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,4,3,6]
[3,6]
[6][3]
[1,4]
[4][1]
Merge−→
[-1,1,2,3,4,6,7,8]
[-1,2,7,8]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,3,4,6]
[3,6]
[6][3]
[1,4]
[4][1]14/17
O (n log n) - Loglinear Time• For every element in a
collection of size n,– log n operations are
performed.
• Examples:– Merge sort: log n levels with
linear work n for each level 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
[1,4,3,6,2,8,-1,7]
[2,8,-1,7]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,4,3,6]
[3,6]
[6][3]
[1,4]
[4][1]
Merge−→
[-1,1,2,3,4,6,7,8]
[-1,2,7,8]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,3,4,6]
[3,6]
[6][3]
[1,4]
[4][1]14/17
O (n log n) - Loglinear Time• For every element in a
collection of size n,– log n operations are
performed.
• Examples:– Merge sort: log n levels with
linear work n for each level 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
[1,4,3,6,2,8,-1,7]
[2,8,-1,7]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,4,3,6]
[3,6]
[6][3]
[1,4]
[4][1]
Merge−→
[-1,1,2,3,4,6,7,8]
[-1,2,7,8]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,3,4,6]
[3,6]
[6][3]
[1,4]
[4][1]14/17
O (n log n) - Loglinear Time• For every element in a
collection of size n,– log n operations are
performed.
• Examples:– Merge sort: log n levels with
linear work n for each level 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
[1,4,3,6,2,8,-1,7]
[2,8,-1,7]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,4,3,6]
[3,6]
[6][3]
[1,4]
[4][1]
Merge−→
[-1,1,2,3,4,6,7,8]
[-1,2,7,8]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,3,4,6]
[3,6]
[6][3]
[1,4]
[4][1]14/17
O (n log n) - Loglinear Time• For every element in a
collection of size n,– log n operations are
performed.
• Examples:– Merge sort: log n levels with
linear work n for each level 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
[1,4,3,6,2,8,-1,7]
[2,8,-1,7]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,4,3,6]
[3,6]
[6][3]
[1,4]
[4][1]
Merge−→
[-1,1,2,3,4,6,7,8]
[-1,2,7,8]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,3,4,6]
[3,6]
[6][3]
[1,4]
[4][1]14/17
O (n log n) - Loglinear Time• For every element in a
collection of size n,– log n operations are
performed.
• Examples:– Merge sort: log n levels with
linear work n for each level 1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
[1,4,3,6,2,8,-1,7]
[2,8,-1,7]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,4,3,6]
[3,6]
[6][3]
[1,4]
[4][1]
Merge−→
[-1,1,2,3,4,6,7,8]
[-1,2,7,8]
[-1,7]
[7][-1]
[2,8]
[8][2]
[1,3,4,6]
[3,6]
[6][3]
[1,4]
[4][1]14/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (2n) - Exponential time
• describes an algorithmwhose growth doubleswith each addition to thedata set.
• Examples:– Brute-force algorithms– Fibonacci series:
0, 1, 1, 2, 3, 5,8, ..
p u b l i c s t a t i c i n t f i b o n a c c i ( i n t n ) {i f ( n == 0 || n == 1 ) {
r e t u r n n ; / / base ca se s} e l s e {
r e t u r n f i b o n a c c i ( n− 1) +f i b o n a c c i ( n−2) ; / / r e c u r s i v e
s tep}
}
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
fib(4)
fib(2)
fib(0)fib(1)
fib(3)
fib(1)fib(2)
fib(0)fib(1) 15/17
O (n!) - Factorial time
• Execution time of analgorithm∝ to the productof all numbers included ininput size (hence factorial!)
• Examples:– permutation problems
vo i d permuta t ion ( S t r i n g s t r ) {pe rmuta t ion ( " " , s t r ) ; }
vo i d permuta t ion ( S t r i n g p r e f i x , S t r i n gs t r ) {
i n t n = s t r . l e n g t h ( ) ;i f ( n == 0) System . out . p r i n t l n ( p r e f i x
) ;e l s e {
f o r ( i n t i = 0 ; i < n ; i ++ )permuta t ion ( p r e f i x + s t r .
cha rA t ( i ) , s t r . s u b s t r i n g(0 , i ) + s t r . s u b s t r i n g ( i+ 1 , n ) ) ; } }
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
"cat"
"t--"
"tca""tac"
"a--"
"atc""act"
"c--"
"cta""cat"
16/17
O (n!) - Factorial time
• Execution time of analgorithm∝ to the productof all numbers included ininput size (hence factorial!)
• Examples:– permutation problems
vo i d permuta t ion ( S t r i n g s t r ) {pe rmuta t ion ( " " , s t r ) ; }
vo i d permuta t ion ( S t r i n g p r e f i x , S t r i n gs t r ) {
i n t n = s t r . l e n g t h ( ) ;i f ( n == 0) System . out . p r i n t l n ( p r e f i x
) ;e l s e {
f o r ( i n t i = 0 ; i < n ; i ++ )permuta t ion ( p r e f i x + s t r .
cha rA t ( i ) , s t r . s u b s t r i n g(0 , i ) + s t r . s u b s t r i n g ( i+ 1 , n ) ) ; } }
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
"cat"
"t--"
"tca""tac"
"a--"
"atc""act"
"c--"
"cta""cat"
16/17
O (n!) - Factorial time
• Execution time of analgorithm∝ to the productof all numbers included ininput size (hence factorial!)
• Examples:– permutation problems
vo i d permuta t ion ( S t r i n g s t r ) {pe rmuta t ion ( " " , s t r ) ; }
vo i d permuta t ion ( S t r i n g p r e f i x , S t r i n gs t r ) {
i n t n = s t r . l e n g t h ( ) ;i f ( n == 0) System . out . p r i n t l n ( p r e f i x
) ;e l s e {
f o r ( i n t i = 0 ; i < n ; i ++ )permuta t ion ( p r e f i x + s t r .
cha rA t ( i ) , s t r . s u b s t r i n g(0 , i ) + s t r . s u b s t r i n g ( i+ 1 , n ) ) ; } }
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
"cat"
"t--"
"tca""tac"
"a--"
"atc""act"
"c--"
"cta""cat"
16/17
O (n!) - Factorial time
• Execution time of analgorithm∝ to the productof all numbers included ininput size (hence factorial!)
• Examples:– permutation problems
vo i d permuta t ion ( S t r i n g s t r ) {pe rmuta t ion ( " " , s t r ) ; }
vo i d permuta t ion ( S t r i n g p r e f i x , S t r i n gs t r ) {
i n t n = s t r . l e n g t h ( ) ;i f ( n == 0) System . out . p r i n t l n ( p r e f i x
) ;e l s e {
f o r ( i n t i = 0 ; i < n ; i ++ )permuta t ion ( p r e f i x + s t r .
cha rA t ( i ) , s t r . s u b s t r i n g(0 , i ) + s t r . s u b s t r i n g ( i+ 1 , n ) ) ; } }
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
"cat"
"t--"
"tca""tac"
"a--"
"atc""act"
"c--"
"cta""cat"
16/17
O (n!) - Factorial time
• Execution time of analgorithm∝ to the productof all numbers included ininput size (hence factorial!)
• Examples:– permutation problems
vo i d permuta t ion ( S t r i n g s t r ) {pe rmuta t ion ( " " , s t r ) ; }
vo i d permuta t ion ( S t r i n g p r e f i x , S t r i n gs t r ) {
i n t n = s t r . l e n g t h ( ) ;i f ( n == 0) System . out . p r i n t l n ( p r e f i x
) ;e l s e {
f o r ( i n t i = 0 ; i < n ; i ++ )permuta t ion ( p r e f i x + s t r .
cha rA t ( i ) , s t r . s u b s t r i n g(0 , i ) + s t r . s u b s t r i n g ( i+ 1 , n ) ) ; } }
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
"cat"
"t--"
"tca""tac"
"a--"
"atc""act"
"c--"
"cta""cat"
16/17
O (n!) - Factorial time
• Execution time of analgorithm∝ to the productof all numbers included ininput size (hence factorial!)
• Examples:– permutation problems
vo i d permuta t ion ( S t r i n g s t r ) {pe rmuta t ion ( " " , s t r ) ; }
vo i d permuta t ion ( S t r i n g p r e f i x , S t r i n gs t r ) {
i n t n = s t r . l e n g t h ( ) ;i f ( n == 0) System . out . p r i n t l n ( p r e f i x
) ;e l s e {
f o r ( i n t i = 0 ; i < n ; i ++ )permuta t ion ( p r e f i x + s t r .
cha rA t ( i ) , s t r . s u b s t r i n g(0 , i ) + s t r . s u b s t r i n g ( i+ 1 , n ) ) ; } }
1 2 3 4
Input size n
0
5
10
15
20
Opera
tions / T
ime
O(n) Notation Comparison
O(1)
O(log(n))
O(n)
O(nlog(n))
O(n2)
O(2n)
O(n!)
"cat"
"t--"
"tca""tac"
"a--"
"atc""act"
"c--"
"cta""cat"
16/17
Thank You!
1 2 3 4
Input size n
0
5
10
15
20
Op
era
tio
ns /
Tim
eO(n) Notation Comparison
O(yeah)
O(nice)
O(ok)
O(k-ish)
O(my)
O(no)
O(mg!)
Alternative Big O Notations
17/17
![Space-E cient String Indexing for Wildcard Pattern Matching · Ref. Space Usage Query Time [4] O(nlogn) words O(m+ σg loglogn+ occ) [2] O(n) words O(m+ σg loglogn+ occ) New O(nlogε](https://static.fdocuments.us/doc/165x107/5fd064be5f33bb2fb3561f03/space-e-cient-string-indexing-for-wildcard-pattern-matching-ref-space-usage-query.jpg)
