Binary Search - Kent State Universityaleitert/fall15/slides/01BinarySearch.pdfStrategy 3: Binary...

Binary Search

Introduction

Problem

Given a dictionary and a word. Which page (if any) contains the givenword?

3 / 26

Strategy 1: Random Search

Randomly select a page until the page containing the word is found.

I Very inefficient, no guarantee that the correct page is chosen.

I Does not detect if word is not in the dictionary.

4 / 26

Strategy 1: Random Search

Randomly select a page until the page containing the word is found.

I Very inefficient, no guarantee that the correct page is chosen.

I Does not detect if word is not in the dictionary.

4 / 26

Strategy 2: Linear Search

Check all pages sequentially starting at page 1.

I Acceptable efficiency, but ...

I strategy is not using all known properties.

5 / 26

Strategy 2: Linear Search

Check all pages sequentially starting at page 1.

I Acceptable efficiency, but ...

I strategy is not using all known properties.

5 / 26

Strategy 3: Binary Search

Question: What is the main property of words in a dictionary?

I Words are sorted.

I After looking on a page, we can say if word is on an earlier or laterpage.

6 / 26

I Words are sorted.

< < < < < < < < <

6 / 26

I Words are sorted.

6 / 26

I Words are sorted.

Select a page. If the word is smaller, look on an earlier pages. If the wordis larger, look on a later pages.

6 / 26

I Words are sorted.

Question: Which page should be selected?

I The middle!? Why?

6 / 26

I Words are sorted.

Question: Which page should be selected?

I The middle!? Why?

6 / 26

Algorithm

Input:

I A sorted array AI Some value k

18 20 22 34 36 52 54 57 64 74 52

7 / 26

Algorithm

1. Set l := 0 and r := A.Length− 1.

2. While l ≤ r

18 20 22 34 36 52 54 57 64 74 52

7 / 26

Algorithm

2.1. Set m :=⌊ l+r

18 20 22 34 36 52 54 57 64 74 52

7 / 26

Algorithm

2.2. If A[m] < k Then Set l := m+ 1.

18 20 22 34 36 52 54 57 64 74 52

7 / 26

Algorithm

2.3. If A[m] > k Then Set r := m− 1.

18 20 22 34 36 52 54 57 64 74 52

7 / 26

Algorithm

2.3. If A[m] = k Then Return m.

18 20 22 34 36 52 54 57 64 74 52

7 / 26

Algorithm – Binary Search

Input:

I A sorted array AI Some value k

1. Set l := 0 and r := A.Length− 1.

2. While l ≤ r2.1 Set m :=

⌊ l+r2

2.2 If A[m] < k Then Set l := m+ 1.2.3 If A[m] > k Then Set r := m− 1.2.4 If A[m] = k Then Return m.

3. Return −1.

8 / 26

Correctness and Invariant

Correctness

What means Binary Search is correct?

Theorem

The algorithm Binary Search returns the index of k in A if k ∈ A,otherwise it returns −1.

We have to show two cases

I k ∈ AI k /∈ A

We will show the second case first.

10 / 26

Correctness

What means Binary Search is correct?

Theorem

The algorithm Binary Search returns the index of k in A if k ∈ A,otherwise it returns −1.

We have to show two cases

I k ∈ AI k /∈ A

We will show the second case first.

10 / 26

Proof of Correctness – k /∈ A

Some notation:

I li and ri denote the value of l and r at the beginning of the i-th loopiteration, i. e., l0 = 0 and r0 = A.Length− 1.

I mi denotes the value of m assigned in the i-th loop iteration.

We have to show that we reach line 3.

I Because k /∈ A, we never run line 2.4.

I Because li ≤ mi ≤ ri, ri+1 − li+1 < ri − li (line 2.2 and line 2.3).

Thus, the algorithm reaches line 3 after a finite amount of iterations. �

11 / 26

Proof of Correctness – k /∈ A

Some notation:

I li and ri denote the value of l and r at the beginning of the i-th loopiteration, i. e., l0 = 0 and r0 = A.Length− 1.

I mi denotes the value of m assigned in the i-th loop iteration.

We have to show that we reach line 3.

I Because k /∈ A, we never run line 2.4.

I Because li ≤ mi ≤ ri, ri+1 − li+1 < ri − li (line 2.2 and line 2.3).

Thus, the algorithm reaches line 3 after a finite amount of iterations. �

11 / 26

Proof of Correctness – k ∈ A

How do we show that we find the right index?

I We show an invariant.

Invariant

The invariant of a loop is a property that is true every time before andafter every iteration of the loop.

Invariants often lead to the correctness of an algorithm.The most common way to prove them is induction.

12 / 26

Invariant

12 / 26

Invariant

12 / 26

Invariant

12 / 26

Invariant for Binary Search

I If k ∈ A, then li ≤ Ind(k) ≤ ri. (Ind(k) is the index of k in A)I ri+1 − li+1 < ri − li

Base Case

I Is true, because l0 = 0 and r0 = A.Length− 1.

Inductive Step

I By induction, li ≤ Ind(k) ≤ ri.I If k = A[mi], the algorithm returns mi. Done.

I If k < A[mi], them Ind(k) < mi.Thus, li+1 = li ≤ Ind(k) ≤ ri+1 = mi − 1 < mi ≤ ri.

I If k > A[mi], them Ind(k) > mi.Thus, li ≤ mi < mi + 1 = li+1 ≤ Ind(k) ≤ ri+1 = ri.

Because ri+1 − li+1 < ri − li, there is an iteration j withlj = Ind(k) = rj. �

13 / 26

Base Case

Inductive Step

13 / 26

Base Case

Inductive Step

13 / 26

Base Case

Inductive Step

13 / 26

Base Case

Inductive Step

13 / 26

Runtime

How fast is Binary Search?

I We count number of iterations.

I r0 − l0 = n− 1I ri+1 − li+1 ≤ (ri − li)/2

For which j is rj − lj ≤ 1?

rj − lj ≤12(rj−1 − lj−1)

≤ 14(rj−2 − lj−2)

≤ 2−j(r0 − l0) = 2−j(n− 1) ≤ 1

log2 n ≤ j

15 / 26

Runtime

I r0 − l0 = n− 1I ri+1 − li+1 ≤ (ri − li)/2

rj − lj ≤12(rj−1 − lj−1)

≤ 14(rj−2 − lj−2)

≤ 2−j(r0 − l0) = 2−j(n− 1) ≤ 1

log2 n ≤ j

15 / 26

Runtime

I r0 − l0 = n− 1I ri+1 − li+1 ≤ (ri − li)/2

rj − lj ≤12(rj−1 − lj−1)

≤ 14(rj−2 − lj−2)

≤ 2−j(r0 − l0) = 2−j(n− 1) ≤ 1

log2 n ≤ j

15 / 26

Runtime

I r0 − l0 = n− 1I ri+1 − li+1 ≤ (ri − li)/2

rj − lj ≤12(rj−1 − lj−1)

≤ 14(rj−2 − lj−2)

≤ 2−j(r0 − l0) = 2−j(n− 1) ≤ 1

log2 n ≤ j

15 / 26

Runtime

Theorem

The algorithm Binary Search requires at most dlog2(n+1)e iterations,where n = |A|.

16 / 26

Logarithmic Runtime

If |A| = 2n we need approx. n iterations.Also, 103 ≈ 210

Input size (n) Runtime (log2 n)

1,000 101,000,000 20

1,000,000,000 30atoms in the universe 266

17 / 26

Lower Time Bound

Is there a faster way than Binary Search?

I We allow a free preprocessing of A.

I We want to know if k is in A and where.

Theorem

In general, finding an element in an array cannot be done in less thanlogarithmic time.

18 / 26

Lower Time Bound

Is there a faster way than Binary Search?

I We allow a free preprocessing of A.

I We want to know if k is in A and where.

Theorem

In general, finding an element in an array cannot be done in less thanlogarithmic time.

18 / 26

Decision Tree

Model of computation: comparison model

I only operations allowed are comparisons (<,≤, >,≥,=, 6=)

I time cost is number of comparisons

Decision Tree

I algorithm is tree of comparisons

I internal nodes are comparisons

I leafs are outcomes of the algorithm

19 / 26

Decision Tree

Model of computation: comparison model

I only operations allowed are comparisons (<,≤, >,≥,=, 6=)

I time cost is number of comparisons

Decision Tree

I algorithm is tree of comparisons

I internal nodes are comparisons

I leafs are outcomes of the algorithm

19 / 26

Decision Tree Example

Binary Search for |A| = 3(For simplicity we ignore the case A[i] = k.)

A[1] < k?

A[0] < k? A[2] < k?

k ≤ A[0] A[0] < k ≤ A[1] A[1] < k ≤ A[2] A[2] < k

Yes Yes

20 / 26

Decision Tree Example

Linear Search for |A| = 3(For simplicity we ignore the case A[i] = k.)

A[0] < k?

A[1] < k?

A[2] < k?

k ≤ A[0]

A[0] < k ≤ A[1]

A[1] < k ≤ A[2] A[2] < k

21 / 26

Decision Tree vs. Algorithm + Proof

Decision Tree Algorithm

internal node binary decision

leaf found answer

root-to-leaf path algorithm execution

path length running time

height of tree worst case running time

Proof of theorem:

I Decision tree is binary

I Tree contains each possible answer as leaf, i. e., n+ 1 leafs

I Height ≥ log2(number of leafs). �

22 / 26

Decision Tree vs. Algorithm + Proof

Decision Tree Algorithm

internal node binary decision

leaf found answer

root-to-leaf path algorithm execution

path length running time

height of tree worst case running time

Proof of theorem:

I Decision tree is binary

I Tree contains each possible answer as leaf, i. e., n+ 1 leafs

I Height ≥ log2(number of leafs). �

22 / 26

Other Variants

Finding 01 in a Bit Array

Given a bit array A where A[0] = 0 and A[n− 1] = 1. Find an i such thatA[i] = 0 and A[i + 1] = 1.

0 0 1 1. . . . . .

24 / 26

Finding 01 in a Bit Array

Given a bit array A where A[0] = 0 and A[n− 1] = 1. Find an i such thatA[i] = 0 and A[i + 1] = 1.

Using binary search:

I If A[m] = 1, search left. If A[m] = 0, search right.

I Invariant: A[l] = 0 and A[r] = 1.

0 0 1 1. . . . . .

24 / 26

Analog Digital Converter

Convert an analog input (voltage) into a digital representation.

I Logic creates digital value (starting with most significant bit).

I Digital-to-analog converter (DAC) creates the analog equivalent Ud .

I Comparator takes both analog siganls an outputs if Ud > Uin.

I If Ud > Uin, logic resets bit.

25 / 26

Nodes in a Complete Binary Tree

Complete binary tree

I All layers (except lowest) are full.

I Lowest layer is filled from left to right.

Question: How many nodes are in the tree?

26 / 26

Naive solution

I Count all nodes.

I Runtime: n

26 / 26

Using binary search

I Determine height hl of left subtree (left side).

I Determine height hr of right subtree (left side).

I If hl = hr , lowest layer of left subtree is full (continue with rightsubtree). Otherwise, lowest layer of right subtree is empty (continuewith left subtree).

I Runtime: log2 n

26 / 26

Using binary search

I Determine height hl of left subtree (left side).

I Determine height hr of right subtree (left side).

I If hl = hr , lowest layer of left subtree is full (continue with rightsubtree). Otherwise, lowest layer of right subtree is empty (continuewith left subtree).

I Runtime: log2 n

26 / 26

Binary Search - Kent State Universityaleitert/fall15/slides/01BinarySearch.pdfStrategy 3: Binary...

Documents

Transcript of Binary Search - Kent State Universityaleitert/fall15/slides/01BinarySearch.pdfStrategy 3: Binary...