Design and Analysis of Algorithms - Chapter 51 Decrease and Conquer 1. Reduce problem instance to...

Design and Analysis of Algorithms - Chapter 5 1

Decrease and ConquerDecrease and Conquer

Reduce problem instance to smaller instance of the same Reduce problem instance to smaller instance of the same problem and extend solutionproblem and extend solution

Solve smaller instanceSolve smaller instance Extend solution of smaller instance to obtain solution to Extend solution of smaller instance to obtain solution to

original problemoriginal problem

Also referred to as Also referred to as inductiveinductive or or incremental incremental approach approach


Examples of Decrease and Examples of Decrease and ConquerConquer

Decrease by one:Decrease by one:• Insertion sortInsertion sort• Graph search algorithms:Graph search algorithms:

– DFSDFS– BFSBFS– Topological sortingTopological sorting

• Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets

Decrease by a constant factorDecrease by a constant factor• Binary search Binary search • Fake-coin problemsFake-coin problems• multiplication multiplication à la russeà la russe• Josephus problemJosephus problem

Variable-size decreaseVariable-size decrease• Euclid’s algorithmEuclid’s algorithm• Selection by partitionSelection by partition


What’s the difference?What’s the difference?

Consider the problem of exponentiation: Compute Consider the problem of exponentiation: Compute aann

Brute Force:Brute Force:

Divide and conquer:Divide and conquer:

Decrease by one:Decrease by one:

Decrease by constant factor:Decrease by constant factor:


Graph TraversalGraph Traversal

Many problems require processing all graph vertices in Many problems require processing all graph vertices in systematic fashionsystematic fashion

Graph traversal algorithms:Graph traversal algorithms:

• Depth-first searchDepth-first search

• Breadth-first searchBreadth-first search


Depth-first searchDepth-first search

Explore graph always moving away from last visited vertexExplore graph always moving away from last visited vertex Similar to preorder tree traversals Similar to preorder tree traversals

Pseudocode for Depth-first-search of graph G=(V,E)Pseudocode for Depth-first-search of graph G=(V,E)DFS(G)DFS(G)count :=0count :=0mark each vertex with 0 (unvisited)mark each vertex with 0 (unvisited)for each vertex vfor each vertex v∈ ∈ V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to vv dodo

if if ww is marked with 0 is marked with 0 dfs(dfs(ww))


Example – undirected graphExample – undirected graph

Depth-first traversal:Depth-first traversal:

a b

e f

c d

g h


Types of edgesTypes of edges

Tree edges:Tree edges: edges comprising forest edges comprising forest

Back edges:Back edges: edges to ancestor nodes edges to ancestor nodes

Forward edgesForward edges: edges to descendants (digraphs only) : edges to descendants (digraphs only)

Cross edges:Cross edges: none of the above none of the above


Example – directed graphExample – directed graph

Depth-first traversal:Depth-first traversal:

a b

e f

c d

g h


Depth-first search: NotesDepth-first search: Notes

DFS can be implemented with graphs represented as:DFS can be implemented with graphs represented as:• Adjacency matrices: Adjacency matrices: ΘΘ((VV22))

• Adjacency linked lists: Adjacency linked lists: ΘΘ((VV+E)+E)

Yields two distinct ordering of vertices:Yields two distinct ordering of vertices:• preorder: as vertices are first encountered (pushed onto stack)preorder: as vertices are first encountered (pushed onto stack)

• postorder: as vertices become dead-ends (popped off stack)postorder: as vertices become dead-ends (popped off stack)

Applications:Applications:• checking connectivity, finding connected componentschecking connectivity, finding connected components

• checking acyclicitychecking acyclicity

• searching state-space of problems for solution (AI)searching state-space of problems for solution (AI)


Breadth-first searchBreadth-first search

Explore graph moving across to all the neighbors of last Explore graph moving across to all the neighbors of last visited vertexvisited vertex

Similar to level-by-level tree traversals Similar to level-by-level tree traversals

Instead of a stack, breadth-first uses queueInstead of a stack, breadth-first uses queue

Applications: same as DFS, but can also find paths from a Applications: same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of vertex to all other vertices with the smallest number of edgesedges


Breadth-first search algorithmBreadth-first search algorithm

bfs(bfs(vv))

count := count + 1count := count + 1

mark mark vv with count with count

initialize queue with initialize queue with vv

while queue is not empty dowhile queue is not empty doaa := front of queue := front of queue

for each vertex for each vertex ww adjacent to adjacent to aa do do

if if ww is marked with 0 is marked with 0

count := count + 1count := count + 1

mark mark ww with count with count

add add ww to the end of the queue to the end of the queue

remove remove a a from the front of the queuefrom the front of the queue

BFS(G)BFS(G)count :=0count :=0mark each vertex with 0mark each vertex with 0for each vertex v V do∈for each vertex v V do∈

bfs(v)bfs(v)


Example – undirected graphExample – undirected graph

Breadth-first traversal:Breadth-first traversal:

a b

e f

c d

g h


Example – directed graphExample – directed graph

Breadth-first traversal:Breadth-first traversal:

a b

e f

c d

g h


Breadth-first search: NotesBreadth-first search: Notes

BFS has same efficiency as DFS and can be implemented BFS has same efficiency as DFS and can be implemented with graphs represented as:with graphs represented as:• Adjacency matrices: Adjacency matrices: ΘΘ((VV22))

• Adjacency linked lists: Adjacency linked lists: ΘΘ((VV+E)+E)

Yields single ordering of vertices (order added/deleted from Yields single ordering of vertices (order added/deleted from queue is the same)queue is the same)


Directed acyclic graph (dag)Directed acyclic graph (dag)

A directed graph with no cyclesA directed graph with no cycles

Arise in modeling many problems, eg:Arise in modeling many problems, eg:• prerequisite structureprerequisite structure

• food chainsfood chains

Imply partial ordering on the domainImply partial ordering on the domain


Topological sortingTopological sorting

Problem: find a total order consistent with a partial orderProblem: find a total order consistent with a partial order Example:Example:

fish

human

shrimp

sheep

wheatplankton

tiger

Order them so that they don’t have to wait for any

of their food(i.e., from lower to higher,

consistent with food chain)

NB: problem is solvable iff graph is dag


Topological sorting AlgorithmsTopological sorting Algorithms

1.1. DFS-based algorithm:DFS-based algorithm:1.1. DFS traversal noting order vertices are popped off stackDFS traversal noting order vertices are popped off stack

2.2. Reverse order solves topological sortingReverse order solves topological sorting

3.3. Back edges encountered?Back edges encountered?→→ NOT a dag! NOT a dag!

2.2. Source removal algorithmSource removal algorithm• Repeatedly identify and remove a Repeatedly identify and remove a sourcesource vertex, ie, a vertex that vertex, ie, a vertex that

has no incoming edgeshas no incoming edges

Both Both ΘΘ((VV+E) using +E) using adjacency linked listsadjacency linked lists


Variable-size-decrease: Binary Variable-size-decrease: Binary search treessearch trees

Arrange keys in a binary tree with the Arrange keys in a binary tree with the binary search tree binary search tree propertyproperty::

k

<k >k • What about repeated keys?What about repeated keys?

Example 1:Example 1: 5, 10, 3, 1, 7, 12, 9

Example 2:Example 2: 4, 5, 7, 2, 1, 3, 6


Searching and insertion in binary Searching and insertion in binary search treessearch trees

• Searching – straightforwardSearching – straightforward

• Insertion – search for key, insert at leaf where search terminatedInsertion – search for key, insert at leaf where search terminated

All operations: worst case # key comparisons = All operations: worst case # key comparisons = hh + 1 + 1 lg lg nn ≤ ≤ h h ≤ ≤ nn – 1 with average (random files) 1.41 lg – 1 with average (random files) 1.41 lg nn Thus all operations have:Thus all operations have:

• worst case: worst case: ΘΘ((nn) )

• average case: average case: ΘΘ(lg(lgnn) )

Bonus:Bonus: inorder traversal produces sorted list ( inorder traversal produces sorted list (treesorttreesort))

Design and Analysis of Algorithms - Chapter 51 Decrease and Conquer 1. Reduce problem instance to...

Documents

Transcript of Design and Analysis of Algorithms - Chapter 51 Decrease and Conquer 1. Reduce problem instance to...