Greedy algo revision 2

Greedy Algorithims

It is animated pic

"Greed, for lack of a better word, is good" - Gordon Gecko, Wall Street (1987)

With a goal of reaching the largest-sum, at each step, the greedy algorithm will choose what appears to be the optimal immediate choice, so it will choose 12 instead of 3 at the second step, and will not reach the best solution, which contains 99.

Revision-2 (7 Dec, 2014)

2Revision-2 (7 Dec, 2014)

Currency Exchange Problem is Greedy

The currency algorithm is optimal in that it uses the fewest number of currency notes among all possible ways to make change for a given amount

4Revision-2 (7 Dec, 2014)

Example: To make change for the amount x = 49 (RUPEES).

Use E = 49/20 = 2. The remainder = 49 – 20E = 9

C= 9/5 = 1. Then the remainder = 9 – 5C = 4,

B = 4/2 = 2 . Finally, the remainder = 4 – 2B = 2,

B = 2/2 =0.

The total number of Currency used = E+E+C +B+ B = 5.

1=A2=B5=C10=D20=E

Revision-2 (7 Dec, 2014) 5

Here’s how the Greedy Algorithm can ruin your morning

If you’ve ever grown impatient waiting for a bus and instead started walking to your destination only to have a bus pass you 10 minutes late, you have fallen prey to the Greedy Algorithm. Sure, it feels good to start walking, but long-term, you’ll be both exhausted and late when you arrive at your destination.

In fact, sometimes it’s better to walk backwards, away from your final destination. That’s called retrograde progress. How can that be true? If the subway is two blocks behind you, and it’s rush hour in New York City.

Explanation of the previous slides

Greedy people pick short term gains over long term gains all the time. Which sometimes work , sometimes counter productive

In the last decades, similar problems called vehicle routing problems have become crucial for many companies, like Fedex and UPS. And these are so difficult, that companies asked help from applied mathematicians! Since then, applied mathematicians have enjoyed the beautifully combinatorial structure of these problems, as well as the millions of dollars companies have been willing to pay for answers!

The VRP (vehicle Routing Problem) is actually known to as hard as the TSP. This means that, when the number of cities isn’t small, there is virtually no hope to solve the VRP exactly in a reasonable amount of time, even with tomorrow’s greatest computer power. In fact, so far, no best solution is known for problems with more than a few hundreds of cities to visit!

To make Santa Claus’ problem a vehicle routing problem, we merely have to assume that Santa Claus’ reindeers cannot carry too heavy quantities. So, for instance, they can only carry presents for 5 cities. This will force Santa to come back to his toy factory several times to reload his sleigh. Typically, a Santa’s journey on the night of December 24th could look as follows:

Assumption to solve VRP

The most straightforward heuristics is the greedy algorithm.

What’s a greedy algorithm?It consists in choosing the next best step without much thinking, like a greedy Santa who is given a bunch of candies to choose from. He’d repeatedly eat his favorite one among the remaining candies, until he’s full… or until Mother Christmas tells him to stop!

It sounds like a great way to choose the best candies!

Note, this is not best solution, if you are interested for the best solution , visit http://www.science4all.org/le-nguyen-hoang/vehicle-routing-problem/

Knapsack Algorithem

The Greedy Method

Given n objects each have a weight wi and a value vi , and given a knapsack of total capacity W. The problem is to pack the knapsack with these objects in order to maximize the total value of those objects packed without exceeding the knapsack’s capacity. More formally, let xi denote the fraction of the object i to be included in the knapsack, 0 xi 1, for 1 i n. The problem is to find values for the xi such that

Note that we may assume because otherwise, we would choose xi = 1 for each i which would be an obvious optimal solution.

iii vxWwx

11maximized. is and

The Knapsack Problem:

(Max value/weight ratio) Sort the objects based on the value to weight ratios, from the highest to the lowest, then select. Example: Given n = 5 objects and a knapsack capacity W = 100 as in Table I..

Fractional knapsack problem

W 10 20 30 40 50 SUMV 20 30 66 40 60

V/W 2.0 1.5 2.2 1.0 1.2

x[i] 1 1 1 0 0.8 164

The Optimal Knapsack Algorithm:Input: an integer n, positive values wi and vi , for 1 i n, and another positive value W.

Output: n values xi such that 0 xi 1 and

Algorithm (of time complexity O(n lgn))(1) Sort the n objects from large to small based on the ratios vi/wi . We assume the arrays w[1..n] and v[1..n] store the respective weights and values after sorting.(2) initialize array x[1..n] to zeros.(3) weight = 0; i = 1(4) while (i n and weight < W) do

(4.1) if weight + w[i] W then x[i] = 1 (4.2) else x[i] = (W – weight) / w[i] (4.3) weight = weight + x[i] * w[i] (4.4) i++

iii vxWwx

11maximized. is and

100 KG50

KG60 PKR

40 PKR

66 PKR

30 PKR10

KG20 PKR

i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0 0 0 0 0

(4) while (i n and weight < W) do

(4.1) if weight + w[i] W then x[i] = 1

(4.2) else x[i] = (W – weight) / w[i]

(4.3) weight = weight + x[i] * w[i](4.4) i++

W=0 n=5before , weight=0i=1W[i]=?X[i]=?After, weight=?

i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0 0 0 0

W=0 n=5before , weight=0i=1W[i]=30X[i]=1After, weight=30

i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0 0 0

i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0, 1 0 0

i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0, 1 0, 0.8 0

W=100 n=5before , weight=60i=4W[i]=50X[i]=0.8After, weight=110

i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0, 1 0, 0.8 0

W=100 n=5before , weight=110Algorithm stops

(1) Sort the n objects from large to small based on the ratios vi/wi . We assume the arrays w[1..n] and v[1..n] store the respective weights and values after sorting.(2) initialize array x[1..n] to zeros. (3) weight = 0; i = 1 (4) while (i n and weight < W) do

(4.1) if weight + w[i] W then x[i] = 1 (4.2) else x[i] = (W – weight) / w[i] (4.3) weight = weight + x[i] * w[i] (4.4) i++

Algorithm (of time complexity O(n lgn))

If the items are already sorted into decreasing order of vi / wi, then the while-loop takes a time in O(n); Therefore, the total time including the sort is in O(n log n).If we keep the items in heap with largest vi/wi at the root. Then• creating the heap takes O(n) time• while-loop now takes O(log n) time (since heap property

must be restored after the removal of root)

Although this data structure does not alter the worst-case, it may be faster if only a small number of items are need to fill the knapsack.

Analysis of Fractional KnapSack

Huffman Code - Greedy

Simplest type of strategy:

1. Take a step that makes the problem smaller. 2. iterate.

Difficulty: Prove that this leads to an optimal solution.

This is not always the case!

Computer Data Encoding: How do we represent data in binary?

Historical Solution:Fixed length codes.Encode every symbol by a unique binary string of a

fixed length. Examples: ASCII (7 bit code), EBCDIC (8 bit code), …

Example (that works) –Huffman code

American Standard Code for Information Interchange

ASCII Example:

AABCAA

A A B C A A1000001 1000001 1000010 1000011 1000001 1000001

dead beef cafe deeded dad. dad faced a faded cab. dad acceded. dad be bad.

There are 12 a's, 4 b's, 5 c's, 19 d's, 12 e's, 4 f's, 17 spaces, and 4 periods, for a total of 77 characters.

Fix Lend Code• 77*3= 231

Variable Length Code=

Total space usage in bits:

Assume an ℓ bit fixed length code.

For a file of n characters

Need nℓ bits.

Desiderata:

Construct a variable length code for a given file with the following properties:

1. Prefix code.2. Using shortest possible codes.3. Efficient.4. As close to entropy as possible.

Consider a binary tree, with: 0 meaning a left turn 1 meaning a right turn.

Consider the paths from the root to each of the leaves A, B, C, D:

A : 0 B : 10 C : 110 D : 111

Observe:

1. This is a prefix code, since each of the leaves has a path ending in it, without continuation.

2. If the tree is full then we are not “wasting” bits.

3. If we make sure that the more frequent symbols are closer to the root then they will have a smaller code.

Revision-2 (7 Dec, 2014)43

Greedy Algorithm Example:

F E D C B A

60 50 40 30 20 10

Alphabet: A, B, C, D, E, F

Frequency table:

Total File Length: 210

http://www.cs.utexas.edu/users/djimenez/utsa/cs3343/lecture13.html

Huffman’s AlgorithmHuffman (C)

n = the size of Cinsert all the elements of C into Q,

using the value of the node as the priorityfor i in 1..n-1 do

z = a new tree nodex = Extract-Minimum (Q)y = Extract-Minimum (Q)left node of z = xright node of z = yf[z] = f[x] + f[y]Insert (Q, z)

end forreturn Extract-Minimum (Q) as the complete tree

Each priority queue operation (e.g. heap): O(log n)

In each iteration: one less subtree.

Initially: n subtrees.

Total: O(n log n) time.

Exercises

SOLVE BFS; DFS using Greedy WAY

Breadth-First Search (BFS):

BFS(v) // visit all nodes reachable from node v (1) create an empty FIFO queue Q, add node v to Q

(2) create a boolean array visited[1..n], initialize all values to false except for visited[v] to true(3) while Q is not empty (3.1) delete a node w from Q (3.2) for each node z adjacent from node w

if visited[z] is false then add node z to Q and set visited[z] to true

Node search order starting with node 1, including two nodes not reached

The time complexity is O(n+e) with n nodes and e edges, if the adjacency lists are used. This is because in the worst case, each node is added once to the queue (O(n) part), and each of its neighbors gets considered once (O(e) part).

Depth-First Search (DFS):

(1) create a boolean array visited[1..n], initialize all values to false except for visited[v] to true

(2) call DFS(v) to visit all nodes reachable via a path

DFS(v) for each neighboring nodes w of v do

if visited[w] is false then set visited[w] to true; call DFS(w) // recursive call

Node search order starting with node 1, including two nodes not reached

The algorithm’s time complexity is also O(n+e) using the same reasoning as in the BFS algorithm.

Minimum Spanning Tree

Definition: A tree is a connected undirected graph withno simple circuits.• Since a tree cannot have a simple circuit, a tree cannotcontain multiple edges or loops.• Therefore, any tree must be a simple graph.• Theorem: An undirected graph is a tree if and only ifthere is a unique simple path between any of itsvertices.

What is Tree

Yes Yes

A spanning tree of a graph is just a subgraph thatcontains all the vertices and is a tree.A graph may have many spanning trees

On a connected graph G=(V, E), aspanning tree:1. • is a connected subgraph2. • acyclic3. is a tree (|E| = |V| - 1)4. contains all vertices of G5. (spanning)

Properties of spanning Tree

Spanning Trees

Suppose you have a connected undirected graph– Connected: every node is reachable from every other node– Undirected: edges do not have an associated direction• ...then a spanning tree of the graph is a connected subgraph inwhich there are no cycles• Total of 16 different spanning trees for the graph below

I can’t find others, can you? Make them 16

The Minimum Spanning Tree for a given graph is the Spanning Tree of minimum cost for that graph.

This is minimum spanning tree

Kruskal’s Prims

Finding Minimum Spanning Tree

Algorithms :

REVISION-2 (7 DEC, 2014) 59

Edge first1. Arrange all edges in a list (L) in non-decreasing order2. Select edges from L, and include that in set T, avoid cycle.3. Repeat 3 until T becomes a tree that covers all vertices

161313

Kruskal’s algorithm

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

Skip, because it makes cycle

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

Skip, because it makes cycle

161313

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

MST is formed, Stops

Prim’s algorithm

1.Start form any arbitrary vertex

2.Find the edge that has minimum weight form all known vertices

3.Stop when the tree covers all vertices

161313

Question, All edges connected to 1, which is the shortest one? See next slide

161313

Question, All edges connected to 1, 3 , which is the shortest one? See next slide

Question, All edges connected to 1, 3 ,8 which is the shortest one? See next slide

Question, All edges connected to 1, 3 , 7, 8 which is the shortest one? See next slideNote: if two edges has same length, choose any one of them until there is no cycle

Question, All edges connected to 1, 3 ,7,8 and 5 which is the shortest one? See next slide

Stop, because you are done here

Prim’s algorithm

1.Start form any arbitrary vertex

2.Find the edge that has minimum weight form all known vertices

3.Stop when the tree covers all vertices

ACurrent:

Next Step:Consider all possible edges connected to Node A and choose the shortest edgeA= 3, 44

3 Previous Step

Next Step:Consider all possible edges connected to either Node A or Node B and choose the shortest edgeA= 44B= 20,54A= 20

3Previous Step

Next Step:Consider all possible edges connected to either Node A , B, or C and choose the shortest edgeA= 44B= 54C= 5,100C= 5

3Previous Step

Next Step:Consider all possible edges connected to either Node A, B, C, D and choose the shortest edgeA= 44 B= 54, C= 100, D= 8,14D=8

3Previous Step

Next Step:Consider all possible edges connected to either Node A , B, C, D, F and choose the shortest edgeA= 44, B= 54, C= 100, D= 14, F= 32,99D=14

3Previous Step

Next Step:Consider all possible edges connected to either Node A , B, C, D, E, F and choose the shortest edgeA= 44 , B= 54, C= 100, D= 54, F= 32,99, 100 , E= 11, 44E=11

3Previous Step

Next Step:Consider all possible edges connected to either Node A, B, C, D, E, F, H and choose the shortest edgeA= 44 , B= 54, C= 100, D= 54, E= 44, F= 32,99,100, H= 14,19, E= 44H=14

3Previous Step

Next Step:Consider all possible edges connected to either Node A,B, C, D, E, F, G, H and choose the shortest edgeA= 44 , B= 54, C=100, D=54, E= 44, 99, F= 32, 99, 100, H= 19, G= 31, 32, H=19

3Previous Step

Next Step: nothing, nodes are occupied

54Practice the same example But with different vertex selection, choose D in this case

Find the edge that has minimumweight form all known vertices

11 Find the edge that has minimumweight form all known vertices

Compare Prim and Kruskal Complexity

Kruskal: O(NlogN)comparison sort for edges

Prim: O(NlogN)search the least weight edge for every vertice

Revision-2 (7 Dec, 2014) 100

Why do we need MST?

• a reasonable way for clustering points in space into naturalgroups• can be used to give approximate solutions to hard problemsSuppose you want to provide solution to :– electric power, Water, telephone lines,network setup• To minimize cost, you could connect locations using MST• However, MST is not necessary the shortest path and it does not apply to cycle

Revision-2 (7 Dec, 2014) 101

MST don’t solve TSP

• Travel salesman problem (TSP) can notbe solved by MST salesman needs to go home (what’s the cost going home?)• TSP is a cycle ( Bad News)• Use MST to approximate• solve TSP by exhaustive approach- try every

permutation on cyclic graph

Greedy algo revision 2

Education

Transcript of Greedy algo revision 2

Finding Large Induced Subgraphs - Applied …kaul/talks/GraphKnapsack-Math.pdfIntroduction Resource Allocation Greedy Algo FPTAS Randomized Algo Largest Induced Subgraph Given a graph

Algo Ch1 Role Algo

… Algo 1 Algo 2 Algo 3 Algo N Meta-Learning Algo.

Automated Inferencedechter/publications/tutorial98.pdf · (tree-width ) Finding minimum w* is NP-complete (Arnb o rg, 1985). Greedy o rdering algo rithms: min-width rdering, min induced-width

An Iterated Greedy Metaheuristic for the Blocking Job Shop ...sezione/wordpress/wp-content/...Based on the AGH heuristics, in [23] a rollout algo-rithm is proposed for solving four

greedy algorithm

The Greedy Approach - cpp.edugsyoung/CS3310/CS3310Notes/Greedy.pdf · CS 331 D&A of Algo. Greedy 1 The Greedy Approach Young CS 331 D&A of Algo. Greedy 2 General idea: Given a problem

Greedy Algorithms:

Elements of Greedy Algorithms Greedy Choice Property for Kruskal…huding/331material/notes6.pdf · 2 Elements of Greedy Algorithms 3 Greedy Choice Property for Kruskal’s Algorithm

Adding One Punch Algo · OnePunch Algo Software, OnePunch Algo Plugin, OnePunch Algo Website, OnePunch Algo Articles or the channel videos. Title: Microsoft Word - Adding One Punch

The Greedy Approach - cpp.edugsyoung/CS530/Notes/CS530Greedy.pdf · CS 530 Adv. Algo. Greedy 1 The Greedy Approach Young CS 530 Adv. Algo. Greedy 2 General idea: Given a problem with

Introduction aux algorigrammes - sitelyceejdarc.orgsitelyceejdarc.org/autodoc/cours/001 1 STI2D/Technologie... · ACCUEIL ALGO COURS ALGO FAQ ALGO FORUM ALGO LIVRES ALGO SOURCES ALGO

Greedy method1

Greedy algorithms Greedy vs. divide and conquer

ANALIZA TRANSPORTNIH MREMREŽŽAA - nastava.sf.bg.ac.rs · Prožždrljivi (Greedy) heuristi drljivi (Greedy) heurističčki algoritmiki algoritmi Proždrljivi (Greedy) algoritmi generišu

Algoritmos Greedy

Greedy Algorithms - SJTUgao-xf/algorithm/Document/Slide04-GreedyAlgorithm.pdf · Greedy Analysis Strategies Greedy algorithm stays ahead. Show that after each step of the greedy algorithm,

Greedy Algoritham

Greedy Algo. for Selecting a Join Order The "greediness" is based on the idea that we want to keep the intermediate relations as small as possible at each.

[9] greedy