6 Greedy

8/13/2019 6 Greedy

1/62

CSE 3101

Prof. Andy Mirzaian

8/13/2019 6 Greedy

2/62

STUDY MATERIAL:

[CLRS] chapter 16

Lecture Notes 7, 8

2

8/13/2019 6 Greedy

3/62

TOPICS

Combinatorial Optimization Problems

The Greedy Method

Problems:

Coin Change Making

Event Scheduling

Interval Point Cover

More Graph Optimization problems considered later

3

8/13/2019 6 Greedy

4/62

COMBINATORIAL

OPTIMIZATION

find the best solution out of finitely many possibilities.

4

8/13/2019 6 Greedy

5/62

Mr. Roboto:

Find the best path from A to B avoiding obstacles

A

B

A

B

There are infinitely many ways to get from A to B.We cant try them all.

But you only need to try finitely many critical paths

to find the best.

With brute-force there are exponentially many paths to try.

There may be a simple and fast (incremental) greedy strategy.

5

8/13/2019 6 Greedy

6/62

Mr. Roboto:

Find the best path from A to B avoiding obstacles

The Visibility Graph: 4n + 2 vertices

(n = # rectangular obstacles)

A

B

6

8/13/2019 6 Greedy

7/62

Combinatorial Optimization Problem (COP)

INPUT: Instance I to the COP.

Feasible Set: FEAS(I)= the set of all feasible (or valid) solutions for instance I,usually expressed by a set of constraints.

Objective Cost Function:Instance Iincludes a description of the objective cost function,

CostI that maps each solution S (feasible or not) to a real number or .Goal: Optimize(i.e., minimize or maximize) the objective cost function.

Optimum Set:OPT(I)= { Sol FEAS(I) | CostI(Sol) CostI(Sol), SolFEAS(I) }

the set of all minimum cost feasible solutions for instance I

Combinatorial means the problem structure implies that only a discrete & finitenumber of solutions need to be examined to find the optimum.

OUTPUT: A solution Sol OPT(I), or report that FEAS(I) = .

7

8/13/2019 6 Greedy

8/62

GREEDY METHOD

Greedy attitude:

Dont worry about the long term consequences of your current action.

Just grab what looks best right now.

8

8/13/2019 6 Greedy

9/62

For such problems it may be possible to build up a solution incrementallyby considering one input object at a time.

GREEDY METHODis one of the most obvious & simplest such strategies:Selects the next input object x that is the incremental bestoption at that time.Then makes a permanent decisionon x (without any backtracking),either committing to it to be in the final solution, or

permanently rejecting it from any further consideration.

C =

Committed objects

R =Rejected objects

U =

Undecided objects

S =

Set of

all inputobjects

Such a strategy may not work on every problem.

But if it does, it usually gives a very simple & efficient algorithm.

Most COPs restrict a feasible solution to be a finite set or sequence fromamong the objects described in the input instance.E.g., a subset of edges of the graph that forms a valid path from A to B.

9

8/13/2019 6 Greedy

10/62

Obstacle avoiding shortest A-B path

AB

d(p,q) = straight-line distance between points p and q.

u = current vertex of the Visibility Graph we are at (u is initially A).

If (u,B) is a visibility edge, then follow that straight edge. Done.Otherwise,from among the visibility edges (u,v) that get you closer to B, i.e., d(v,B) < d(u,B),make the following greedy choice for v:

(1) Minimize d(u,v) take shortest step while making progress

(2) Minimize d(v,B) get as close to B as possible(3) Minimize d(u,v) + d(v,B) stay nearly as straight as possible

Which of these greedy choices is guaranteed to produce the shortest A-B path?

u

v

B

greedy choice (1)greedy choice (2)greedy choice (3)shortest path

Later we will investigate a greedy strategy that works.10

8/13/2019 6 Greedy

11/62

Conflict freeobject subsets

S = the set of all input objects in instance I

FEAS(I)= the set of all feasible solutionsDefinition: A subset C S is conflict free if it is contained in some

feasible solution:

true if Sol FEAS(I): C Sol

ConflictFreeI(C) =

false otherwise

ConflictFreeI(C) = false

means C has some internal conflictand no matter what additional inputobjects you add to it, you cannot get a feasible solution that way.E.g., in any edge-subset of a simple path from A to B, every vertex hasin-degree at most 1. So a subset of edges, in which some vertex hasin-degree more than 1, has conflict.

11

8/13/2019 6 Greedy

12/62

The Greedy Loop Invariant

The following are equivalent statements of the generic greedy Loop Invariant:

The decisions we have made so far are safe, i.e.,they are consistent with some optimum solution (if there is any feasible solution).

Either there is no feasible solution, or

there is at least one optimum solutionSol OPT(I) thatincludesevery object we have committedto (set C), andexcludesevery object that we have rejected(set R = SUC).

S =

Set of

all input

objects

R

CU

Sol

LI: FEAS(I) = or [ SolOPT(I) : C Sol C U ].12

8/13/2019 6 Greedy

13/62

Pre-Cond: S is the set of objects in input instance I

Post-Cond: output COPT(I) or FEAS(I) =

U SC

LI: FEAS(I) = orSol OPT(I) :C Sol C U

returnC

Greedy Choice:

select x U with best

Cost(C{x}) possibleU U{x} permanently decide on x

if ConflictFree(C{x}) &

Cost(C{x}) better than Cost(C)

then C C {x} ----------- commit to xelse ------------------------------------ reject x

YES

NO

FEAS(I) = or C OPT(I)

U =

MP = |U|

LI & exit-cond

PostLoopCond

Pre-Cond &

PreLoopCode

LI

PostLoopCond

& PostLoopCode

Post-Cond

LI & exit-cond& LoopCodeLI

???

Greedy

method

13

8/13/2019 6 Greedy

14/62

LI & exit-cond& LoopCode LI

U & LI: FEAS(I) = or Sol OPT(I) : C Sol C UGreedy Choice:

select x U with maximum

Cost(C{x}) possibleU U{x} permanently decide on xif ConflictFree(C{x}) &

Cost(C{x}) > Cost(C)then C C {x} ----------- commit to x

else ------------------------------------ reject x

LI: FEAS(I) = or SolnewOPT(I) : C SolnewC U

Case 1:

x committed

Case 2:

x rejected

NOTE

Solnew

may ormay not

be the

same as

Sol

14

8/13/2019 6 Greedy

15/62






else ------------------------------------ reject x


Case 1a:

x committed

and

x Sol

OK. Take

Solnew =Sol

Solnew

may ormay not

be the

same as

Sol

15

8/13/2019 6 Greedy

16/62






else ------------------------------------ reject x


Case 1b:

x committed

andx Sol

Needs

Investigation

Solnew

may ormay not

be the

same as

Sol

16

8/13/2019 6 Greedy

17/62






else ------------------------------------ reject x


Case 2a:

x rejected

andx Sol

OK. Take

Solnew =Sol

Solnew

may ormay not

be the

same as

Sol

17

8/13/2019 6 Greedy

18/62






else ------------------------------------ reject x


Case 2b:

x rejected

andx Sol

Needs

Investigation

Solnew

may ormay not

be the

same as

Sol

18

8/13/2019 6 Greedy

19/62

SolOPT

R


Case 1b: x committed and x Sol

Show:SolnewOPTC

y

Trade off

x

Show y SolC such thatSolnew Sol {x}{y} satisfies:

(1) Solnew FEAS, &(2) Cost(Solnew) is no worse than Cost(Sol)

Some times more

than one object on

either side is traded

off.

19

8/13/2019 6 Greedy

20/62

SolOPT

R


Case 2b: x rejected and x Sol

Show:SolnewOPTC y

Trade off

x

Show thatSolnew Sol {y}{x}, for some y U - Solsatisfies:

(1) Solnew FEAS, &(2) Cost(Sol

new

) is no worse than Cost(Sol)

x could not havecreated a conflict. So, it

must have not improved

the objective function.

Some times more

than one object on

either side is traded

off.

20

8/13/2019 6 Greedy

21/62

COINCHANGE MAKING

Use minimum number of coins to make change for a given amount.

Greedy:pick the largest coin that fits first, then iterate.

21

8/13/2019 6 Greedy

22/62

The Coin Change Making Problem

Greedy Strategy (the obvious one):

Commit to the largest coin denomination that fits within the (remaining) amount,

then iterate.

Greedy(S) = the greedy solution for amount S.

Optimum(S) = the optimum solution for amount S.

PROBLEM:Within a coin denomination system, make change for a given amount S,

using the fewest number of coins possible.

Example 2: Imaginary coin denomination system: 7, 5, 1 kraaps.

S = 10 = 5 + 5 (Optimum) = 7 + 1 + 1 + 1 (Greedy).

Greedy(10) Optimum(10) in this system.

Example 1: Canadian coin denomination system: 25, 10, 5, 1 cents.

S = 98 (Two of many feasible solutions are shown below.)

= 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 5 + 1 + 1 + 1 13 coins used= 25 + 25 + 25 + 10 + 10 + 1 + 1 + 1 Optimum sol uses 8 coins.

Greedy(98) = Optimum(98) in the Canadian system.

22

8/13/2019 6 Greedy

23/62

The Problem Formulation

INPUT: Coin denomination system a = a1, a2, , an, a1> a2> > an =1,and S (all positive integers).

OUTPUT:The solution x = x1, x2, , xn, wherext = the number of coin type at used, for t = 1..n.

FEAS: a1x1+ a2x2 + +anxn = S, &xt is a non-negative integer, for t=1..n.

GOAL: Minimize objective cost x1+ x2+ + xn.

minimize x1+x2+ +xnsubject to:

(1) a1x1+ a2x2 + +anxn = S(2) xtN, for t = 1..n

objective

function

feasibility

constraints

We need an=1

to have a

feasible solnfor

every S.

23

8/13/2019 6 Greedy

24/62

Greedy Choice: choose the largest coin that fits

Conflict Free: 0 =1

Problem Objective Cost: =1

Greedy Objective Cost: + =1

At the end: 0 Greedy Objective = Problem objective

(is an unspecifiedprohibitively large

positive number)

The Greedy Choice & Objective

24

8/13/2019 6 Greedy

25/62

The Greedy Loop Invariant

Generic Greedy Loop Invariant (feasible solution) :SolOPT : C Sol C U.

Current greedy solution: x1,x2, ,xn Committed to: x1,x2, ,xt , for some t 1..n.Rejected:no more of a1,a2, , at-1.Considering:any more of at?

Not yet considered:xk= 0 for k > t.

There is U amount remaining to reach the target value S.

Loop Invariant:

x1,x2, ,xnFEAS(SU), (need U more to reach target S)

Sol = y1,y2, ,ynOPT(S):

yk = xk , for k < t, (Sol excludes what Greedy has rejected)

yt xt, (Sol includes what Greedy has committed to)

xk

= 0 , for k > t. (not yet considered)25

Algorithm: Greedy Coin Change

8/13/2019 6 Greedy

26/62

Pre-Cond: input is a = a1,a2, ,an, a1>a2> >an =1; and S (all pos integers)

Post-Cond: output x1,x2, ,xnOPT(S)

U S; t 1; x1,x2, ,xn0,0, ,0

LI: x1,x2, ,xnFEAS(SU),Sol = y1,y2, ,ynOPT(S):yk = xk for k < t, yt xt, xk=0 for k > t.

return x1, x2, ,xn

if U < at

then t t+1 reject: no more at

else xt

xt

+1 commit to another at

U U - at

YES

NOU = 0

Algorithm: Greedy Coin Change

LI & exit-cond

PostLoopCond

Pre-Cond &

PreLoopCode

LI

PostLoopCond

&

PostLoopCode

Post-Cond


???

x1,x2, ,xnOPT(S)

MP = U + (n-t) 26

8/13/2019 6 Greedy

27/62

Efficient implementation

minimize x1+x2+ +xnsubject to:

(1) a1x1+ a2x2 + +anxn = S(2) xtN, for t = 1..n

objective

function

feasibility

constraints

Algorithm Greedy(a1, a2, , an, S) takes O(n) timeCoinCount 0; U Sfor t 1 .. n do largest coin denomination first

xtU divat xtU/atU U modat U U atxtCoinCount CoinCount + xt objective value

end-for

return(x1, x2, , xn; CoinCount)end

27

8/13/2019 6 Greedy

28/62

Is G(S) = Opt(S) ?

A coin denomination system is called Regular if in that system G(S) = Opt(S) S.

Questions:(1) In the Canadian Coin System, is G(S) = Opt(S) for all S?

(2) In a given Coin System, is G(S) = Opt(S) for all S?

(3) In a given Coin System, is G(S) = Opt(S) for a given S?

Answers:(1) YES. Its Regular. We will see why shortly.

(2) YES/NO: Yes if the system is Regular. NO otherwise.There is a polynomial timeRegularity Testing algorithm(described next) to find the YES/NO answer.

(3) YES/NO: Regular system: always yes.Non-regular system: YES for some S, NO for other S.Exponential timeseems to be needed to find the Yes/No answer.(This is one of the so called NP-hard problems we will study later.)

28

8/13/2019 6 Greedy

29/62

How to Test Regularity

Question: Is a given Coin Denomination System Regular,

i.e., in that system is G(S) = Opt(S) for all S?

Greedy(a1, a2, , an, S) = (x = x1, x2, , xn ; G(S) = Stxt)Optimum(a1, a2, , an, S) = (y = y1, y2, , yn ; Opt(S) = Styt)

YES or NO: For the coin denomination system a1, a2, , an:

S: G(S) = Opt(S).

There are infinitelymany values of S to try.

However, if there is any counter-example S, there must be one among

polynomially manycriticalvaluesthat need to be tested todetermine the Yes/No answer. And, each test can be done fast.

What are these critical values? How are they tested?

29

8/13/2019 6 Greedy

30/62

Regularity& Critical Values

Consider the smallest multiple of each coin value that is not smaller than

the next larger coin value:

St= atmt, mt= at-1/ at, for t = 2..n.

Necessary Conditionfor correctness of Greedy:

G(St) mt, for t = 2..n. WHY?

This also turns out to be sufficient (under some mild pre-condition that is

satisfied by virtually any reasonable coin denomination system):

FACT 1: [Regularity Theorem of Magazine, Nemhauser, Trotter, 1975]

Pre-Condition: St< at-2, for t = 3..n S: G(S) = Opt(S) G(St) mt, for t = 2..n.

Proof: See Lecture Note 7.

30

8/13/2019 6 Greedy

31/62

Lets Test the Canadian System

t 1 2 3 4 5 6

coin at 200 100 25 10 5 1

mt =at-1 / at 2 4 3 2 5

St = atmt 200 100 30 10 5

Greedy: G(St) 1 1 2 1 1

Pre-Cond:St < at-2 yes yes yes yes

Test:G(St) mt yes yes yes yes yes

This table can be constructed and tested in O(n2) time.

FACT 1: [Magazine, Nemhauser, Trotter, 1975]

Pre-Condition: St< at-2, for t = 3..n

S: G(S) = Opt(S) G(St) mt, for t = 2..n.

31

8/13/2019 6 Greedy

32/62

Another System

t 1 2 3 4 5 6

coin at 200 100 25 11 5 1

mt =at-1 / at 2 4 3 3 5

St = atmt 200 100 33 15 5

Greedy: G(St) 1 1 5 5 1

Pre-Cond:St < at-2 yes yes yes yes

Test:G(St) mt yes yes NO NO yes

This table can be constructed and tested in O(n2) time.



S: G(S) = Opt(S) G(St) mt, for t = 2..n.

32

8/13/2019 6 Greedy

33/62

What if Pre-Condition doesnt hold?

FACT 2: [Pearson, 2005]

S: G(S) = Opt(S) O(n2) critical values test OK in O(n3) time.



S: G(S) = Opt(S) G(St) mt, for t = 2..n.[This can be tested in O(n2) time.]

See also Lecture Note 7.

33

8/13/2019 6 Greedy

34/62

The Optimum Sub-Structure Property

We just noticed an important property that will be used many times later:

The optimum sub-structure property:any sub-structure of an optimum structure is itself an

optimum structure (for the corresponding sub-instance). Problems with this property are usuallyamenable to more efficient algorithmic solutions

than brute-force or exhaustive search methods.

This property is usually shown by a cut-&-paste argument (see below).

Example 1: The Coin Change Making Problem.

Consider an optimum solution Sol OPT(S). Let G1 be a group of coins in Sol.Suppose G1 FEAS(U). Then we must have G1 OPT(U). Why?Because if G1 OPT(U), then we could cut G1 from Sol, and paste in the optimum sub-structure G2 OPT(U) instead. By doing so, we would get a new solution Sol FEAS(S)that has an even better objective value than Sol. But that would contradict Sol OPT(S).

Example 2: The Shortest Path Problem.Let P be a shortest path from vertex A to vertex B in the given graph G.

Let P be any (contiguous) sub-path of P. Suppose P goes from vertex C to D.Then P must be a shortest path from C to D. If it were not, then there must be an even

shorter path P that goes from C to D. But then, we could replace P portion of P byP and get an even shorter path than P that goes from A to B.

That would contradict the optimality of P.34

8/13/2019 6 Greedy

35/62

EVENTSCHEDULING

A banquet hall manager has received a list of reservation requests

for the exclusive use of her hall for specified time intervals

She wishes to grant the maximum number of reservation requeststhat have no time overlap conflicts

Help her select the maximum number of conflict free time intervals

35

8/13/2019 6 Greedy

36/62

The Problem Statement

INPUT: A set S = { I1, I2, , In} of n event time-intervals Ik= sk, fk, k =1..n,where sk= start time of Ik,

fk= finishing time of I

k,

( sk< fk).

OUTPUT: A maximum cardinality subset C S of mutually compatibleintervals(i.e., no overlapping pairs).

time

Example:S = the intervals shown below,C = the blue intervals, C is not the unique optimum.|C| =4. Can you spot another optimum solution?

36

S G

8/13/2019 6 Greedy

37/62

Some Greedy Heuristics

Greedy iteration step:From among undecided intervals, select the interval Ithat looks BEST.

Commit to I if its conflict-free(i.e., doesnt overlap with the committed ones so far).

Reject I otherwise.

Greedy 1: BEST= earliest start time (min sk).

Greedy 2: BEST= latest finishing time (max fk).

Greedy 3: BEST= shortest interval (min fksk).

Greedy 4: BEST= overlaps with fewest # of undecided intervals.

37

E li t Fi i hi Ti Fi t

8/13/2019 6 Greedy

38/62

Earliest Finishing Time First

S = the set of n given intervals

C = committed intervals

R = rejected intervals

U = undecided intervals

MaxF(X) = max{ fk | Ik X }MinF(X) = min{ fk | Ik X }Last = MaxF(C)

LOOP INVARIANT:Sol Opt(S) : C Sol CU,

MaxF(C) = Last MinF(U).

time

1 2 3 4 5 6 7 8 9

Last Last Last Last

C { IkS | fk Last }CR { IkS | fk < Last }

U { IkS | fk Last }

38

Algorithm: Greedy Event Schedule

8/13/2019 6 Greedy

39/62

Pre-Cond: input is S = {I1, I2, , In} of n intervals, Ik= sk, fk, sk< fk, k =1..n

Post-Cond: output C OPT(S)

U S; C ; Last

LI: Sol Opt(S):C Sol C U,

MaxF(C) = Last MinF(U)

return C

Greedy choice: selectIkU with min fk

U U{Ik } decide on Ik

ifskLast

then C C {Ik } commit to Ik

Last fk

else --------------------------- reject Ik

YES

NOU =

Algorithm: Greedy Event Schedule

LI & exit-cond

PostLoopCond

Pre-Cond &PreLoopCode

LI

PostLoopCond

&

PostLoopCode

Post-Cond


???

C OPT(S)

MP = |U|39

8/13/2019 6 Greedy

40/62


U &LI: Sol Opt(S): C Sol C U,MaxF(C) = Last MinF(U)

Case 1:

Ik rejected

Case 2:

Ik committed

Solnewmay ormay not

be the

same as

Sol

Greedy choice: selectIkU with min fk

U U{Ik } decide on IkifskLast

then C C {Ik } commit

Last fk

else --------------------------- reject

LI: SolnewOpt(S): C SolnewC U,MaxF(C) = Last MinF(U)

40

8/13/2019 6 Greedy

41/62


U U{Ik }.

Case 1: sk < LastIkrejectedSolnew= Sol maintains LI.

So, C Sol C (U{Ik }). Thus, Sol still satisfies 1stline of the LI.

Removing Ik from U cannot reduce MinF(U).

Also, C and Last dont change.

So, 2ndline of the LI is also maintained.


Last

C

fksk IkUsk< Last fk.Ikhas conflict with C,

hence with Sol.

So, IkSol.

41

I I

8/13/2019 6 Greedy

42/62

ftst It

Sol has no other interval here



U U{Ik }.Case 2: sk Last

Ikcommitted(C C {Ik }; Lastfk)

Solnew= ??? maintains LI.

Last

C

fksk

Ik

Ikis conflict-free with C.So, C{Ik} FEAS(S).So, | Sol| |C{Ik}| = |C|+1.So, SolC U.Choose ItSolC with min ft.

t = k or t k. ItSolC U. Greedy choice ftfk.New solution: Solnew(Sol{It}){Ik}.

Solnewis conflict-free & |Solnew| = |Sol| SolnewOPT(S).& (C {Ik })Solnew(C {Ik })(U{Ik }).

Therefore, Solnewmaintains 1stline of the LI.

2ndline of the LI is also maintained. 42

Effi i t i l t ti

8/13/2019 6 Greedy

43/62

Efficient implementation

Algorithm GreedyEventSchedule( S = I1, I2, , In) O(n log n) time

1. SORT S in ascending order of interval finishing times.WLOGA I1, I2, , Inis the sorted order, i.e., f1f2 fn.

2. C ; Last 3. for k 1 .. n do4. if Last sk then C C {Ik }5. Last fk6. end-for

7. return(C)

end

time

1 2 3 4 5 6 7 8 9

Last Last Last Last43

8/13/2019 6 Greedy

44/62

INTERVALPOINT COVER

We have a volunteer group to canvas homes & businesses along Yonge Street.

Each volunteer is willing to canvas a neighboring stretch of houses and shops.

Help us select a minimum number of these volunteersthat can collectively canvas every house and business along the street.

44


8/13/2019 6 Greedy

45/62


INPUT: A set P = { p1, p2, , pn} of n points, and

a set I = { I1= s1, f1, I2= s2, f2, , Im= sm, fm} of m intervals,all on the real line.

OUTPUT: Find out whether or not I collectively covers P.If yes, then report a minimum cardinality subset C I of (possibly overlapping)intervals that collectively cover P.If not, then report a point pP that is not covered by any interval in I.

Example 1:

Example 2:

Not covered by I

45


8/13/2019 6 Greedy

46/62


Greedy choice: pick the incrementally BESTundecided interval first.

Greedy 2: the interval that covers most uncovered points first.

Greedy 1: the longest interval first.

Greedy 3: Let pPbe an uncovered point thats covered by fewest intervals.If p is not covered by any interval, then report it.

Otherwise, pick an interval that covers p and max # other uncovered points.

46

Cover leftmost uncovered point first

8/13/2019 6 Greedy

47/62

Cover leftmost uncovered point first

C I (committed intervals), U P (uncovered points)

LOOP INVARIANT(exposedp is not covered by I) &

(I does not cover P or Sol Opt(I,P): C Sol) &MaxF(C) = Last

8/13/2019 6 Greedy

48/62

Algorithm: Opt Interval Point Cover

Algorithm OptIntervalPointCover ( P = { p1, p2, , pn}, I= {I1, I2, , Im} )

1. U P; C ; Last ; exposed false2. while U & not exposed do

3. p leftmost point in U greedy choice 14. Iset of intervals that cover p5. if I = then exposed true6. else do

7. Select IkI with max fk greedy choice 28. C C {Ik } ; Last fk9. U U {qP | q Last}10. end-else

11. end-while

12. if exposed thenreturn( p is not covered by I )

13. else return( C )

end

Loop Invariant: (exposed p is not covered by I) &(I does not cover P or Sol Opt(I,P): C Sol) &MaxF(C) = Last

8/13/2019 6 Greedy

49/62

Case 1. [exposed = true]:

exposedp is not covered by I. (1stline of LI)p is not covered by I.Case 2. [U = & exposed = false]:C covers P. (3rdline of LI)

C Sol C Opt(I,P). (2ndline of LI) C is an optimum cover.

PreLoopCode: U P; C ; Last ; exposed falseexit-cond: U =or exposed.LI: (exposedp is not covered by I) &

(I does not cover P or Sol Opt(I,P): C Sol) &MaxF(C) = Last

8/13/2019 6 Greedy

50/62


Case 1: I =

p is exposedexposed true

LI is maintained.

Suppose ItSolC covers p. So, ItI. We have t = k or t k.ftfk (by greedy choice 2).New solution: Solnew(Sol{It}) {Ik}.Solnewcovers every point of P covered by Sol , and |Solnew| = |Sol|.

Therefore, SolOPT(I,P) SolnewOPT(I,P).C {Ik }Solnew. Therefore, Solnewstill maintains 3rdline of the LI.

Remaining lines of LI are also maintained.

U & not exposed&LI: (exposedp is not covered by I) &

( I does not cover P or

Sol Opt(I,P): C Sol) &MaxF(C) = Last

8/13/2019 6 Greedy

51/62

Efficient Implementation

To carry out the greedy choices fast:

Line-Scan critical eventtimes t left-to-right on the real line.

Classify each interval Ik= sk, fkI:Inactive: t < sk ( t hasnt reached Ik)Active: skt fk ( t is covered by Ik)Dead: fk< t ( t has passed Ik)

Activated.

Classify event e:

e P point event (point activation time = p)e=(sk, Ik) interval activation event (interval activation time = sk)

ActInts= Max Priority Queue of activated (= active/dead) intervals Ik [priority = fk].

Events= Min Priority Queue of unprocessed events [priority = activation time].

Iterate: e DeleteMin(Events); process event e.

Minor trick: to avoid DeleteMax on empty ActInts, insert as first activated event a

dummy interval I0= -, -. I0will remain in ActInts till the end.

51

Algorithm: Efficient implementation

8/13/2019 6 Greedy

52/62

Algorithm: Efficient implementation

Algorithm OptIntervalPointCover ( P = { p1, p2, , pn}, I= {I1, I2, , Im} )

1. C ; Last ; I0,; I I {I0}2. Events ConstructMinHeap(PI) O(n+m) time3. while Events do O(n + m) iterations4. e DeleteMin(Events) next event to process, O(log(n+m)) time5. if e = (sk, Ik) then Insert(Ik , ActInts) activate interval

6. else event e is a point in P

7. if e > Last then do greedy choice 1: e=leftmost uncovered point8. IkDeleteMax(ActInts) greedy choice 2, O(log m) time9. if fk< e then return (point eP is not covered by I)10. else do

11. C C {Ik}12. Last fk13. end-else14. end-if

15. end-while

16. return( C )

endO( (n+m) log(n+m) ) time

52

Bibliography

8/13/2019 6 Greedy

53/62

Bibliography

If you want to dig deeper, roots of greedy algorithms are in the theory of matroids:

Hassler Whitney, On the abstract properties of linear dependence,American Journal ofMathematics, 57:509-533, 1935.

Jack Edmonds, Matroids and the greedy algorithm,Mathematical Programming, 1:126-136,

1971.

Eugene L. Lawler, Combinatorial Optimization: Networks and Matroids,Holt, Rinehart, and

Winston, 1976.

Christos Papadimitriou and Kenneth Steiglitz, Combinatorial Optimization: Algorithms and

Complexity,Prentice-Hall, 1982. [2ndedition, Courier Dover (publisher), 1998.]

The two cited references on the coin change making problem are:

M.J. Magazine, G.L. Nemhauser, L.E. Trotter, Jr., When the greedy solution solves a class of

knapsack problems,Operations Research, 23(2):207-217, 1975.

David Pearson A polynomial-time algorithm for the change-making problem,Operations

Research Letters, 33(3):231-234, 2005.

53

8/13/2019 6 Greedy

54/62

Exercises

54

1. The shortest obstacle avoiding path: As we discussed, the scene consists of a pair of

8/13/2019 6 Greedy

55/62

points A and B among n pairwise disjoint rectangular obstacles with horizontal andvertical sides. We listed 3 greedy heuristics. We saw that all three fail to find theshortest obstacle avoiding A-B path on some instances.An interesting question arises: How badly can these heuristics fail?(a) Explore to find the worst scene for each of these heuristics. By worse, we mean the

ratio of the length of the path found by the heuristic compared with the length ofthe shortest path, expressed as a function of n, is as high as possible. How badcould it be? Is it unbounded? Supper-linear? Linear? Sub-linear?

(b) Does the answer improve if the obstacles are congruent squares?

2. Coin Change making: For each of the following coin denomination systems

either argue that the greedy algorithm always yields an optimum solution forany given amount, or give a counter-example:(a) Coins c0, c1, c2, , cn-1, where c is an integer > 1.(b) Coins 1, 7, 13, 19, 61.(c) Coins 1, 7, 14, 20, 61.

3. [CLRS, Exercise 16.2-5, page 428] Smallest unit length interval covering set:

Describe an efficient algorithm that, given a set P = { p1, p2, , pn} of points on thereal line, determines the smallest set of unit-length closed intervals that covers all ofthe given points. Argue that your algorithm is correct.

4. Interval Point Cover: What is the time complexity of the Interval Point Coverproblem? We developed an O((n+m) log (n+m)) time algorithm for this problem.

Derive a lower bound on this problem by a reduction from the Min-Gap Problem. 55

5. [CLRS, Exercise 16.1-4, page 379] Minimum number of lecture halls:

8/13/2019 6 Greedy

56/62

5. [CLRS, Exercise 16.1 4, page 379] Minimum number of lecture halls:Suppose we have a set of events to schedule among a large number of lecture halls.We wish to schedule all the events using as few lecture halls as possible.Design and analyze an efficient greedy algorithm to determine which event should usewhich lecture hall. Prove the optimality of the solution produced by your algorithm.

[This is also known as the interval-graph colouring problem. We can create aninterval graph whose vertices are the given events and whose edges connectincompatible events. The smallest number of colours required to colour every vertexso that no two adjacent vertices are given the same colour corresponds to finding thefewest lecture halls needed to schedule all the given events.]

6. Smallest Hitting Set: Design and analyze an efficient greedy algorithm for thefollowing problem:Input: A set P = { p1, p2, , pn} of points, and a set I = { I1, I2, , Im} of

intervals, all on the real line.These intervals and points are given in noparticular order. Each interval is given by its starting and finishing times.

Output: (i) A minimum cardinality subset H of P such that every interval in I ishit by (i.e., contains) at least one point in H, or

(ii) an interval IkI that is not hit by any point in P.

7. Given a set of black and white intervals, select a smallest number of white intervalsthat collectively overlap every black interval. State your greedy choice and prove its

correctness. 56

8. One Machine Scheduling with Deadlines:

8/13/2019 6 Greedy

57/62

8. O e ac e Sc edu g w t ead es:You are given a set {J1, J2, , Jn} of n jobs to be processed on a single sequentialmachine. Associated with each job Jkis a processing time tkand a deadline dkby whichit must be completed. A feasible schedule is a permutation of the jobs such that if thejobs are processed in that order, then each job finishes by its deadline.

Design & analyze a simple greedy strategy that finds a feasible schedule if there is any.

9. Two Machine Scheduling:We are given a set {J1, J2, , Jn} of n jobs that need to be processed by two machines Aand B. These machines perform different operations and each can process only one jobat a time. Each job has to be processed by both machines; first by A, then by B.Job J

k

requires a given duration Ak

on machine A, and a given duration Bk

on B.

We wish to find the minimum total duration required to process all n jobs by bothmachines, as well as the corresponding optimum schedule.A schedule is the sequencing of the n jobs through the two machines.Both machines will process the jobs based on the scheduled order. Each job J, in thescheduled order, is first processed on machine A as soon as A completes its previously

scheduled job. Upon completion by A, job J is processed by B as soon as B becomesavailable.

Design and analyze an efficient greedy algorithm for this problem.Prove the optimality of the schedule produced by your algorithm.

57

10. The widely popular Spanish search engine El Googneeds to do a serious amount of

8/13/2019 6 Greedy

58/62

y p p p g gcomputation every time it recompiles its index. Fortunately, the company has at itsdisposal a single large supercomputer, together with an essentially unlimited supply ofhigh-end PCs.They have broken the overall computation into n distinct jobs, labeled J1, J2, , Jn,

which can be performed completely independently of one another. Each job consists oftwo stages: first it needs to be preprocessedon the supercomputer, and then it needs tobe finishedon one of the PCs. Let's say that job Jkneeds pkseconds of time on thesupercomputer, followed by fkseconds of time on a PC.

Since there are at least n PCs available on the premises, the finishing of the jobs can beperformed fully in parallel - all the jobs can be processed at the same time. However, the

supercomputer can only work on a single job at a time. So the system managers need towork out an order in which to feed the jobs to the supercomputer. As soon as the firstjob in order is done on the supercomputer, it can be handed off to a PC for finishing; atthat point in time a second job can be fed to the supercomputer; when the second job isdone on the supercomputer, it can proceed to a PC regardless of whether or not the firstjob is done (since PCs work independently in parallel), and so on.

Let's say that a scheduleis an ordering of the jobs for the supercomputer, and thecompletion timeof the schedule is the earliest time at which all jobs will have finishedprocessing on the PCs. This is an important quantity to minimize, since it determineshow rapidly El Goog can generate a new index.

Design and analyze an efficient greedy algorithm to find a minimum completion time

schedule. 58

11. The Factory Fueling Problem:A remotely-located factory has a fuel reservoir which,

8/13/2019 6 Greedy

59/62

y g y y ,when full, has enough fuel to keep the factory's machines running for M days. Due tothe factory's remote location, the fuel supply company does not deliver fuel on-demand.Instead, its trucks make visits to the factory's area according to a preset annual scheduleand if requested, would fill the reservoir. The schedule is given as an array S[1..n]

where S[i] is the date of the i-th visit in the year. Each time the reservoir is filled, afixed delivery charge is applied regardless of how much fuel is supplied. The factorymanagement would like to minimize these delivery charges and, at the same time,minimize the idle, empty reservoir periods. Given M and S, describe an efficientgreedy algorithm to obtain an optimal annual filling schedule; i.e., determine for eachof the n visits whether the reservoir should be filled. Prove that your algorithm satisfiesthe greedy-choice property.

12. The Loading Problem: Consider a train that travels from station 1 to station n withintermediate stops at stations 2, 3, ..., (n - 1). We have p[i,j] packages that need to bedelivered from point i to point j where 1 i < j n. Packages have the same size. Themaximum number at any point in the train must not exceed its capacity C. We want todeliver as many packages as possible.

a) Give a strategy for dropping and picking up packages at each point.b) Prove your strategy maximizes the number of delivered packages.[Hint: Use the greedy idea twice in the solution of this problem. At point 1 loadpackages in increasing order of their destination till capacity is reached. When the trainarrives at point 2, (conceptually) unload and reload according to the same principle asabove. If some packages that were picked up at point 1 get left at point 2, do not load

them at point 1 in the first place!] 59

13. Compatible Capacitated Assignment:

8/13/2019 6 Greedy

60/62

13. Compatible Capacitated Assignment:

Input: Two sorted arrays A[1..n] and B[1..n] of reals; a positive integer capacity C, and a

positive Compatibility real number a.Definitions:(i) A pair (i,j) is called compatible, if | A[i] - B[j] | a.

(ii) An assignmentis a pairing among elements of arrays A and B so that each element ofeach array appears in at least one pairing with an element of the other array.

(iii) An assignment is compatibleif each pairing in that assignment is compatible.(iv) A validassignment is a compatible one in which no element is paired with more than C

elements from the other array.Output: A valid assignment, or nil if no such assignment exists.

Design analyze and carefully prove the correctness of an efficient greedy algorithm for thisproblem. [Hint: Show the following claims are true:

(i) If (i,j1) and (i,j2)are compatible, then so is every pair (i,j) such that j is between j1and j2.Similarly, if (i1,j) and (i2,j) are compatible, then so is every pair (i,j) such that i is

between i1and i2.

(ii) If there is a valid assignment, then there is a valid assignment with no crossing, where acrossing consists of two assigned pairs (i1,j1) and (i2,j2) such that i1< i2but j1> j2, ori1> i2but j1< j2.

(iii) An uncrossed valid assignment has the property that if (i,j1) and (i,j2) are pairs in that

assignment, then so are all pairs of the form (i,j) with j between j1and j2. Similarly, if(i1,j) and (i2,j) are pairs in the assignment, then so are all pairs (i,j) such that i is

between i1and i2.(iv) We can assign pairs by scanning arrays A and B in a greedy merge fashion.] 60

14. Egyptian Fractions:

8/13/2019 6 Greedy

61/62

gyp

Input: A rational fraction x/y, where x and y are integers and 0

8/13/2019 6 Greedy

62/62

END

6 Greedy

Documents

Transcript of 6 Greedy