[13] b-and-bb

Proiectarea algoritmilor: Backtracking,Branch-and-Bound

Dorel Lucanu

Faculty of Computer ScienceAlexandru Ioan Cuza University, Iasi, Romania

[email protected]

PA 2014/2015

D. Lucanu (FII - UAIC) Backtracking, Branch-and-bound PA 2014/2015 1 / 23

Outline

1 Probleme tractabile/netractabile

2 Backtracking

3 Branch-and-Bound


Probleme tractabile/netractabile

Plan


2 Backtracking

3 Branch-and-Bound



Probleme tractabile

Teza lui Cobham, cunoscuta si sub numele de teza lui Cobham-Edmonds(dupa Alan Cobham si Jack Edmonds), afirma ca problemele care pot firezolvate practic (feasibly computed) sunt cele care pot fi rezolvate ntimp polinomial.

Cu toate acestea, un algoritm cu timpul polinomial nu este totdeauna unulpractic. De exemplu, daca timpul de executie este n20, atunci nu esterezonabil sa spunem ca este eficient, cu exceptia doar a instantelor dedimensiuni mici.



Probleme netractabile

Probleme care sunt cunoscute ca fiind netractabile sunt cele EXP-dificile.

Daca P 6= NP, atunci problemele NP-complete sunt de asemeneanetractabile n acest sens.

Ca si tractabilitatea, este discutabil ce este netractabilitatea n practica.

De exemplu, problema deciziei n aritmetica Presburger nu este n P, dars-au proiectat algorimi eficienti ce rezolva problema n timp rezonabil ncele mai multe cazuri.

Problema rucsacului poate fi rezolvata n timp patratic n foarte multesituatii.

Un alt exemplu este SAT. Asa numite SAT-solvere, care stau la bazamultor demonstratoare automate, sunt capabile sa rezolve satisfiabilitateaunor formule boleene de dimensiuni mari.



Cum procedam cu problemele NP-complete

1 Renunta la algoritmii polinomiali si spera ca algoritmul proiectatrezolva n timp rezonabil problema pentru instantele ntalnite npractica (backtracking, branch-and-bound - acest curs).

2 Renunta la optimalitate si ncearca una din urmatoarele posibilitati:1 algoritmi de aproximare (cursul urmator)2 euristici

1 cautare locala (cursul de AG)2 cautare tabu (cursul de AG)3 hill climbing (cursul de IA)4 simulated anealing (cursul de IA)

3 algoritmi genetici (optional anul II)


Backtracking

Plan


2 Backtracking

3 Branch-and-Bound


Backtracking

Modelul: prezentare generala

Ia avantajul structurii inerente pe care le au multe problemeNP-complete.De multe ori certificatul pentru o prblema NP-completa consta ntr-omultime de alegeri.

Iar verificarea certificarii consta ntr-un test simplu care demonstreazaca o configuratie este una de succes pentru instanta x .

Backtracking propune o cautare inteligenta n spatiul configuratiilor.


Backtracking

Modelul: schema algoritmica

Sursa: Algorithm Design Foundations, Analysis, and Internet Examples by M. T. Goodrich and R. Tamassia

628 Chapter 13. NP-Completeness

Algorithm Backtrack(x):Input: A problem instance x for a hard problemOutput: A solution for x or no solution if none existsF {(x,)}. {F is the frontier set of subproblem configurations}while F = doselect from F the most promising configuration (x,y)expand (x,y) by making a small set of additional choiceslet (x1,y1), (x2,y2), . . ., (xk,yk) be the set of new configurations.for each new configuration (xi,yi) doperform a simple consistency check on (xi,yi)if the check returns solution found thenreturn the solution derived from (xi,yi)

if the check returns dead end thendiscard the configuration (xi,yi) {Backtrack}

elseF F {(xi,yi)} {(xi,yi) starts a promising search path}

return no solution

Algorithm 13.18: The template for a backtracking algorithm.

Filling in the Details

In order to turn the backtracking strategy into an actual algorithm, we need only fillin the following details:

1. Define a way of selecting the most promising candidate configuration fromthe frontier set F .

2. Specify the way of expanding a configuration (x,y) into subproblem config-urations. This expansion process should, in principle, be able to generate allfeasible configurations, starting from the initial configuration, (x,).

3. Describe how to perform a simple consistency check for a configuration (x,y)that returns solution found, dead end, or continue.

If F is a stack, then we get a depth-first search of the configuration space. Infact, in this case we could even use recursion to implement F automatically asa stack. Alternatively, if F is a queue, then we get a breadth-first search of theconfiguration space. We can also imagine other data structures to implement F ,but as long as we have an intuitive notion of how to select the most promisingconfiguration from F with each iteration, then we have a backtracking algorithm.

So as to make this approach more concrete, let us work through an applicationof the backtracking technique to the CNF-SAT problem.


Backtracking

Reamintire: CSAT

CSATInstance O formula propozitionala F cu variabile din X = {x1, . . . , xn}

si n forma normala conjunctiva.Question Exista o atribuire a : X {0, 1} (sau a : X {false, true})

care satisface F?

Exemplu de instanta:

X = {x , y , z}F = (x y z) (x z) (x y z x)


Backtracking

Backtracking pentru CSAT

Data o frontiera F , cea mai promitatoare alegere este o subformula cu ceamai mica clauza. Aceasta este subformula cu cele mai multe constrangerisi de aceea este de asteptat de a ajunge la un dead end cat mai repedeposibil, atunci cand formula este nesatisfiabila.

Generarea subproblemelor dintr-o subformula F se face prin localizareaunei celei mai mici clazue C n F si alegand o variabila xi care apare n C .Vor fi create doua subprobleme: una pentru xi = 0 si una pentru xi = 1.


Backtracking

Backtracking pentru CSAT

Verificarea consistentei pentru o atribuire de valori pentru o variabila xidin F :

1 se reduce orice clauza ce include xi ;2 daca n procesul de reducere se obtine o clauza cu un singur literal, xj

sau xj , se atribuie valoarea potrivita pentru xj care face aceastaclauza satisfiabila;

3 procesul continua pana nu se mai sunt clauze constand ntr-un singurlietral;

4 daca la un moment dat se ajunge la o contradictie (clauza vida, sauclauzele xi s xi ), atunci va ntoarce dead end;

5 daca subformula F este redusa la 1, atunci n toarce solution foundmpreuna cu atribuirile variabilelor gasite pana atunci;

6 altfel, se deriveaza o noua subformula F astfel ncat orice clauza arecel putin doi literali mpreuna cu atribuirile variabilelor care conduc dela F la F . Acest pas se numeste propagarea atribuirii pentru xi F .


Backtracking

Exemplu

Sursa: Algorithms by Dasgupta, C. H. Papadimitriou, and U. V. Vazirani.Figure 9.1 Backtracking reveals that is not satisfiable.

(), (y z)(y z), (y), (y z)

(z), (z)

(x y), (y z), (z), (z)

(x y), (y), ()(x y), ()

(w x y z), (w x), (x y), (y z), (z w), (w z)

(x y z), (x), (x y), (y z)

x = 1

()

z = 0 z = 1

()

()

y = 1

z = 1z = 0

y = 0

w = 1w = 0

x = 0

In the case of SAT, this test declares failure if there is an empty clause, success if there areno clauses, and uncertainty otherwise. The backtracking procedure then has the followingformat.

Start with some problem P0Let S = {P0}, the set of active subproblemsRepeat while S is nonempty:choose a subproblem P S and remove it from Sexpand it into smaller subproblems P1, P2, . . . , PkFor each Pi:If test(Pi) succeeds: halt and announce this solutionIf test(Pi) fails: discard PiOtherwise: add Pi to S

Announce that there is no solution

For SAT, the choose procedure picks a clause, and expand picks a variable within that clause.We have already discussed some reasonable ways of making such choices.

With the right test, expand, and choose routines, backtracking can be remarkably effec-tive in practice. The backtracking algorithm we showed for SAT is the basis of many successfulsatisfiability programs. Another sign of quality is this: if presented with a 2SAT instance, itwill always find a satisfying assignment, if one exists, in polynomial time (Exercise 9.1)!

9.1.2 Branch-and-boundThe same principle can be generalized from search problems such as SAT to optimizationproblems. For concreteness, lets say we have a minimization problem; maximization will

272


Branch-and-Bound

Plan


2 Backtracking

3 Branch-and-Bound


Branch-and-Bound

Modelul: prezentare generala

Algoritmii proiectati cu paradigma backtracking sunt efectivi doarpentru probleme de decizie.

Pentru problemele de optimizare, pe langa conditia ce trebuiesatisfacuta pentru certificare mai exista o functie de minimizat sau demaximizat.

Branch-and-bound extinde schema algoritmica backtrackingpentru a putea lucra cu probleme de optimizare.


Branch-and-Bound

Modelul: schema algoritmicaSursa: Algorithm Design Foundations, Analysis, and Internet Examples by M. T. Goodrich and R. Tamassia

632 Chapter 13. NP-Completeness

13.5.2 Branch-and-Bound

The backtracking algorithm is effective for decision problems, but it is not designedfor optimization problems, where, in addition to having some feasibility conditionbe satisfied for a certificate y associated with an instance x, we also have a costfunction f (x) that we wish to minimize or maximize (without loss of generality, letus assume the cost function should be minimized). Nevertheless, we can extend thebacktracking algorithm to work for such optimization problems, and in so doingderive the algorithmic design pattern known as branch-and-bound.

The branch-and-bound design pattern, given in Algorithm 13.20, has all the el-ements of backtracking, except that rather than simply stopping the entire searchprocess any time a solution is found, we continue processing until the best solutionis found. In addition, the algorithm has a scoring mechanism to always choosethe most promising configuration to explore in each iteration. Because of this ap-proach, branch-and-bound is sometimes called a best-first search strategy.

Algorithm Branch-and-Bound(x):Input: A problem instance x for a hard optimization (minimization) problemOutput: An optimal solution for x or no solution if none existsF {(x,)} {Frontier set of subproblem configurations}b (+,) {Cost and configuration of current best solution}while F = doselect from F the most promising configuration (x,y)expand (x,y), yielding new configurations (x1,y1), . . ., (xk,yk)for each new configuration (xi,yi) doperform a simple consistency check on (xi,yi)if the check returns solution found thenif the cost c of the solution for (xi,yi) beats b thenb (c,(xi,yi))

elsediscard the configuration (xi,yi)

if the check returns dead end thendiscard the configuration (xi,yi) {Backtrack}

elseif lb(xi,yi) is less than the cost of b thenF F {(xi,yi)} {(xi,yi) starts a promising search path}

elsediscard the configuration (xi,yi) {A bound prune}

return b

Algorithm 13.20: The template for a branch-and-bound algorithm. This algorithmassumes the existence of a function, lb(xi,yi), that returns a lower bound on thecost of any solution derived from the configuration (xi,yi).


Branch-and-Bound

Reamintire: KNAPSACK 0/1

KNAPSACK 0/1Instance wi Z+, pi Z, i = 1, . . . , n

M Z+K Z+ (scop)

Question Exista xi {0, 1}, i = 1, . . . , n cu proprietatea ca scopul esteatins si obiectele alese ncap n rucsac:n

i=1xi pi K

ni=1

xi wi M?


Branch-and-Bound

Branch-and-Bound pentru KNAPSACK

Obiectele sunt ordonate descrescator dupa piwi (ca la algoritmul greedypentru varianta continua).

O configuratie Sj este data de starea (de la programarea dinamica)Rucsac(j,X) (s-au facut alegeri dintre primele j obiecte). S0 esteconfiguratia vida si corespunde lui j = 0.

Se considera o coada de prioritate Q, care initial va include S0.

La iteratia curenta se selecteaza cea mai promitatoare configuratie cdin Q. Vom vedea mai tarziu cum se calculeaza.


Branch-and-Bound


Daca j este ultimul indice considerat pentru c , atunci expandam c cudoua noi configuratii: una care include obiectul j + 1 si cealalta carenu include obiectul j + 1.

O configuratie este fezabila (verifica criteriul simplu de consistenta)daca satisface constrangerea de dimensiune.

Daca una dintre cele doua noi configuratii este mai buna decat ceamai buna gasita pana n acel moment, se actualizeaza cea mai bunasolutie.


Branch-and-Bound


Pentru a putea calcula cea mai promitatoare configuratie, este nevoiede a asocia un scor configuratiilor, care sa reprezinte valoareapotentiala.

Vom considera o margine superioara a profitului calculat pentruversiunea relaxata a problemei, n care se pot lua parti fractionaredintre obiectele neconsiderate nca j + 1, j + 2, . . .. Aceasta marginesuperioara (upper bound) este calculata de algoritmul greedy:Notatii:

pc - profitul total calculat pentru configuratia c ;wc - dimnesiunea toatala calculata pentru configurtia c ;k cel mai mare indice cu proprietatea

ki=j+1 wj M wc ;


Branch-and-Bound


Marginea superioara (upper bound) este data de:

ub(c) = pc +k

i=j+1

pi + (M wc k

i=j+1

wi ) pk+1wk+1

Daca k = n, atuncipk+1wk+1

se considera zero.

Daca algoritmul greedy ntoarce solutie ntreaga, atunci aceasta esteoptima pentru configuratia curenta (nu mai trebuie expandat).


Branch-and-Bound

Exemplu numericex-5.1-5.3 Foundations of Operations Research Prof. E. Amaldi

5.2 Branch-and-Bound for 0-1 knapsack

The integer linear programming formulation for the problem is

max 16x1 + 22x2 + 12x3 + 8x4

5x1 + 7x2 + 4x3 + 3x4 14x1, x2, x3, x4 {0, 1}

and its linear programming relaxation is

max 16x1 + 22x2 + 12x3 + 8x4

5x1 + 7x2 + 4x3 + 3x4 140 xi 1 i {1, 2, 3, 4} .

To find an optimal solution to the linear programming relaxation of the knapsack problem,there is no need to use the two-phase Simplex method. We can use the following simple greedyalgorithm. First, sort the variables by nonincreasing revenue and cost ratio. Note that in ourcase

(16/5, 22/7, 12/4, 8/3) = (3.2, 3.14, 3, 2.66)

and the variables are already ordered appropriately. Then, consider the variables in that orderand assign the largest possible value to the variable under consideration xi as long as

i

Branch-and-Bound

Exemplu numeric

x = (1, 1, 12, 0)

b = ub = 44

x = (1, 1, 12, 0)

b = 16ub = 44

x = (1, 1, 12, 0)

b = 38ub = 44

x = (1, 1, 0, 23

)b = 38ub = 43 1

3

x = (1, 1, 0, 0)b = 38ub = 38

x = (1, 0, 1, 1)b = 38ub = 36

x = (0, 1, 1, 1)b = 42ub = 42

t0

t1

t2

t3

t4

t5

t6

x1 = 1

x2 = 1

x3 = 0

x4 = 0

x2 = 0

x1 = 0


Probleme tractabile/netractabileBacktrackingBranch-and-Bound

[13] b-and-bb

Documents

Transcript of [13] b-and-bb