16.410/413PrinciplesofAutonomyandDecisionMakingLecture14: InformedSearchEmilioFrazzoliAeronauticsandAstronauticsMassachusettsInstituteofTechnologyNovember1,2010E.Frazzoli (MIT) L05: InformedSearch November1,2010 1/46Outline1Informedsearchmethods: IntroductionShortestPathProblemsonGraphsUniform-costsearchGreedy(Best-First)Search2Optimalsearch3DynamicProgrammingE.Frazzoli (MIT) L05: InformedSearch November1,2010 2/46AstepbackWehaveseenhowwecandiscretizecollision-freetrajectoriesintoanitegraph.Searchingforacollision-freepathcanbeconvertedintoagraphsearch.Hence,wecansolvesuchproblemsusingthegraphsearchalgorithmsdiscussedinLectures2and3(Breadth-FirstSearch,Depth-FirstSearch,etc.).However,roadmapsarenotjustgenericgraphs.Somepathsaremuchmorepreferablewithrespecttoothers(e.g.,shorter,faster,lesscostlyintermsoffuel/tolls/fees,morestealthy,etc.).Distanceshaveaphysicalmeaning.Goodguessesfordistancescanbemade,evenwithoutknowingoptimalpaths.Canweutilizethisinformationtondecientpaths,eciently?E.Frazzoli (MIT) L05: InformedSearch November1,2010 3/46AstepbackWehaveseenhowwecandiscretizecollision-freetrajectoriesintoanitegraph.Searchingforacollision-freepathcanbeconvertedintoagraphsearch.Hence,wecansolvesuchproblemsusingthegraphsearchalgorithmsdiscussedinLectures2and3(Breadth-FirstSearch,Depth-FirstSearch,etc.).However,roadmapsarenotjustgenericgraphs.Somepathsaremuchmorepreferablewithrespecttoothers(e.g.,shorter,faster,lesscostlyintermsoffuel/tolls/fees,morestealthy,etc.).Distanceshaveaphysicalmeaning.Goodguessesfordistancescanbemade,evenwithoutknowingoptimalpaths.Canweutilizethisinformationtondecientpaths,eciently?E.Frazzoli (MIT) L05: InformedSearch November1,2010 3/46AstepbackWehaveseenhowwecandiscretizecollision-freetrajectoriesintoanitegraph.Searchingforacollision-freepathcanbeconvertedintoagraphsearch.Hence,wecansolvesuchproblemsusingthegraphsearchalgorithmsdiscussedinLectures2and3(Breadth-FirstSearch,Depth-FirstSearch,etc.).However,roadmapsarenotjustgenericgraphs.Somepathsaremuchmorepreferablewithrespecttoothers(e.g.,shorter,faster,lesscostlyintermsoffuel/tolls/fees,morestealthy,etc.).Distanceshaveaphysicalmeaning.Goodguessesfordistancescanbemade,evenwithoutknowingoptimalpaths.Canweutilizethisinformationtondecientpaths,eciently?E.Frazzoli (MIT) L05: InformedSearch November1,2010 3/46ShortestPathProblemsonGraphsInput: V, E, w, start, goal:V: (nite)setofvertices.E V V: (nite)setofedges.w: E R>0, e w(e): afunctionthatassociatestoeachedgeastrictlypositiveweight(cost,length,time,fuel,prob. ofdetection).start, goal V: respectively,startandendvertices.Output: PPisapath(startinginstartandendingingoal,suchthatitsweightw(P)isminimalamongallsuchpaths.Theweightofapathisthesumoftheweightsofitsedges.E.Frazzoli (MIT) L05: InformedSearch November1,2010 4/46Example: point-to-pointshortestpathFindtheminimum-weightpathfromstoginthegraphbelow:Solution: asimplepathP= g, d, a, s(P= g, d, b, swouldbeacceptable,too),withweightw(P) = 8.E.Frazzoli (MIT) L05: InformedSearch November1,2010 5/46Uniform-CostSearchQ start; // Initialize the queue with the starting nodewhileQisnotemptydoPick(andremove)thepathPwithlowestcostg= w(P)fromthequeueQ;if head(P)=goalthenreturnP; // Reached the goalforeachvertexvsuchthat(head(P), v) E,do //forallneighborsadd v, PtothequeueQ; // Add expanded pathsreturnFAILURE; // Nothing left to consider.Note: novisitedlist!E.Frazzoli (MIT) L05: InformedSearch November1,2010 6/46Uniform-CostSearchQ start; // Initialize the queue with the starting nodewhileQisnotemptydoPick(andremove)thepathPwithlowestcostg= w(P)fromthequeueQ;if head(P)=goalthenreturnP; // Reached the goalforeachvertexvsuchthat(head(P), v) E,do //forallneighborsadd v, PtothequeueQ; // Add expanded pathsreturnFAILURE; // Nothing left to consider.Note: novisitedlist!E.Frazzoli (MIT) L05: InformedSearch November1,2010 6/46ExampleofUniform-CostSearch: Step1Q:path costs 0sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 7/46ExampleofUniform-CostSearch: Step2Q:path costa, s 2b, s 5sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 8/46ExampleofUniform-CostSearch: Step3Q:state costc, a, s 4b, s 5d, a, s 6sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 9/46ExampleofUniform-CostSearch: Step4Q:state costb, s 5d, a, s 6d, c, a, s 7sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 10/46ExampleofUniform-CostSearch: Step5Q:state costd, a, s 6d, c, a, s 7g, b, s 10sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 11/46ExampleofUniform-CostSearch: Step6Q:state costd, c, a, s 7g, d, a, s 8g, b, s 10sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 12/46ExampleofUniform-CostSearch: Step7Q:state costg, d, a, s 8g, d, c, a, s 9g, b, s 10sstartabcdg2234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 13/46RemarksonUCSUCSisanextensionofBFStotheweighted-graphcase(UCS=BFSifalledgeshavethesamecost).UCSiscompleteandoptimal(assumingcostsboundedawayfromzero).UCSisguidedbypathcostratherthanpathdepth,soitmaygetintroubleifsomeedgecostsareverysmall.Worst-casetimeandspacecomplexityO

bW/

,whereWistheoptimal cost, andis such that all edge weights are no smaller than .E.Frazzoli (MIT) L05: InformedSearch November1,2010 14/46Greedy(Best-First)SearchUCSexplorespathsinalldirections,withnobiastowardsthegoalstate.Whatifwetrytogetclosertothegoal?Weneedameasureofdistancetothegoal. Itwouldbeidealtousethelengthoftheshortestpath... butthisisexactlywhatwearetryingtocompute!Wecanestimatethedistancetothegoalthroughaheuristicfunction,h : V R0. Inmotionplanning,wecanuse,e.g.,theEuclideandistancetothegoal(asthecrowies).Areasonablestrategyistoalwaystrytomoveinsuchawaytominimizetheestimateddistancetothegoal: thisisthebasicideaofthegreedy(best-rst)search.E.Frazzoli (MIT) L05: InformedSearch November1,2010 15/46Greedy(Best-First)SearchQ start; // Initialize the queue with the starting nodewhileQisnotemptydoPickthepathPwithminimumheuristiccosth(head(P))fromthequeueQ;if head(P) = goalthenreturnP; // We have reached the goalforeachvertexvsuchthat(head(P), v) E,doadd v, PtothequeueQ;returnFAILURE; // Nothing left to consider.E.Frazzoli (MIT) L05: InformedSearch November1,2010 16/46ExampleofGreedy(Best-First)Search: Step1Q:path cost hs 0 10s10starta2b3c1d4g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 17/46ExampleofGreedy(Best-First)Search: Step2Q:path cost ha, s 2 2b, s 5 3s10starta2b3c1d4g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 18/46ExampleofGreedy(Best-First)Search: Step3Q:path cost hc, a, s 4 1b, s 5 3d, a, s 6 4s10starta2b3c1d4g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 19/46ExampleofGreedy(Best-First)Search: Step4Q:path cost hb, s 5 3d, a, s 6 4d, c, a, s 7 4s10starta2b3c1d4g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 20/46ExampleofGreedy(Best-First)Search: step5Q:path cost hg, b, s 10 0d, a, s 6 4d, c, a, s 7 4s10starta2b3c1d4g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 21/46RemarksonGreedy(Best-First)SearchGreedy(Best-First)searchissimilarinspirittoDepth-FirstSearch: itkeepsexploringuntilithastobackupduetoadeadend.Greedysearchisnotcompleteandnotoptimal,butisoftenfastandecient,dependingontheheuristicfunctionh.Worst-casetimeandspacecomplexityO(bm).E.Frazzoli (MIT) L05: InformedSearch November1,2010 22/46Outline1Informedsearchmethods: Introduction2OptimalsearchAsearch3DynamicProgrammingE.Frazzoli (MIT) L05: InformedSearch November1,2010 23/46TheAsearchalgorithmTheproblemsUniform-Costsearchisoptimal,butmaywanderaroundalotbeforendingthegoal.Greedysearchisnotoptimal,butinsomecasesitisecient,asitisheavilybiasedtowardsmovingtowardsthegoal. Thenon-optimalitycomesfromneglectingthepast.TheideaKeeptrackbothofthecostofthepartialpathtogettoavertex,sayg(v),andoftheheuristicfunctionestimatingthecosttoreachthegoalfromavertex,h(v).Inotherwords,chooseasarankingfunctionthesumofthetwocosts:f (v) = g(v) + h(v)g(v): cost-to-come(fromthestarttov).h(v): cost-to-goestimate(fromvtothegoal).f (v): estimatedcostofthepath(fromthestarttovandthentothegoal).E.Frazzoli (MIT) L05: InformedSearch November1,2010 24/46TheAsearchalgorithmTheproblemsUniform-Costsearchisoptimal,butmaywanderaroundalotbeforendingthegoal.Greedysearchisnotoptimal,butinsomecasesitisecient,asitisheavilybiasedtowardsmovingtowardsthegoal. Thenon-optimalitycomesfromneglectingthepast.TheideaKeeptrackbothofthecostofthepartialpathtogettoavertex,sayg(v),andoftheheuristicfunctionestimatingthecosttoreachthegoalfromavertex,h(v).Inotherwords,chooseasarankingfunctionthesumofthetwocosts:f (v) = g(v) + h(v)g(v): cost-to-come(fromthestarttov).h(v): cost-to-goestimate(fromvtothegoal).f (v): estimatedcostofthepath(fromthestarttovandthentothegoal).E.Frazzoli (MIT) L05: InformedSearch November1,2010 24/46ASearchQ start; // Initialize the queue with the starting nodewhileQisnotemptydoPickthepathPwithminimumestimatedcostf (P) = g(P) + h(head(P))fromthequeueQ;if head(P) = goalthenreturnP; // We have reached the goalforeachvertexvsuchthat(head(P), v) E,doadd v, PtothequeueQ;returnFAILURE; // Nothing left to consider.E.Frazzoli (MIT) L05: InformedSearch November1,2010 25/46ExampleofASearch: Step1Q:path g h fs 0 10 10s10starta2b3c1d5g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 26/46ExampleofASearch: step2Q:path g h fa, s 2 2 4b, s 5 3 8s10starta2b3c1d5g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 27/46ExampleofASearch: step3Q:path g h fc, a, s 4 1 5b, s 5 3 8d, a, s 6 5 11s10starta2b3c1d5g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 28/46ExampleofASearch: step4Q:path g h fb, s 5 3 8d, a, s 6 5 11d, c, a, s 7 5 12s10starta2b3c1d5g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 29/46ExampleofASearch: step5Q:path g h fg, b, s 10 0 10d, a, s 6 5 11d, c, a, s 7 5 12s10starta2b3c1d5g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 30/46RemarksontheAsearchalgorithmAsearchissimilartoUCS,withabiasinducedbytheheuristich. Ifh = 0,A=UCS.TheAsearchiscomplete,butisnotoptimal. Whatiswrong?(Recallthatifh = 0thenA=UCS,andhenceoptimal...)ASearchChooseanadmissibleheuristic,i.e.,suchthath(v) h(v).(Thestarmeansoptimal.)TheAsearchwithanadmissibleheuristiciscalledA,whichisguaranteedtobeoptimal.E.Frazzoli (MIT) L05: InformedSearch November1,2010 31/46ExampleofASearch: step1Q:path g h fs 0 10 10s6starta2b3c1d1g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 32/46ExampleofASearch: step2Q:path g h fa, s 2 2 4b, s 5 3 8s6starta2b3c1d1g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 33/46ExampleofASearch: step3Q:path g h fc, a, s 4 1 5d, a, s 6 1 7b, s 5 3 8s6starta2b3c1d1g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 34/46ExampleofASearch: Step4Q:path g h fd, a, s 6 1 7b, s 5 3 8d, c, a, s 7 1 8s6starta2b3c1d1g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 35/46ExampleofASearch: step5Q:path g h fg, d, a, s 8 0 8b, s 5 3 8d, c, a, s 7 1 8s6starta2b3c1d1g02234552E.Frazzoli (MIT) L05: InformedSearch November1,2010 36/46Proof(sketch)ofAoptimalityBycontradictionAssumethatAreturnsP,butw(P) > w(wistheoptimalpathweight/cost).FindtherstunexpandednodeontheoptimalpathP,callitn.f (n) > w(P),otherwisewewouldhaveexpandedn.f (n) = g(n) + h(n)bydenition= g(n) + h(n)becausenisontheoptimalpath. g(n) + h(n)becausehisadmissible= f (n) = WbecausehisadmissibleHenceW f (n) > W,whichisacontradiction.E.Frazzoli (MIT) L05: InformedSearch November1,2010 37/46AdmissibleheuristicsHowtondanadmissibleheuristic?i.e.,aheuristicthatneveroverestimatesthecost-to-go.Examplesofadmissibleheuristicsh(v) = 0: thisalwaysworks! However,itisnotveryuseful,andinthiscaseA= UCS.h(v) = distance(v, g)whentheverticesofthegraphsarephysicallocations.h(v) = v gp,whentheverticesofthegrapharepointsinanormedvectorspace.AgeneralmethodChoosehastheoptimalcost-to-gofunctionforarelaxedproblem,thatiseasytocompute.(Relaxedproblem: ignoresomeoftheconstraintsintheoriginalproblem)E.Frazzoli (MIT) L05: InformedSearch November1,2010 38/46AdmissibleheuristicsHowtondanadmissibleheuristic?i.e.,aheuristicthatneveroverestimatesthecost-to-go.Examplesofadmissibleheuristicsh(v) = 0: thisalwaysworks! However,itisnotveryuseful,andinthiscaseA= UCS.h(v) = distance(v, g)whentheverticesofthegraphsarephysicallocations.h(v) = v gp,whentheverticesofthegrapharepointsinanormedvectorspace.AgeneralmethodChoosehastheoptimalcost-to-gofunctionforarelaxedproblem,thatiseasytocompute.(Relaxedproblem: ignoresomeoftheconstraintsintheoriginalproblem)E.Frazzoli (MIT) L05: InformedSearch November1,2010 38/46AdmissibleheuristicsHowtondanadmissibleheuristic?i.e.,aheuristicthatneveroverestimatesthecost-to-go.Examplesofadmissibleheuristicsh(v) = 0: thisalwaysworks! However,itisnotveryuseful,andinthiscaseA= UCS.h(v) = distance(v, g)whentheverticesofthegraphsarephysicallocations.h(v) = v gp,whentheverticesofthegrapharepointsinanormedvectorspace.AgeneralmethodChoosehastheoptimalcost-to-gofunctionforarelaxedproblem,thatiseasytocompute.(Relaxedproblem: ignoresomeoftheconstraintsintheoriginalproblem)E.Frazzoli (MIT) L05: InformedSearch November1,2010 38/46Admissibleheuristicsforthe8-puzzleInitialstate:1 52 6 37 4 8Goalstate:1 2 34 5 67 8Whichofthefollowingareadmissibleheuristics?h = 0h = 1h =numberoftilesinthewrongpositonh =sumof(Manhattan)distancebetweentilesandtheirgoalposition.E.Frazzoli (MIT) L05: InformedSearch November1,2010 39/46Admissibleheuristicsforthe8-puzzleInitialstate:1 52 6 37 4 8Goalstate:1 2 34 5 67 8Whichofthefollowingareadmissibleheuristics?h = 0 YES,alwaysgoodh = 1h =numberoftilesinthewrongpositonh =sumof(Manhattan)distancebetweentilesandtheirgoalposition.E.Frazzoli (MIT) L05: InformedSearch November1,2010 39/46Admissibleheuristicsforthe8-puzzleInitialstate:1 52 6 37 4 8Goalstate:1 2 34 5 67 8Whichofthefollowingareadmissibleheuristics?h = 0 YES,alwaysgoodh = 1NO,notvalidingoalstateh =numberoftilesinthewrongpositon YES,teleporteachtiletothegoalinonemoveh =sumof(Manhattan)distancebetweentilesandtheirgoalposition.E.Frazzoli (MIT) L05: InformedSearch November1,2010 39/46Admissibleheuristicsforthe8-puzzleInitialstate:1 52 6 37 4 8Goalstate:1 2 34 5 67 8Whichofthefollowingareadmissibleheuristics?h = 0 YES,alwaysgoodh = 1NO,notvalidingoalstateh =numberoftilesinthewrongpositon YES,teleporteachtiletothegoalinonemoveh =sumof(Manhattan)distancebetweentilesandtheirgoalposition. YES,moveeachtiletothegoalignoringothertiles.E.Frazzoli (MIT) L05: InformedSearch November1,2010 39/46ApartialorderofheuristicfunctionsSomeheuristicsarebetterthanothersh = 0isanadmissibleheuristic,butisnotveryuseful.h = hisalsoanadmissibleheuristic,anditthebestpossibleone(itgiveustheoptimalpathdirectly,nosearches/backtracking)PartialorderWesaythath1dominatesh2ifh1(v) h2(v)forallverticesv.Clearly,hdominatesalladmissibleheuristics,and0isdominatedbyalladmissibleheuristics.ChoosingtherightheuristicIngeneral,wewantaheuristicthatisasclosetohaspossible. However,suchaheuristicmaybetoocomplicatedtocompute. Thereisatradeobetweencomplexityofcomputinghandthecomplexityofthesearch.E.Frazzoli (MIT) L05: InformedSearch November1,2010 40/46ConsistentheuristicsAnadditionalusefulpropertyforAheuristicsiscalledconsistencyAheuristich : X R0issaidconsistentifh(u) w (e= (u, v)) + h(v), (u, v) E.Inotherwords,aconsistentheuristicssatisesatriangleinequality.Ifhisaconsistentheuristics,thenf = g+ hisnon-decreasingalongpaths:f (v) = g(v) + h(v) = g(u) + w(u, v) + h(v) f (u).Hence,thevaluesoff onthesequenceofnodesexpandedbyAisnon-decreasing: therstpathfoundtoanodeisalsotheoptimalpath noneedtocomparecosts!E.Frazzoli (MIT) L05: InformedSearch November1,2010 41/46Outline1Informedsearchmethods: Introduction2Optimalsearch3DynamicProgrammingE.Frazzoli (MIT) L05: InformedSearch November1,2010 42/46DynamicProgrammingTheoptimalityprincipleLetP= (s, . . . , v, . . . g)beanoptimalpath(fromstog). Then,foranyv P,the sub-path S= (v, . . . , g) is itself an optimal path (from vto g).UsingtheoptimalityprincipleEssentially,optimalpathsaremadeofoptimalpaths. Hence,wecanconstructlongcomplexoptimalpathsbyputtingtogethershortoptimalpaths,whichcanbeeasilycomputed.Fundamentalformulaindynamicprogramming:h(u) = min(u,v)E[w( (u, v) ) + h(v)] .Typically,itisconvenienttobuildoptimalpathsworkingbackwardsfromthegoal.E.Frazzoli (MIT) L05: InformedSearch November1,2010 43/46AspecialcaseofdynamicprogrammingDijkstrasalgorithmQ V {Allstatesgetinthequeue}.forallv V,h(v) = (ifv VG, 0otherwise)while Q = dou arg minvQ h(v) {Pickminimum-costvertexinQ}forall e= (v, u) E doh(v) min{h(v),h(u) + w(e)} {Relaxcosts}RecoveringoptimalpathsTheoutputofDijkstrasalgorithmisinfacttheoptimalcost-to-gofunction,h.Fromanyvertex,wecancomputetheoptimaloutgoingedgeviathedynamicprogrammingequation.E.Frazzoli (MIT) L05: InformedSearch November1,2010 44/46Dijkstrasalgorithm: exampleInf InfInf0465216 240465215 240465215 24046521Dynamicprogrammingrequiresthecomputationofalloptimalsub-paths,fromallpossibleinitialstates(curseofdimensionality).On-linecomputationiseasyviastatefeedback: convertanopen-loopproblemintoafeedbackproblem. Thiscanbeusefulinreal-worldapplications,wherethestateissubjecttoerrors.E.Frazzoli (MIT) L05: InformedSearch November1,2010 45/46ConcludingremarksAoptimalandveryeectiveinmanysituations. However,insomeapplications,itrequirestoomuchmemory. SomepossibleapproachestoaddressthisproblemincludeBranchandboundConict-directedAAnytimeAOthersearchalgorithmsDandD-lite: versionsofAforuncertaingraphs.Hillsearch: movetotheneighboringstatewiththelowestcost.Hillsearchwithbackup: movetotheneighboringstatewiththelowestcost,keeptrackofunexploredstates.Beamalgorithms: keepthebestkpartialpathsinthequeue.E.Frazzoli (MIT) L05: InformedSearch November1,2010 46/46MIT OpenCourseWarehttp://ocw.mit.edu 16.410 / 16.413 Principles of Autonomy and Decision MakingFall 2010 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .