EFFICIENT SEQUENTIAL DECISION‐MAKING ALGORITHMS FOR...

EFFICIENTSEQUENTIALDECISION‐MAKINGALGORITHMSFORCONTAINERINSPECTION

OPERATIONS

SushilMi;alandFredRobertsRutgersUniversity&DIMACS

DavidMadiganColumbiaUniversity&DIMACS

•CurrentlyinspecLngonlysmall%ofcontainersarrivingatports

PortofEntryInspecLonAlgorithms

•Goal:FindwaystointerceptillicitnuclearmaterialsandweaponsdesLnedfortheU.S.viathemariLmetransportaLonsystem

PortofEntryInspecLonAlgorithms

Aim:Developdecisionsupportalgorithmsthatwillhelpusto“opLmally”interceptillicitmaterialsandweaponssubjecttolimitsondelays,manpower,andequipment

Findinspec*onschemesthatminimizetotalcostincludingcostoffalseposi*vesandfalsenega*ves

MobileVACIS:truck‐mountedgammarayimagingsystem

SequenLalDecisionMakingProblem• Containersarrivingareclassifiedintocategories• Simplecase:0=“ok”,1=“suspicious”• Containershavea;ributes,eitherinstate0or1• Samplea)ributes:

– Doestheship’smanifestsetoffanalarm?– IstheneutronorGammaemissioncountabovecertainthreshold?

– DoesaradiographimagereturnaposiLveresult?– DoesaninducedfissiontestreturnaposiLveresult?

• Inspec3onscheme:– specifieswhichinspec*onsaretobemadebasedonpreviousobserva*ons

• Different“sensors”detectpresenceorabsenceofvariousa;ributes

•SimplestCase:A;ributesareinstate0or1

•Then:Containerisabinarystringlike011001

•So:ClassificaLonisadecisionfunc*onF thatassignseachbinarystringtoacategory.

F011001 0 or 1

Ifa;ributes2,3,and6arepresent,assigncontainertocategoryF(011001).

SequenLalDecisionMakingProblem

•Iftherearetwocategories,0and1,decisionfuncLonFisa Booleanfunc*on.

•Example:

•ThisfuncLonclassifiesacontainerasposiLveiffithasatleasttwoofthea;ributes.

abcF(abc)00000010010001111000101111011111

SequenLalDecisionMakingProblem

BinaryDecisionTreeApproach•BinaryDecisionTree:

–Nodesaresensorsorcategories(0or1)–Twoarcsexitfromeachsensornode,labeledlegandright.–Taketherightarcwhensensorsaysthea;ributeispresent,legarcotherwise

abcF(abc)00000010010001111000101111011111

CostofaBDT• CostofaBDTcomprisesof:– CostofuLlizaLonofthetreeand– CostofmisclassificaLon

0 0|0 0|0 1|0 1|0

1 0|1 0|1 1|1 1|1

0 0|0 1|0 1|0 1|0 1|0

1 0|1 0|1 0|1 1|1 0|1 1|1 0|1

( ) ( )

( )

( )

( )

a a b a b c a c

a a b a b c a c

a b c a c FP

a b a b c a c FN

f P C P C P P C P C

P C P C P P C P C

P P P P P P C

P P P P P P P P C

! = + + +

+ + + +

+ +

+ + +

ABDT, τwithn=3

P1ispriorprobabilityofoccurrenceofabadcontainer

Pi|j isthecondiLonalprobabilitythatgiventhecontainerwasinstatej,itwasclassifiedas i

SensorThresholds

Ps=0|0 + Ps=1|0 = 1Ps=1|1 + Ps=0|1 = 1

•Tscanbeadjustedforminimumcost

•Anandet.al.reportedthecheapesttreesobtainedfromanextensivesearchoverarangeofsensorthresholds.Forexample:forn=4,194,481testswereperformedwiththresholdsvaryingbetween[‐4,4]withastepsizeof0.4

• Approach:– BuildsonideasofStroudandSaeger1atLosAlamosNaLonalLaboratory

– InspecLonschemesareimplementedasBinaryDecisionTreeswhichareobtainedfromvariousBooleanfuncLonsofdifferenta;ributes

– Only“Complete”and“Monotonic”BooleanfuncLonsgivepotenLallyacceptableBinarydecisiontrees

– n=4

1Stroud,P.D.andSaegerK.J.,“Enumera*onofIncreasingBooleanExpressionsandAlterna*veDigraphImplementa*onsforDiagnos*cApplica*ons”,ProceedingsVolumeIV,Computer,Communica*onandControlTechnologies,(2003),328‐333

Previouswork:Aquickoverview

OpLmumThresholdComputaLon• ExtensivesearchoverarangeofthresholdshassomepracLcaldrawbacks:– Largenumberofthresholdvaluesforeverysensor

– Largestepsize– GrowsexponenLallywiththenumberofsensors(computaLonallyinfeasibleforn>4)

• Therefore,weuLlizenon‐linearopLmizaLontechniqueslike:

– Gradientdescentmethod

– Newton’smethod

SearchingthroughaGeneralizedTreeSpace

• WeexpandthespaceoftreesfromStroudandSaeger’s“Complete”and“Monotonic”BooleanFuncLonstoCompleteandMonotonicBDTs,because…

• UnlikeBooleanfuncLons,BDTsmaynotconsiderallsensoroutputstogiveafinaldecision

• Advantages:– Allowsmore,potenLallyusefultreestoparLcipateintheanalysis

– HelpsdefininganirreducibletreespaceforsearchoperaLons

– MovesfocusfromBooleanFuncLonstoBinaryDecisionTrees

RevisiLngMonotonicity• MonotonicDecisionTrees– Abinarydecisiontreewillbecalledmonotonicifall

thelegleafsareclass“0”andalltherightleafsareclass“1”.

• Example:

abcF(abc)00000010010101111000101111001111

RevisiLngCompleteness• CompleteDecisionTrees– Abinarydecisiontreewillbecalledcompleteifeverysensoroccursatleastonceinthetreeandatanynon‐leafnodeinthetree,itslegandrightsub‐treesarenotidenLcal.

• Example:

abcF(abc)00000011010101111000101111011111

TheCMTreeSpace

No.ofaPributes

Dis*nctBDTsTreesFromCMBooleanFunc*ons

CompleteandMonotonicBDTs

2 74 4 4

3 16,430 60 114

4 1,079,779,602 11,808 66,000

TreeSpaceTraversal• GreedySearch

1. RandomlystartatanytreeintheCMtreespace2. FinditsneighboringtreesusingneighborhoodoperaLons3. Movetotheneighborwiththelowestcost4. IterateLllthesoluLonconverges

– TheCMTreespacehasalotoflocalminima.Forexample:9inthespaceof114treesfor3sensorsand193inthespaceof66,000treesfor4sensors.

• ProposedSoluLons• StochasLcSearchMethodwithSimulatedAnnealing• GeneLcAlgorithmsbasedSearchMethod

TreeSpaceIrreducibility

• WehaveprovedthattheCMtreespaceisirreducibleundertheneighborhoodoperaLons

• SimpleTree:– AsimpletreeisdefinedasaCMtreeinwhicheverysensoroccursexactlyonceinsuchawaythatthereisexactlyonepathinthetreewithallsensorsinit.

ToProve:Givenanytwotreesτ1,τ2inCMtreespace,τn,τ2canbereachedfromτ1byanarbitrarysequenceofneighborhoodoperaLons

Weprovethisinthreedifferentsteps:1. Anytreeτ1canbeconvertedtoasimpletreeτs12. Anysimpletreeτs1canbeconvertedtoanyothersimple

treeτs23. Anysimpletreeτs2canbeconvertedtoanytreeτ2

CMTreespace,τn

Simpletreesτ1

τs1 τs2

τ2

Results

• SignificantcomputaLonalsavingsoverpreviousmethods

• Haverunexperimentswithupto10sensors

• GeneLcalgorithmsespeciallyusefulforlargerscaleproblems

CurrentWork

• Treeequivalence

• TreereducLonandirreducibletrees

• CanonicalformrepresentaLonoftheequivalenceclassoftrees

• RevisiLngcompletenessandmonotonicity

ThankYou!

MonotonicBooleanFunc3ons:•Giventwostringsx1x2…xn, y1y2…yn

•Fismonotoniciffxi ≥ yi foralliimpliesthatF(x1x2…xn) ≥ F(y1y2…yn).

CompleteBooleanFunc3ons:•BooleanfuncLonFisincompleteifFcanbecalculatedbyfindingatmostn-1a;ributesandknowingthevalueoftheinputstringonthosea;ributes•Inotherwords,Fiscompleteifallthea;ributescontributetowardstheoutput

Previouswork:Aquickoverview

Previouswork:Aquickoverview• StroudandSaeger:“bruteforce”algorithmforenumeraLng

binarydecisiontreesimplemenLngcomplete,monotonicBooleanfuncLonsandchoosingleastcostBDT.

263,515,92068945x10185

11,8081141,079,779,6024

60916,4303

42742

BDTsfromCMBooleanFunc*ons

CMBooleanExpressions

Dis*nctBDTsNo.ofaPributes

Infeasiblebeyondn>4!

ProblemswithStandardApproaches• GradientDescentMethod:SetngthevalueofthestepsizeheurisLcally,since:– Toosmallstepsize:longLmetoconverge

– Toobigstepsize:mightskiptheminimum

• Newton’sMethod:

– TheconvergencedependslargelyonthestarLngpoint– OccasionallydrigsinthewrongdirecLonandhencefailstoconverge.

• SoluLon:combina*onofgradientdescentandNewton’smethods

TheCombinedMethod1. IniLalizeTasvectorofrandomsensorthresholdvalues

2. Compute∂f ,Hf(τ)3. IfHf(τ)isnotposiLvedefinite,thenfindacloseapproximaLon

4. IfHf(τ)isnotwell‐condiLoned,thentakeafewstepsusinggradientdescentunLlitbecomeswell‐condiLoned

5. TakeastepusingNewton’smethod6. Repeatsteps1‐5unLlthesoluLonconverges7. Repeatsteps1‐6afewLmesandchoosetheoverallminimumcost

TreeNeighborhoodandTreeSpace• Structurebasedmethods

• ClassificaLonbasedmethods

• Wechoosestructurebasedneighborhoodmethodsbecause:

• Smallchangesintreestructuredonoteffectthecostsignificantly,and…

• AllBDTswithsameBooleanfuncLonmaydifferalotincost

TreeNeighborhoodandTreeSpace

• DefinetreeneighborhoodsuchthattheCompleteandMonotonic(CM)treespaceisirreducible

• Irreducibility– AnytreeintheCMtreespacecanbereachedfromanyothertreebyusingtheneighborhoodoperaLonsrepeLLvely

– AnirreducibleCMtreespacehelps“search”forthecheapesttreesusingneighborhoodoperaLons

SearchOperaLons

• SplitPickaleafnodeandreplaceitwithasensorthatisnotalreadypresentinthatbranch,andtheninsertarcsfromthatsensorto0andto1.

SearchOperaLons

• SwapPickanon‐leafnodeinthetreeandswapitwithitsparentnodesuchthatthenewtreeissLllmonotonicandcompleteandnosensoroccursmorethanonceinanybranch.

SearchOperaLons

• MergePickaparentnodeoftwoleafnodesandmakeitaleafnodebycollapsingthetwoleafnodesbelowit,orpickaparentnodewithoneleafnode,collapsebothofthemandshigthesub‐treeupinthetreebyonelevel.

SearchOperaLons

• ReplacePickanodewithasensoroccurringmorethanonceinthetreeandreplaceitwithanyothersensorsuchthatnosensoroccursmorethanonceinanybranch.

StochasLcSearchMethod1. RandomlystartatanytreeinCMspace

2. Finditsneighboringtrees,andfindtheiropLmumcosts

3. Selectmoveaccordingtothefollowingprobability.Ifweareattheithtreeτi,thentheprobabilityofgoingtoitskthneighborτik,isgivenby

whereniisthenumberofneighboringtreesofτi4. IniLalizethetemperaturet = 1,andloweritindiscreteunequal

stepsagereverymhopsunLlthesoluLonconverges

5. Repeatsteps1‐4afewLmesandchoosetheoverallminimum

( )

( )

1

1

1

( ) ( )

( ) ( )i

t

i ik

ki nt

i ij

j

f fP

f f

! !

! !=

=

"


1. τ1τs1:

•Repeatedsubtreemerger•Toremoveanodeatdepthk, atmostk-2 needtobecheckedforcompleteness•WeprovethatthereisatleastonenodeinasubtreeatanyLme,thatcanbemergedwithoutdisturbingtheoverallcompletenessconstraint


2. τs1τs2:

•Firstconvertτs1tohavesimilar“skeleton”asτs2•ThenuserepeatedSwapoperaLons

SPLIT SPLITMERGE

MERGE

SWAPSWAPSWAP


3. τs2τ2:

•TheprocessofgoingfromatreetoasimpletreeisenLrelyreversible.Forexample:

–anysplitoperaLoncanbereversedusingamergeoperaLonandvice‐versa

–swapandreplaceoperaLonscanbereversedbyoppositeswapandreplaceoperaLons,respecLvely

•Therefore,τ2τs2impliesτs2τ2

GeneLcAlgorithmsbasedSearch

• TheunderlyingideaistogetapopulaLonof“be;er”treesfromacurrentpopulaLonof“good”treesbyusingthebasicoperaLons:– SelecLon– Crossover–MutaLon

• “be;er”decisiontreescorrespondtotheonescheaperthanthecurrentones(“good”)


• Selec*on:– Selectarandom,iniLalpopulaLonofNtreesfromCMtreespace

• Crossover:– PerformedkLmesbetweeneverypairoftreesinthecurrentbestpopulaLon,bestPop


– ForeachcrossoveroperaLonbetweentwotreesτiandτj,werandomlyselectanodeineachtreeandexchangetheirsubtrees

– However,weimposecertainrestricLonontheselecLonofnodes,sothattheresultanttreessLlllieinCMtreespace


• Muta*on:

– Performedagereverym generaLonsofthealgorithm

– WedotwotypesofmutaLons:

1. Generateallneighborsofthecurrentbest

populaLonandputthemintothegenepool

2. ReplaceafracLonofthetreesofbestPopwithrandomtreesfromtheCMtreespace

ResultsI‐ThresholdOpLmizaLon

• ManyLmestheminimumobtainedusingtheopLmizaLonmethodwasconsiderablylessthantheonefromtheextensivesearchtechnique.

0 20 40 60 80 100

100

150

200

250

300

350

400

450

500

Tree Number

Total Cost

Tree costs at optimum thresholds

Combined Optimization

Extensive search

ResultsII‐SearchingCMTreeSpace

• StochasLcSearchMethod:• Successfullyperformedexperimentsforupton =5• Forexample,for4sensors(66,000trees)

– 100differentexperimentswereperformed

– Eachexperimentwasstarted10LmesrandomlyatsometreeandchainswereformedbymakingstochasLcmovesintheneighborhood,unLlconvergence

– Only4890treeswereexaminedonaverageforeveryexperiment

– Globalminimumwasfound82Lmeswhilethesecondbesttreewasfound10Lmes


• GeneLcAlgorithmsbasedMethod:• Successfullyperformedexperimentsforupton =10• For4sensors(66,000trees)

– 100differentexperimentswereperformed

– EachexperimentwasstartedwitharandompopulaLonof20treesandwasconLnuedfor27generaLonseach;themutaLonsareperformedagerevery3generaLons

– Only1440treeswereexaminedonaverageforeveryexperiment

– Globalminimumwasfoundall100Lmes– ThealgorithmreturnsawholepopulaLonofgoodtreesmostofwhichbelongto50besttrees


• Similarly,forn=5,thetreespaceconsistsofmorethan22.5billiontrees,wealwaysobtainedoneofthefollowingbesttrees:

• Eachofthesetreescosts41.4668


• Forn=10,followingwerethebesttreesoverafewruns:

CurrentWork• TreeEquivalence

–DecisionEquivalence:TwoormoredecisiontreesarecalleddecisionequivalentiftheirunderlyingBooleanfuncLonissame–CostEquivalence:Twotreesarecalledcostequivalentifftheyare“transposes”ofeachother.Forexample:

–ThesizeoflargestequivalenceclassalsoincreasesmorethandoubleexponenLallywithn–Therefore,wedefineaspaceofequivalenceclassesofdecisiontrees,withaunique,canonicalrepresentaLonofeachclass

CurrentWork• TreeReducLonandIrreducibleTrees

–Atransposeofacompletetreecanbeincomplete.Forexample:

–IrreducibleTrees:Atreewillbecalledirreducible,ifallthetreesbelongingtoitsequivalenceclassarecomplete

CurrentWork• CanonicalFormRepresentaLon

– WechosealexicographicrepresentaLonoftheequivalenceclass– “Pull‐up”thelexicographicallysmallestsensorastheroot

nodeandrecursivelyrepeattheprocedureinthelegandrightsubtrees

– AcanonicalformrepresentaLonofanequivalenceclassenablesusto“shrink”thetreespace

– Everytreeisfirstconvertedtoitscanonicalform,beforecheckingforitscost,thereforecheckingthecostofonlyonetreefromanequivalenceclassissufficient

CurrentWork• CanonicalFormRepresentaLon:Example

CurrentWork• RevisiLngCompleteness:

1. Atanynodeinatree,thelegandrightsubtreesshouldnotbe

cost‐equivalent

2. Atanynodeinatree,thelegandrightsubtreesshouldnothave

idenLcalBooleanfuncLon

• 2covers1,therefore…

• Equi‐completeBDT:Abinarydecisiontreewillbecalledequi‐

completeifeverysensoroccursatleastonceinthetreeand,atany

non‐leafnode,thelegandrightsubtreesdonotcorrespondtosame

BooleanfuncLon.

CurrentWork• RevisiLngMonotonicity:

1. Acost‐equivalenttreeofamonotonictreecanbenon‐monotonic(‘0’asrightleaf,‘1’aslegleaforboth).

• Equi‐monotonicBDT:Abinarydecisiontreewillbecalledequi‐monotonic,ifallthetreesbelongingtoitsequivalenceclassaremonotonic.

Discussion

1. TheexhausLvesearchmethod,forfindingtheopLmumthresholdsforagiventree,becomepracLcallyinfeasiblebeyondaverysmallnumberofsensors.

2. ThethresholdopLmizaLontechniquediscussedinourworkprovidefasterandbe;erwaystocalculatetheopLmaltotalcostofatree.

3. TheexhausLvesearchmethod,forfindingthecheapesttreeintheenLrespaceoftreesisalsohardtoextendbeyondaverysmallnumberofsensors.

4. WedescribedacoupleofefficientsearchmethodstofindthebesttreesintheCMtreespace

Discussion

5. ExpandingtheideasofmonotonicityandcompletenessfromBDFstoBDTsisreasonablebecause:• certaintreesobtainedfromincomplete/non‐

monotonicBDFsarepotenLallyvalidBDTsand,• itfacilitatestreesearchalgorithms

6. WeprovedthattheproposedCMtreespaceisirreducibleunderthedefinedneighborhoodoperaLons.

7. WediscussedtheideasoftreeequivalenceandtreereducLonthathelpus“shrink”thetreespace

8. Wedescribewaytorepresentanequivalenceclasswithaunique,canonicalform.

FutureWork

• Amorebasicandrigorousanalysisofmonotonicityisrequired

• DifferentinstancesofasensorinatreecanbesettodifferentthresholdsforopLmumcost

• Sensormodels,otherthantheoneweusecouldbetried

• Dr.FredRoberts

• DIMACS,NSFandONR

• Dr.PeterMeerandOncelTuzel

• Dr.EndreBoros

Acknowledgements

EFFICIENT SEQUENTIAL DECISION‐MAKING ALGORITHMS FOR...

Documents

Transcript of EFFICIENT SEQUENTIAL DECISION‐MAKING ALGORITHMS FOR...