Using agent-based models and machine learning to enhance spatial decision support systems

Using agent-based models and machine learning to enhance spatial decision support systems

Application to resource allocation in situations of urban catastrophes

Defended by CHU Thanh-Quang, the 1st July 2011

Supervisor: M. Alexis DROGOUL, DR, MSI/IRD, UMI 209 UMMISCOCo-supervisor: M. Alain BOUCHER, Prof. AUF, MSI/IRD UMI 209 UMMISCOReviewers: M. Nicolas BREDECHE, MdC HDR, LRI, Université Paris-Sud M. Bernard PAVARD, Prof., IRIT, Université Paul Sabatier, ToulouseExaminators: M. NGUYEN Hong Phuong, Prof., VAST, Hanoi, Vietnam Mme. Julie DUGDALE, MdC, LIG, Université Pierre Mendès-France M. Christophe GONZALES, Prof., LIP6, UPMC, Paris, France

PhD thesis of University Pierre and Marie Curie, Paris, France

1

Outline• Context: Spatial Decision Support System (SDSS)

for resource allocation in emergency response

• Proposal:

• ABM&GIS: Agent-Based Modeling and Geographic Information System to build the underlying models of SDSS,

• PD: Participatory Design to involve users in the design process and to enhance the realism of the models,

• ML: Machine Learning algorithms to automate the extraction of knowledge from stakeholders

• Experiments and results

• Conclusion and prospects2

Disasters

• Natural disasters

• Earthquake

• Tsunami

• Flooding, etc.

• Causing huge loss of human life and property

• Cities are especially vulnerable to disasters:

• Density of population, buildings and infrastructure

No of events: 3,341

No of people killed: 1,144,006

Average killed per year: 39,448

No of people affected: 4,742,092,443

Average affected per year:

163,520,429

Ecomomic Damage (US$ X 1,000):

673,457,207

Ecomomic Damage per year (US$ X 1,000):

23,222,662

Natural disasters in Asia (1980 - 2010)

http://www.preventionweb.net/3

http://www.preventionweb.net/

http://www.preventionweb.net/

Emergency response & resource allocation

4

• Emergency response [CPC, 07]:

• Reducing life-threatening conditions

• Providing life-sustaining aid

• Stopping additional damage to property

• Resource allocation (particularly important in urban areas):

• Where and when do rescue resources need to be allocated?

• How to organize and coordinate these allocations?

Loss

Response effectiveness

Spatial decision support systems (SDSSs)

• Decision support systems aim at:

• supporting decisions of stakeholders

• training stakeholders to solve problems

• Spatial DSSs involve location in decisions [CPC, 07], e.g.:

• design evacuation and rescue routes

• allocate evacuees to shelters

• select optimal locations for rescue teams

5

pointing operations, a wireless connection is immediately

A multiagent-based simulation with a large number of was performed in

parallel with the experiment in real space. See-through head-mounted displays are not suitable for presenting the simulation of augmented experiments, since it is unsafe to mask the views of passengers. As described above, since we used mobile phones, small and low-resolution images of three dimensional virtual spaces are difficult to understand. Instead of displaying visual simulations, the mobile phones

in this

in real space.

Figure 4. Outdoor Experiment

GPS

Outdoor Real Space2D Virtual Space

GPS

Outdoor Real Space2D Virtual Space

Digital City, from [Ishida et al., 07]

Literature of SDSSs for emergency response

• DrillSim [Balasubramanian et al., 06], [Massaguer et al., 06],

• ALADDIN [Adams et al., 08], [Gianni et al., 08],

• DEFACTO [Marecki et al., 05], [Schurr et al., 05],

• Plan-C [Narzisi et al., 07],

• Digital City (JST CREST) [Ishida et al., 07], etc.

• Modeling and Simulation with ABM & GIS are core techniques to:

• model emergency situations

• design response solutions

6

In summary, an augmented experiment consists of 1) to represent human

Figure 3. Indoor Experiment

Indoor Real Space3D Virtual Space

Camera

Indoor Real Space3D Virtual Space

Camera


ALADDIN (Autonomous Learning Agents for Decentralized Data and Information Networks)

[Adams et al., 08], [Gianni et al., 08]

• Evacuating a building on fire

• Improve situational awareness

• data collection

• data fusion

• Improve path planing and coordination strategy

• auction methods

• coalition methods

• learning in games

7

DEFACTO (Demonstrating Effective Flexible Agent Coordination of Teams through Omnipresence)

[Marecki et al., 05], [Schurr et al., 05]

• Fire evacuation

• Improve situational awareness

• 3D visualization

• human-interaction

• Focus on modeling

• detailed-level of situations

8

(a) (b)

(c) (d)

(e) (f)

Plan-C (Planning with Large Agent-Networks against Catastrophes)

[Narzisi et al., 07]

• Emergency planning, medical relief operations

• use evolutionary algorithms

• Response planning as a problem of multi-objectives optimization

9

Project Application Main limitationRealism of situations Lack of (behavioral realism)

DrillSim

DEFACTO

ALADDIN

ResQ Freiburg

PLAN C

Damas Rescue

Digital City

Fire evacuation Difficultly generalized Small scale

Fire evacuation Manual modeling 3D with OpenGL Learning from users’ solution

Fire evacuation Poor user-interface Simple GIS

Search&Rescue Lack of reusability Simple GIS Interest on domain knowledge

Medical relief operations

Limited configurability of agent behavior

GIS Interest on domain knowledge

Search&Rescue Lack of flexibility Simple GIS Interest on domain knowledge

Large-scale evacuation Lack of solution support GIS Learning from users’ solution

•Lack of realism of emergency situations

• Environments are simply represented in small scale

•Lack of realism of rescue activities (i.e. agent behaviors)

• Small interest on domain knowledge to improve response effectiveness

10

Lack of flexibility and realism

Proposal

11

• Problem: Lack of realism of emergency situations

• Step 1: Using ABM&GIS (geospatial data of Hanoi and earthquake loss estimation of IG-VAST) to build a realistic rescue model

• Problem: Lack of realism of rescue activities

• Step 2: Using Participatory Design to involve practitioners, experts of emergency to improve agent behaviors

• Problem: The improvement of agent behaviors has to be made manually and offline by modelers

• Step 3: Using Machine Learning to automate the acquisition of experts’ knowledge

Step 1: Building a realistic rescue model

12

• Collect from Earthquake Loss Estimation System of IG-VAST [Nguyen-Hong, 03]:

• Real GIS data of Hanoi

• Disaster impact data: building damage and casualties

!"#!

!"#$%&'%()*!!(++),"-.+)"-)/#(-0(+)1023)*-!4567)83(),"%%"109:)+!-((9+3"2)+3"1+)13&2)0+)$"++0'%()1023);<)*9&%=+2)*-!456)(>2(9+0"9?)+")23(-()&-()!(-2&09%=)'(&#20,#%)2309:+)%(,2)2")@"7)

!!"#$$%&'()*+,-*./01*234*(5*63738$6*9:;<6;%8&*;%*=36;%'*1;&)#;")*(5*>3%(;*?;)@*

)*%(>0+)&9@)A3#"9:)$-"$"+(@).()2")@")&%%)230+)1"-B),-".)C-&9!()@#-09:)23()=(&-7)5)10%%)!"9209#().=)+2#@0(+)09):("%":=)230+)=(&-?)&9@)02)10%%)@($(9@)"9)23()D"%#.()",).=)!%&++)+!3(@#%(7)E#2)02)0+)+".(2309:)23&2)-(&%%=)092(-(+2+).(?)+")13=)9"2F)))

"""# $%&&"'"(")*!%+!+,),-.!"/).-/&0"1!*9"23(-)$"++0'0%02=)1()(9D0+&:(@)0+)2")+(2)#$)&9"23(-)092(-9+30$),"-)9(>2)=(&-?)."+2)%0B(%=)09)23()59+202#2()",)4("$3=+0!+G)

A$9B(#;$%)$6) H!"..#90!&20"9?) $("$%() &1&-(9(++IG) 5) 3&@) +".() !&$&'0%020(+) 09)1('.&$$09:)2(!39"%":0(+)+")5)!"#%@)#+()23(.)2")@")+".()1"-B)09)23()!"9209#02=)",)23()"9()5)&%-(&@=).&@()230+)20.(?)1()!"#%@)&%+")#+()23()*-!456)6(-D(-+)&9@)"23(-)1(')$-"@#!2+),-".)J6K5?)'(!&#+()23()54A)!3"+()230+)$-"D0@(-7))

L"-B)"9)23()5;$<6)H@($(9@09:)"9)'#@:(2+IG)230+) 0+).=)'0::(+2)10+37)59@((@?)5)1"#%@)'()D(-=)092(-(+2(@)2")1"-B)&%+")"9)23():-"#9@)1023)+".():("$3=+0!&%)@(D0!(+) 09)23(),#2#-(?)&9@)230+)1"#%@),02)1023).=)2-&0909:+)",)9(>2)=(&-7)

6".() &;7:<3);(%&) ",) 23() +(0+.0!) 9(21"-B+F) 830+) 0+) &) 2-&!B) 130!3) *%(>0+) 23"#:32)&'"#2?)&9@)&!2#&%%=)A3#"9:?)*%(>0+)&9@).=+(%,)3&D()&)%"2)",)0@(&)2")!"9209#()1"-B09:)2":(23(-)09)23(),0(%@)",)9&2#-&%)@0+&+2(-+)&9@)-(+!#()+0.#%&20"9+7)

6")23(+()&-()$"++0'0%020(+)'"-9)23-"#:3)230+)!"%%&'"-&20D()1"-B7)


12




• Rescue agents: inspired from the agents found in RobocupRescue simulations [www.robocuprescue.org]

!"#!

!"#$%&'%()*!!(++),"-.+)"-)/#(-0(+)1023)*-!4567)83(),"%%"109:)+!-((9+3"2)+3"1+)13&2)0+)$"++0'%()1023);<)*9&%=+2)*-!456)(>2(9+0"9?)+")23(-()&-()!(-2&09%=)'(&#20,#%)2309:+)%(,2)2")@"7)

!!"#$$%&'()*+,-*./01*234*(5*63738$6*9:;<6;%8&*;%*=36;%'*1;&)#;")*(5*>3%(;*?;)@*

)*%(>0+)&9@)A3#"9:)$-"$"+(@).()2")@")&%%)230+)1"-B),-".)C-&9!()@#-09:)23()=(&-7)5)10%%)!"9209#().=)+2#@0(+)09):("%":=)230+)=(&-?)&9@)02)10%%)@($(9@)"9)23()D"%#.()",).=)!%&++)+!3(@#%(7)E#2)02)0+)+".(2309:)23&2)-(&%%=)092(-(+2+).(?)+")13=)9"2F)))

"""# $%&&"'"(")*!%+!+,),-.!"/).-/&0"1!*9"23(-)$"++0'0%02=)1()(9D0+&:(@)0+)2")+(2)#$)&9"23(-)092(-9+30$),"-)9(>2)=(&-?)."+2)%0B(%=)09)23()59+202#2()",)4("$3=+0!+G)

A$9B(#;$%)$6) H!"..#90!&20"9?) $("$%() &1&-(9(++IG) 5) 3&@) +".() !&$&'0%020(+) 09)1('.&$$09:)2(!39"%":0(+)+")5)!"#%@)#+()23(.)2")@")+".()1"-B)09)23()!"9209#02=)",)23()"9()5)&%-(&@=).&@()230+)20.(?)1()!"#%@)&%+")#+()23()*-!456)6(-D(-+)&9@)"23(-)1(')$-"@#!2+),-".)J6K5?)'(!&#+()23()54A)!3"+()230+)$-"D0@(-7))

L"-B)"9)23()5;$<6)H@($(9@09:)"9)'#@:(2+IG)230+) 0+).=)'0::(+2)10+37)59@((@?)5)1"#%@)'()D(-=)092(-(+2(@)2")1"-B)&%+")"9)23():-"#9@)1023)+".():("$3=+0!&%)@(D0!(+) 09)23(),#2#-(?)&9@)230+)1"#%@),02)1023).=)2-&0909:+)",)9(>2)=(&-7)

6".() &;7:<3);(%&) ",) 23() +(0+.0!) 9(21"-B+F) 830+) 0+) &) 2-&!B) 130!3) *%(>0+) 23"#:32)&'"#2?)&9@)&!2#&%%=)A3#"9:?)*%(>0+)&9@).=+(%,)3&D()&)%"2)",)0@(&)2")!"9209#()1"-B09:)2":(23(-)09)23(),0(%@)",)9&2#-&%)@0+&+2(-+)&9@)-(+!#()+0.#%&20"9+7)

6")23(+()&-()$"++0'0%020(+)'"-9)23-"#:3)230+)!"%%&'"-&20D()1"-B7)

http://www.robocuprescue.org



12




• Rescue agents: inspired from the agents found in RobocupRescue simulations [www.robocuprescue.org]

• GAMA (GIS and agent-based modeling platform [Amouroux et al., 07]) is used to build model

!"#!

!"#$%&'%()*!!(++),"-.+)"-)/#(-0(+)1023)*-!4567)83(),"%%"109:)+!-((9+3"2)+3"1+)13&2)0+)$"++0'%()1023);<)*9&%=+2)*-!456)(>2(9+0"9?)+")23(-()&-()!(-2&09%=)'(&#20,#%)2309:+)%(,2)2")@"7)

!!"#$$%&'()*+,-*./01*234*(5*63738$6*9:;<6;%8&*;%*=36;%'*1;&)#;")*(5*>3%(;*?;)@*

)*%(>0+)&9@)A3#"9:)$-"$"+(@).()2")@")&%%)230+)1"-B),-".)C-&9!()@#-09:)23()=(&-7)5)10%%)!"9209#().=)+2#@0(+)09):("%":=)230+)=(&-?)&9@)02)10%%)@($(9@)"9)23()D"%#.()",).=)!%&++)+!3(@#%(7)E#2)02)0+)+".(2309:)23&2)-(&%%=)092(-(+2+).(?)+")13=)9"2F)))

"""# $%&&"'"(")*!%+!+,),-.!"/).-/&0"1!*9"23(-)$"++0'0%02=)1()(9D0+&:(@)0+)2")+(2)#$)&9"23(-)092(-9+30$),"-)9(>2)=(&-?)."+2)%0B(%=)09)23()59+202#2()",)4("$3=+0!+G)

A$9B(#;$%)$6) H!"..#90!&20"9?) $("$%() &1&-(9(++IG) 5) 3&@) +".() !&$&'0%020(+) 09)1('.&$$09:)2(!39"%":0(+)+")5)!"#%@)#+()23(.)2")@")+".()1"-B)09)23()!"9209#02=)",)23()"9()5)&%-(&@=).&@()230+)20.(?)1()!"#%@)&%+")#+()23()*-!456)6(-D(-+)&9@)"23(-)1(')$-"@#!2+),-".)J6K5?)'(!&#+()23()54A)!3"+()230+)$-"D0@(-7))

L"-B)"9)23()5;$<6)H@($(9@09:)"9)'#@:(2+IG)230+) 0+).=)'0::(+2)10+37)59@((@?)5)1"#%@)'()D(-=)092(-(+2(@)2")1"-B)&%+")"9)23():-"#9@)1023)+".():("$3=+0!&%)@(D0!(+) 09)23(),#2#-(?)&9@)230+)1"#%@),02)1023).=)2-&0909:+)",)9(>2)=(&-7)

6".() &;7:<3);(%&) ",) 23() +(0+.0!) 9(21"-B+F) 830+) 0+) &) 2-&!B) 130!3) *%(>0+) 23"#:32)&'"#2?)&9@)&!2#&%%=)A3#"9:?)*%(>0+)&9@).=+(%,)3&D()&)%"2)",)0@(&)2")!"9209#()1"-B09:)2":(23(-)09)23(),0(%@)",)9&2#-&%)@0+&+2(-+)&9@)-(+!#()+0.#%&20"9+7)

6")23(+()&-()$"++0'0%020(+)'"-9)23-"#:3)230+)!"%%&'"-&20D()1"-B7)



Organization of rescue agents

• Rescue agents are organized in multiple levels

• Agent decision models are represented as sets of rules

• Agents coordinate by exchanging messages

13

14

Behaviors of agents dedicated to resource allocation

• Agent “center” assigns rescue agencies to damaged districts

• Agencies allocate rescuers to damaged wards

Hanoi City Ba-Dinh District

This model is a foundation to build the targeted SDSS

15

Restrictions of the current model and proposal

• Restrictions:

• The agent behaviors are not realistic enough

• The simulated rescue activities are not performant

16

Restrictions of the current model and proposal

• Restrictions:

• The agent behaviors are not realistic enough

• The simulated rescue activities are not performant

• Next step of the proposal:

• Make stakeholders (experts) play the role of agents to control the rescue activities

• Acquire the knowledge of stakeholders to improve the behavior of agents

16

Step 2: improving agents’ behavior by Participatory Design

17

Design process of agent-based participatory simulations, from [Guyot & Honiden, 06]

)

MADFAM, from [Nguyen-Duc & Drogoul, 07]


Applying participatory design to the rescue model

18

A first experiment

• Involving 27 master students of the IFI in a half-day

• They play simulations to improve the behaviors of ambulances (i.e. reducing the “number of deaths”)

• Students:

• execute separately from 5 to 8 playing sessions

• follow the same progression of 4 scenarios

• take 5 minutes of discussion between two playing sessions

• attend a final 30 minutes of debriefing session

• Results:

• 11 students showed real improvements

• they reached the maximal improvement in the first scenario

• No student reached the optimal result (=8) for all four scenarios

ImprovementNumber of

students

0 16/27

2 4/27

3 1/27

4 2/27

5 2/27

6 1/27

7 1/2719

Requirements

• User-interface must be friendly and interactive

• Scenarios

• must be understandable, realistic, rich, varied

• sound progression from simple to complex ones

• Experimental protocol with well-design questionnaires (for debriefing sessions)

20

Limitations of the current participatory design process

• A effective model requires:

• a large number of playing sessions

• the analysis of a large base of user trace

• Limitations:

• Manual analysis of modelers takes a lot of time

• Offline change of model lacks an immediate feedback

21

from [Nguyen-Duc & Drogoul, 07]

Step 3: automating the acquisition of experts’ knowledge by ML

• Machine learning

• Automatically extract the behaviors of users

• Online and interactive learning

• Immediately improve the behaviors of agents

• Let agents intelligently negotiate with users

• Help agents learn more quickly the users’ decision-making

I will save victimX, he’s very close.

No, I prefer victimY he’s in a more critical state

Ok, so the gravity is more important than the distance

22

Requirements of an online and interactive learning

• Being effective under constraints of time and resources

• Being supervised (by the user)

• Being incremental

• Providing visualizable and understandable "outcomes"

• SVM, KNN, Neural Network, HMM are not suitable

• Decision Tree, Bayesian Network are more suitable

• Supported by an interactive interface and a language

• to allow negotiations between users and agents23

Learning the behavior of agents

• Limitations of these methods:

• Outcomes are difficultly visualized in a understandable way

• Lack of interaction with stakeholders (i.e. learning without human supervisors)

• Need of large training sets of examples

24

• Layered learning of Robocup-Soccer [Stone, 98]

• Real-time Belief Space Search (RTBSS) of Damas-Rescue [Paquet, 06]

Method Effective SupervisedVisualizable &

Understandable Incremental Interactive

RTBSS

Layered

v v x v x

v v x v x

My choice: combining decision tree and utility function

• Binary decision tree [Payne & Meisel, 77], [Cerny et al., 79]

• to treat categorial data

• to solve classification problems

• to filter alternatives

• Additive utility function [Keeney & Raiffa, 76]

• to treat numerical data

• to solve regression problems

• to represent preferences

25

Decision model of agents

An utility function to choose a target district for hospitalsAn utility function to choose a target district for police officesAn utility function to choose a target district for fire-stations

Hospital has an UF to choose a target ward for ambulancesPolice office has an UF to choose a target ward for police forcesFire-station has an UF to choose a target ward for firefighters

Each ambulance, firefighter, police force has:- A decision tree to choose target type- For each type, an utility function to choose a precise target

Behavior of an ambulance

• Ambulance have two questions:

• Go to an onsite victim for first-aid or take the carried victims to hospital?

• If the type is determined, which precise target will be chosen?

• Decision model of ambulance contains:

• One decision tree to choose a target type (victim or hospital)

• Two utility functions to choose a target of a specific type

26

F(Vk) = ∑ wi * Ck

i

The vic(m Vmax will be selected if: Vmax = ArgMax{F(Vk)}

Criteria to choose a victim Min/Max

Name

Distance (from ambulance to victim) (-) C1

Gravity (of victim) (+) C2

Distance (from victim) to closest other victim (-) C3

Number of victims nearby (+) C4

Max gravity of victims nearby (+) C5

Hospital

Can carry more

No Yes

Victim

Serious victim carried

No

Hospital

Yes

27

Learning decision tree

Victim

I will go to V1 because:I can carry more victimand V1 is close to me

Hospital

Can carry more

No Yes

User change decision

{V1, V2, V3, V4, H1, H2}Alternatives:

Decision:

Reasoning for change

High(freeBedNumber)

SeriousVictimCarried

Numericalcriteria:

Boolean function:

H1

You must go to H1 becauseyou carry a victim in critical state

and H1 has free beds

27


Victim


Hospital

Can carry more

No Yes

User change decision


Decision:

Reasoning for change

High(freeBedNumber)

SeriousVictimCarried

Numericalcriteria:

Boolean function:

H1

You must go to H1 becauseyou carry a victim in critical state

and H1 has free beds

27


Victim


No

• find the leaf-node corresponding to current context

• replace the leaf-node by a subtree

• boolean condition of sub-tree is defined by users

Hospital

Can carry more

No Yes


Hospital

Yes

Ambulance1 choose a target


{V1}Decision:

Reasoning for decision

Numericalcriteria: Low(distance)

Boolean function: CanCarryMore

Learning utility function

28

I will go to V1 because:s/he is close to me

F(Vk)= distance-1*



{V1}Decision:





28


You must go to V2 because:s/he’s in a more critical state

F(Vk)= distance-1*



{V1}Decision:





28



F(Vk)= distance-1*

gravity

• Add new numerical criteria (identified by user) to the function



{V1}Decision:





• Update criteria’ weights by solving “inequalities system” (Simplex method for linear programming [Vanderbei, 08])

28



F(Vk)= distance-1*-0.4*

gravity+0.6*

• Add new numerical criteria (identified by user) to the function

Experiments

• Test with an "Oracle" to validate:

• Learning decision tree

• Learning utility function

• Real-life test involves PhD students of MSI

• Ten scenarios to improve the behaviors of ambulances

• Improvement means the reduction in “number of deaths”

• Evaluation by the best result with all participants

29

Validation of learning decision tree

Tree of the Oracle

30

Victim Hospital

Tree learnt by ambulance

Victim carried

YesNo

Have onsite victim

Victim carried

Wait

Yes

No

Hospital

Yes

Victim carried

No

Victim

No Yes

Can not carry more

No

Hospital

Yes


YesNo

Victim Hospital


Tree of the Oracle

30

Victim

Hospital


Situation1 Victim carried

YesNo

No

Have not onsite victim

Wait

Yes

Have onsite victim

Victim carried

Wait

Yes

No

Hospital

Yes

Victim carried

No

Victim

No Yes

Can not carry more

No

Hospital

Yes


YesNo

Victim Hospital


Tree of the Oracle

30

Victim HospitalVictim


Situation 2Victim carried

YesNo

No


Wait

Yes

Have onsite victim

YesNo

Have onsite victim

Victim carried

Wait

Yes

No

Hospital

Yes

Victim carried

No

Victim

No Yes

Can not carry more

No

Hospital

Yes


YesNo

Victim Hospital


Tree of the Oracle

30

Victim Hospital

Victim

Tree learnt by ambulanceSituation 3

Victim carried

YesNo

No


Wait

Yes

Have onsite victim

YesNo

No


Yes

Hospital

Have onsite victim

Victim carried

Wait

Yes

No

Hospital

Yes

Victim carried

No

Victim

No Yes

Can not carry more

No

Hospital

Yes


YesNo

Victim Hospital


Tree of the Oracle

30

Victim Hospital

Victim


• The same set of rules generated from the two trees

Victim carried

YesNo

No


Wait

Yes

Have onsite victim

YesNo

No

No


Yes

Hospital

Have onsite victim

Victim carried

Wait

Yes

No

Hospital

Yes

Victim carried

No

Victim

No Yes

Can not carry more

No

Hospital

Yes


YesNo

Victim Hospital

Can not carry more

Yes

Hospital

Validation of learning utility function

• The function of agent converges towards UF of the Oracle

31

Time (in simulation steps)

Diff

eren

ce

First ambulance

Second ambulance

Difference(kmin) = ∑| ai – kmin* wi | with kmin= ArgMin{Difference(k)}

Where: ai are coefficients of the

function of Oracle: Fo(Vk) = ∑ ai * Cki

wi are coefficients of the function of

agent: Fa(Vk) = ∑ wi * Cki

Error in the utility function of agents

32

F(Vk)= distance-1*

F(Hk)= distance-1*

Victim Hospital

Real-life test with usersVictim carried

YesNo

32

F(Vk)= distance-1*

gravity0.6*

F(Hk)= distance-1*

Yes

Victim

Hospital

Scenario1

Reduce 2 deaths

-0.4*


YesNo

Have onsite victim

Wait

No

32

F(Vk)= distance-1*

gravity0.6*

distance to closest other victim-0.1*

F(Hk)= distance-1*

Yes

Victim

No

Hospital

Victim

Scenario 2

Reduce 1 death

-0.4*

0.7*

-0.2*


YesNo

Have onsite victim

Wait

No

Have onsite victim

Yes

Can not carry more

Yes

Hospital

No

32

F(Vk)= distance-1*

gravity0.6*


number of victims nearby0.3*

F(Hk)= distance-1*

Yes

Victim

No

Hospital

Yes

Victim

Scenario 3

Have reachable

victimsNo

Hospital

Reduce 3 deaths

-0.4*

0.7*

-0.2*

-0.1*

0.5*

-0.1*


YesNo

Have onsite victim

Wait

No

Have onsite victim

Yes

Can not carry more

Yes

Hospital

No

32

F(Vk)= distance-1*

gravity0.6*


number of victims nearby

distance to closest ambulance

0.3*

0.07*

F(Hk)= distance-1*

number of free beds0.1*

Yes

Victim

No

Hospital

Yes

Yes

Victim

Have reachable

victimsNo

Hospital Have reachable savable victims

Hospital

No

Scenario 4

Reduce 2 deaths

-0.4*

0.7*

-0.2*

-0.1*

0.5*

-0.1*

0.13*

-0.03*

0.67*

-0.9*


YesNo

Have onsite victim

Wait

No

Have onsite victim

Yes

Can not carry more

Yes

Hospital

No

The final decision model of ambulances

Criteria to choose a victim Min/Max

Weight

Gravity (of victim) (+) 0.5459

Number of victims nearby (+) 0.1345

Distance (from ambulance to victim) (-) 0.1034

Distance (from victim) to closest other ambulance (+) 0.0725

Max gravity of victims nearby (+) 0.0665

Distance (from victim) to closest other victim (-) 0.0635

Distance (from victim) to closest hospital (-) 0.0137

Criteria to choose a hospital Min/Max

Weight

Distance (from ambulance to hospital) (-) 0.4106

Number of free beds (of hospital) (+) 0.2477

Number of victims nearby (+) 0.1267

Distance (from hospital) to closest ambulance (+) 0.0975

Max gravity of victims nearby (+) 0.0674

Distance (from hospital) to closest other victim (-) 0.0365

Distance (from hospital) to closest other hospital (-) 0.013633

Victim carried

Have onsite victim

Wait

Yes

No

Victim

Yes

Have onsite victim

No

Hospital

No Yes

Can not carry more

YesNo

HospitalHave

reachable victims

YesNo

Hospital Have reachable savable victims

Hospital

YesNo

Have reachable savable victims with

safe path

YesNo

Hospital

Have serious reachable savable victims with safe

path

YesNo

Victim (serious, reachable, savable,

safe path)Hospital


YesNo

Victim (reachable, savable, safe path)

Results for all ten scenarios

ScenarioScenario

ParametersParametersParametersParametersImprovement

(in reducing the

number of deaths)

Hospital

number

Ambulance

number

Victim

number

Ambulance

capacity

Improvement

(in reducing the

number of deaths)

1

2

3

4

5

6

7

8

9

10

1 1 6 1 2

1 1 8 2 1

1 1 18 3 3

2 2 33 3 2

2 4 42 4 4

2 4 54 5 3

5 15 67 6 6

5 15 86 8 8

6 24 128 10 7

6 24 242 10 12

34

Conclusions• Concerning the design of a SDSS, my proposal:

• automatically acquire part of the stakeholders’ knowledge

• enhance the realism and the effectiveness of system

• reduce the number of tests and focus on a few prototypes

• The outcomes of this PhD thesis can be easily generalized to support the modeling of different socio-environmental systems:

• My proposal of PD augmented with ML can be used in any applicative context

• I designed the interactive interface, such that it can be reused in any context of decision-making

• I designed the combination of DT and UF in order to be adaptable to model any agent behaviors

35

Prospects

• Improving user/agent interaction with a more friendly interface and a more natural language

• Currently, learning process requires a lot of efforts from the users when playing with the agents

• Improving learning algorithm to support fault-tolerance

• Currently, learning algorithm requires a high-level consistency in decisions of users

• Designing experiments with real practitioners and experts of emergency

• 2006: meeting with the Population Committee of Vietnam

• 2007: meeting with the Vietnam Search and Rescue Committee (VINASARCOM)

• ...

36

Thanks and Questions?

37

• Step 1: Using ABM&GIS (geospatial data of Badinh and earthquake loss estimation of IG-VAST) to build a realistic rescue model

• to solve the lack of realism of emergency situations

• Step 2: Using Participatory Design to improve agent behaviors

• to solve the lack of realism of rescue activities

• Step 3: Using online interactive learning (DT and UF) to automate the acquisition of experts’ knowledge

• to tackle the manual, offline improvement of agent behaviors, which is done by modelers

Using agent-based models and machine learning to enhance spatial decision support systems

Technology

Transcript of Using agent-based models and machine learning to enhance spatial decision support systems