Sensitivity Analysis of an Agent-Based ... - dma.unive.it · Sensitivity analysis for ABM...
Transcript of Sensitivity Analysis of an Agent-Based ... - dma.unive.it · Sensitivity analysis for ABM...
Sensitivity Analysis of anAgent-Based Model of
Culture’s Consequences for Trade
Saskia Burgers, Gert Jan Hofstede,Catholijn Jonker, Tim Verwaart
September 9-10, 2010 - Treviso (Italy)
Sensitivity analysis
• Generally considered “good modelingpractice”
• Of the essence i actual parameter values areuncertain
• A powerful tool in the process of modelverification and validation
• Specific problems arise when performingsensitivity analysis for agent-based models
Sensitivity analysis for ABM• Agent-based models may be very
sensitive to parameter changes inparticular parts of parameter space:– Nothing may happen in large areas in the joint
parameter space– Areas may exist where the system responds
dramatically to slight changes• Parameters may significantly interact• Sensitivity may be studied for aggregated
individual level outputs
The model
• Agent-based model of a gamingsimulation
• Models effects of deceit and trust in trade• Concepts of transaction cost economics(contract incompleteness and opportunism)
Simulation gaming
Agent roles in the game
producermiddleman
retailer
consumer
tracing agency
Strategies a buyer can chose
• trust• trace random samples• in addition to random tracing, negotiate
some refund in case quality of non-compliance
• require certification• buy low quality (no risk)
Process Model (TCE)
Partnerselection
DeliverNegotiation
Trust or Monitorand enforce
F e e d b a c k : b e l i e f u p d a t e
Information asymmetry:opportunity to defect
Purpose of the Sensitivity Analysis
• To explore the behaviour of the model allover the parameter space
Decision models
Endowment factorNegative update factorBelief updateTrust (initial value)Honesty decay factorMinimal honestyDeceit and trustRisk aversionQuality preferenceImpatienceNegotiation speedConcession factorNegotiationPreference (initial value)LearningLoyaltyPartner selectionParameter or variableActivity
Influence of culture
• Culture modifies parameter values in thedecision functions
• Describe culture based on Hofstede’s fivedimensions of national cultures
• Relational attributes have differentsignificance in different cultures:– Group distance– Status difference– Interpersonal trust
Total of 21 parameters
• Which areas in parameter space result inrealistic behavior?
• In which areas of parameter space cantipping points occur?
• Which parameters have significant effectsfor which outputs?
• Which interactions between culture andother parameters are important?
• Are the answers different betweenaggregate and individual level?
Approach is based on Jansen et al.
• meta modeling of results of parameter setsdrawn at random from the joint distribution(Monte Carlo analysis), by GLM;
• analysis of contributions of Top MarginalVariance (TMV) and Bottom MarginalVariance (BMV) of individual parametersor groups of parameters to the varianceexplained by the meta model
Step 1
• Generate random parameter sets, drawnfrom the joint distribution:
• Values of parameters were drawnindepentently, from uniform distributions,over the maximal realistic range
• Note: uniform distribution, because thepurpose is to explore model behavior, notto assess uncertainty
• Check for correlations
Step 2
• Run simulations for each parameter set• Observe relevant outputs• Replications with different values of the
random seed, to estimate the influence ofinternal stochastic effects on outputvariance
Observed outputs
• number of transactions;• number of failed negotiations;• average duration of negotiations;• number of high quality transactions;• number of deceitful transactions;• number of traces requested;• number of fines issued by the tracing agency;• average loyalty
Step 3
• The standard approach is to fit a linearmodel for each output yk
yk = β0 + Σ βi xi
• And analyse the contributions of theparameters xi to output variance
• However: system behavior is far from linear
Standard solution
• If linear model does not fit, try higher orderterms, e.g.
yk = β0 + Σ βi xi + Σ βiʹ′xi2
• Or interaction terms, e.g.
yk = β0 + Σ βi xi + Σ γij xi xj
• But this didn’t result in good fit (R2 > 80%)
First finding of further exploration
• For many of the parameter sets drawn atrandom, no transactions occur
• No obvious regions in parameter spacecould be discovered where transactionsoccur / no transations occur, nor by visualinspection, no by PCA
• Logistic regression could be applied todiscover the parts of parameter spacewhere transactions occur
transactions are unlikely to occur if
• group distance and LTO are both high;• status difference and concession factor are both low;• MAS, LTO, and negotiation speed are all high or all low;• status difference and intitial trust are both low;• IDV, LTO, and quality preference are all high;• status difference is low and initial partner preference is high;• initial partner preference is low and quality preference is high;• IDV and minimal honesty are both high;• honesty decay factor and endowment factor are both high
Conclusion of this first exercise
• For a model with complex interactions, it isvirtually impossible to understand why itbehaves like it behaves
• For further sensitivity analysis, focus onthe regions in parameter space wheretransactions occur, by resampling from theoriginal sample
Metamodel with quadratic terms,and 30 forwardly selected two-factor
interactions (R2 = 80 %)
0.00.0Honesty decay factor3.01.2Initial partner preference
0.00.0Minimal honesty6.32.7Initial trust
0.00.1Endowment factor1.90.0Status difference
0.30.9Negative update factor5.10.3Mean status
3.00.0Risk avoidance6.82.7Group distance
0.71.5Quality preference7.72.0-- MAS
2.50.6Impatience6.80.7-- LTO
39.331.8Negotiation speed3.40.8-- UAI
25.09.1Concession factor5.30.2-- IDV
0.10.0Loyalty decay factor0.60.0-- PDI
0.00.0Loyalty parameterIndex of culture
BMV(%)
TMV (%)ParameterBMV(%)
TMV(%)
Parameter
Sensitivity of other outputs• negotiation failure:
– negotiation speed, concession factor, impatience• negotiation duration:
– negotiation speed, UAI, MAS• quality:
– intitial trust, LTO, quality preference• deceit:
– initial trust, MAS• tracing:
– MAS• fines:
– MAS• loyalty:
– initial partner preference, negotiation speed
Conclusions
• Number of transactions is most sensitiveto negotiation speed and concessionfactor
• Cultural and relational factors have theireffects only in interaction with other factors
• Analyse sensitivity culture by culture,using a sample of 62 actual nationalcultures
Sensitivity differences (TMV)
15.80.10.930.81.68.30.00.5Japan
6.94.811.527.12.63.80.00.0Austria
0.00.656.210.30.40.93.11.3Iran
2.10.028.846.20.31.60.00.7Netherlands
0.00.024.123.45.58.51.74.9Uruguay
11.52.41.437.61.911.50.00.0Hungary
0.00.037.917.54.73.38.70.7Morocco
0.00.040.911.60.00.20.116.9Indonesia
1.60.530.224.81.83.41.04.1mean
riskavoid.
qualitypref.
negot.speed
conces.factor
partnerpref.
initialtrust
meanstatus
groupdistance
nationalculture
Conclusion
• Sensitivity analysis and the interpretationof its results are much easier if parametershaving many interactions are fixed
• For specific situations the results of thispartial analysis may be more useful thanglobal results
Overall conclusion• Logistic regression can be used to predict
regions where realistic behavior occurs.• Parameters that have significant effects can be
identified through metamodeling, even forcomplex systems. However, the analysis is notstraightforward.
• When keeping culture constant, straightforwardmethods for sensitivity analysis can be applied.Results differ considerably across cultures.
• Sensitivity of individual agents can differconsiderably from aggregate level sensitivity.