Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study...
Transcript of Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study...
![Page 1: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/1.jpg)
Bayesian Analysis of Bayesian Analysis of Stochastic System Stochastic System
DynamicsDynamics
Rudolf KulhavýRudolf Kulhavý
![Page 2: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/2.jpg)
Why to study stochastic systems?Why to study stochastic systems?
� Dynamic modeling of the overall performance of
– value chains
– value networks
– virtual enterprisesValueadded
Valueadded
Price
Price
CC
AA
BB
Value added
Density
![Page 3: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/3.jpg)
Why to study stochastic systems?Why to study stochastic systems?
� Dynamic modeling of the overall performance of
– value chains
– value networks
– virtual enterprises
� Estimation of probabilitiesof critical events or specificquantiles of randomvariables
Valueadded
Valueadded
Price
Price
CC
AA
BB
Value added
Density
Assets – Liabilities
Probabilitydensityfunction
Dynamic model ofassets & liabilities
Insolvencyprobability
![Page 4: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/4.jpg)
Stochastic dynamic modelStochastic dynamic model
Generalization B
Conditionalprobability
density functions
Generalization A
Sampling period
State andmeasurement
“noise”
Discrete-timevalues
Timeindex
![Page 5: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/5.jpg)
Stochastic dynamic modelStochastic dynamic model
Stochasticdifferential equationrepresentation
Conditional probability
representation
![Page 6: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/6.jpg)
Markov chainMarkov chain
States… …
![Page 7: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/7.jpg)
Controlled Markov chainControlled Markov chain
Exogenous inputs
States… …
![Page 8: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/8.jpg)
Partially observed, controlled Markov Partially observed, controlled Markov
chainchain
Exogenous inputs
Measurements
States… …
![Page 9: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/9.jpg)
Partially observed, controlled Markov Partially observed, controlled Markov
chain, with unknown parameterschain, with unknown parameters
… …
Parameters
Inputs
States
Measurements
![Page 10: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/10.jpg)
Partially observed, controlled Markov Partially observed, controlled Markov
chain, with unknown parameterschain, with unknown parameters
… …
Parameters
Inputs
States
Measurements
![Page 11: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/11.jpg)
Unknown model parameters can be Unknown model parameters can be
treated as extra statestreated as extra states
The augmentation of the state vector
� increases the dimensionality of the problem (and, thereby, uncertainty of the original states);
� adds additional nonlinearities.
On the other hand, it allows for explicit modelingof parameter variations.
![Page 12: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/12.jpg)
Summary of modelSummary of model
Exogenousinputs
Measurements
States andparameters
…
![Page 13: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/13.jpg)
Estimation of states (and parameters)Estimation of states (and parameters)
Time update
Measurement updateObservations
Past data sequences
QuantificationQuantification
of all uncertaintyof all uncertainty
via via probabilityprobability
BayesianBayesianinferenceinference
![Page 14: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/14.jpg)
Functional recursionsFunctional recursions
Time updateTime update
Measurement updateMeasurement update
Likelihood PriorPosterior
Transitionprobabibility
PosteriorNext-stepprior
Product ruleProduct rule
Sum ruleSum rule
ProbabilityTheory
![Page 15: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/15.jpg)
Sequential Monte Carlo approximationSequential Monte Carlo approximation
Time update
Measurement update
Replacing probabilities with samples
![Page 16: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/16.jpg)
Particle filter (step 1)Particle filter (step 1)
![Page 17: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/17.jpg)
Particle filter (step 2)Particle filter (step 2)
![Page 18: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/18.jpg)
Data updateData update
Likelihood
Posterior
stateAlgorithm:
Resample from posterior samples with probabilities proportional to the likelihood values, then draw a sample from the corres-ponding kernel
density
pdf Replace posterior w/ samples
state
Likelihood
pdf Replace samples w/ smoothkernels
state
Likelihood
No need for explicit sampling, except Step 1
Samples come from the preced-ing iteration step
![Page 19: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/19.jpg)
Particle filter (steps 3, 4)Particle filter (steps 3, 4)
![Page 20: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/20.jpg)
Time updateTime update
Algorithm:
Pick up randomly one of the posterior samples, then draw a new sample from the corresponding transition probability density function
![Page 21: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/21.jpg)
Particle filter has Particle filter has
been applied successbeen applied success--
fully in many areasfully in many areas
� Automated target recognitionAnuj Srivastava, Michael Miller , Ulf Grenander
� Bayesian networksDaphne Koller; Kevin Murphy
� Computational anatomyUlf Grenander, Michael Miller
� Mobile roboticsDieter Fox, Wolfram Burgard, Sebastian Thrun
� Neural networksNando de Freitas
� Signal processingPetar Djurić
� Tracking and guidanceDavid Salmond, Neil Gordon
� Visual shape and motionAndrew Blake, Michael Isard, John MacCormick
![Page 22: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/22.jpg)
Illustrative exampleIllustrative example
� Consider a service company whose economic results depend critically on the performance of both the sales and service staff.
� The stocks of Sales Capacity and Service Capacityare measured in multiples of full-time equivalents (FTE) of an average sales or service person.
– This relates the labor capacity to the total performance of a team rather than the number of physical persons.
– Thus, hiring an additional person can increase the stock by more or less than one, depending on the actual person’s productivity.
![Page 23: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/23.jpg)
Model structureModel structure
![Page 24: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/24.jpg)
SimulationSimulation
resultsresults
estimate
![Page 25: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/25.jpg)
EstimationEstimation
resultsresults
estimate
measurement
![Page 26: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/26.jpg)
Model comparisonModel comparison
Posterior probability functionPosterior probability function
Model class index
Predictive density functionPredictive density function
Model class index
![Page 27: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/27.jpg)
Approximate model comparisonApproximate model comparison
Monte Carlo approximationMonte Carlo approximation
Samples from posterior pdfSamples from posterior pdf
![Page 28: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/28.jpg)
ReverendThomas BayesPierre-Simon,
Marquis de Laplace
So, what has So, what has Bayesian InferenceBayesian Inference to do to do
with System Dynamics?with System Dynamics?
Jay Wright Forrester
![Page 29: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/29.jpg)
System Dynamics x Bayesian InferenceSystem Dynamics x Bayesian Inference
� System dynamics provides the modeler with practical methodology for convert-ing prior information into a dynamic model structure (highly informative priors).
Bayesian inference
� gives precise meaning to all modeling concepts;
� yields a coherent frame-work for consistently up-dating the prior state of knowledge with numerical evidence at hand;
� captures and combines all manifestations of uncertainty (stochastic fluctuations, measurement errors, unknown model parameters, unknown model structure).
Theoretical modelsWhite-box models
Phenomenological modelsGrey-box models
Empirical modelsBlack-box models
Problemoriented
Structurefocused
SDfocus
![Page 30: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/30.jpg)
“[T]here is a loose connection between simplicity and plausibility, because the more complicated a set of possible hypotheses, the larger the manifold of conceivable alternatives, and so the smaller must be the prior probability of any particular hypothesis in the set.”
Edwin Jaynes, Probability Theory: The Logic of Science
Thus, among models of comparable predictive power, Bayesian inference assigns higher posterior probability to “simpler” ones.
Occam’s razorOccam’s razor
The amount of prior probability contained in the high likelihood region of parameter space
The maximum likelihood value
![Page 31: Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study stochastic systems? Dynamic modeling of the overall performance of –value chains –value](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f5354e69f921d171121b101/html5/thumbnails/31.jpg)
ConclusionConclusion
� The progress made in sequential Monte Carlo methods has made Bayesian inference an attractive option for system dynamics modeling, especially for problems where quantification of the state (and parameter) uncertainty is critical.