A Beginner's Guide to Stochastic Simulationhomepages.inf.ed.ac.uk/jeh/SB/gilmore16-11-05.pdf ·...

The deterministic and stochastic approachesComparing stochastic simulation and ODEs

Stochastic simulation algorithms

A Beginner’s Guide to Stochastic Simulation

Stephen Gilmore

Laboratory for Foundations of Computer ScienceSchool of Informatics

University of Edinburgh

Systems Biology Club talk, 16th November 2005

Stephen Gilmore. Informatics, University of Edinburgh. A Beginner’s Guide to Stochastic Simulation



Background

The modelling of chemical reactions using deterministic ratelaws has proven extremely successful in both chemistry andbiochemistry for many years.

This deterministic approach has at its core the law of massaction, an empirical law giving a simple relation betweenreaction rates and molecular component concentrations.

Given knowledge of initial molecular concentrations, the lawof mass action provides a complete picture of the componentconcentrations at all future time points.




Background: Law of Mass Action

The law of mass action considers chemical reactions to bemacroscopic under convective or diffusive stirring, continuousand deterministic.

These are evidently simplifications, as it is well understoodthat chemical reactions involve discrete, random collisionsbetween individual molecules.

As we consider smaller and smaller systems, the validity of acontinuous approach becomes ever more tenuous.

As such, the adequacy of the law of mass action has beenquestioned for describing intracellular reactions.




Background: Application of Stochastic Models

Arguments for the application of stochastic models for chemicalreactions come from at least three directions, since the models:

1 take into consideration the discrete character of the quantityof components and the inherently random character of thephenomena;

2 are in accordance with the theories of thermodynamics andstochastic processes; and

3 are appropriate to describe “small systems” and instabilityphenomena.




Acknowledgements

H. Bolouri, J.T. Bradley, J. Bruck, K. Burrage, M. Calder, Y. Cao,K.-H. Cho, A.J. Duguid, C. van Gend, M.A. Gibson, D.T. Gillespie,J. Hillston, M. Khammash, W. Kolch, U. Kummer, D. Orrell,L. Petzold, S. Ramsey, H.E. Samad, S. Schnell, N.T. Thomas,T.E. Turner, M. Ullah, O. Wolkenhauer




Outline

1 The deterministic and stochastic approaches

2 Comparing stochastic simulation and ODEs

3 Stochastic simulation algorithms




Deterministic: The law of mass action

The fundamental empirical law governing reaction rates inbiochemistry is the law of mass action.

This states that for a reaction in a homogeneous, free medium, thereaction rate will be proportional to the concentrations of theindividual reactants involved.




Deterministic: Michaelis-Menten kinetics

Consider the simple Michaelis-Menten reaction

S + Ek1

k−1

Ck2

→ E + P

For example, we have

dC

dt= k1SE − (k−1 + k2)C

Hence, we can express any chemical system as a collection ofcoupled non-linear first order differential equations.




Stochastic: Random processes

Whereas the deterministic approach outlined above isessentially an empirical law, derived from in vitro experiments,the stochastic approach is far more physically rigorous.

Fundamental to the principle of stochastic modelling is theidea that molecular reactions are essentially random processes;it is impossible to say with complete certainty the time atwhich the next reaction within a volume will occur.




Stochastic: Predictability of macroscopic states

In macroscopic systems, with a large number of interactingmolecules, the randomness of this behaviour averages out sothat the overall macroscopic state of the system becomeshighly predictable.

It is this property of large scale random systems that enables adeterministic approach to be adopted; however, the validity ofthis assumption becomes strained in in vivo conditions as weexamine small-scale cellular reaction environments withlimited reactant populations.




Stochastic: Propensity function

As explicitly derived by Gillespie, the stochastic model uses basicNewtonian physics and thermodynamics to arrive at a form oftentermed the propensity function that gives the probability aµ ofreaction µ occurring in time interval (t, t + dt).

aµdt = hµcµdt

where the M reaction mechanisms are given an arbitrary index µ(1 ≤ µ ≤ M), hµ denotes the number of possible combinations ofreactant molecules involved in reaction µ, and cµ is a stochasticrate constant.




Stochastic: Fundamental hypothesis

The rate constant cµ is dependent on the radii of the moleculesinvolved in the reaction, and their average relative velocities – aproperty that is itself a direct function of the temperature of thesystem and the individual molecular masses.

These quantities are basic chemical properties which for mostsystems are either well known or easily measurable. Thus, for agiven chemical system, the propensity functions, aµ can be easilydetermined.




Stochastic: Grand probability function

The stochastic formulation proceeds by considering the grandprobability function Pr(X; t) ≡ probability that there will bepresent in the volume V at time t, Xi of species Si , whereX ≡ (X1,X2, . . . XN) is a vector of molecular species populations.

Evidently, knowledge of this function provides a completeunderstanding of the probability distribution of all possible statesat all times.




Stochastic: Infinitesimal time interval

By considering a discrete infinitesimal time interval (t, t + dt) inwhich either 0 or 1 reactions occur we see that there exist onlyM + 1 distinct configurations at time t that can lead to the stateX at time t + dt.

Pr(X; t + dt)

= Pr(X; t) Pr(no state change over dt)

+∑M

µ=1 Pr(X− vµ; t) Pr(state change to X over dt)

where vµ is a stoichiometric vector defining the result of reaction µon state vector X, i.e. X→ X + vµ after an occurrence ofreaction µ.




Stochastic: State change probabilities

Pr(no state change over dt)

1−M∑

µ=1

aµ(X)dt

Pr(state change to X over dt)

M∑µ=1

Pr(X− vµ; t)aµ(X− vµ)dt




Stochastic: Partial derivatives

We are considering the behaviour of the system in the limit as dttends to zero. This leads us to consider partial derivatives, whichare defined thus:

∂ Pr(X; t)

∂t= lim

dt→0

Pr(X; t + dt)− Pr(X; t)

dt




Stochastic: Chemical master equation

Applying this, and re-arranging the former, leads us to animportant partial differential equation (PDE) known as theChemical master equation.

∂ Pr(X; t)

∂t=

M∑µ=1

aµ(X− vµ) Pr(X− vµ; t)− aµ(X) Pr(X; t)

Partial differential equations are difficult to solve numerically.Stochastic simulation algorithms are used to solve this PDE.Before investigating this we see two examples where thedeterministic and stochastic approaches give markedly differentresults.




Example: A monostable system

Molecular noise acting upon dynamical structures can generatedisorderly behaviour in homeostatic systems.

Consider a simple example where protein molecules X and Y aresynthesized from the reservoirs A and B at an equal rate k. X andY are assumed to associate irreversibly with association rateconstant ka in the formation of a heterodimer C . Molecules of Xand Y can also decay with first-order rate constant α1 and α2

respectively.




Example: A monostable system (deterministically)

Ak→ X

α1→ φ; Bk→ Y

α2→ φ; X + Yka→ C

dφ1

dt= k − α1φ1 − kaφ1φ2

dφ2


k = 10α1 = 10−6

α2 = 10−5

ka = 10−5

φss1 '

√kka

= 1000 φss2 ' 100




Example: A monostable system (deterministically)

Ak→ X

α1→ φ; Bk→ Y

α2→ φ; X + Yka→ C

dφ1


dφ2


k = 103

α1 = 10−4

α2 = 10−3

ka = 10−3

φss1 '

√kka

= 1000 φss2 ' 100




Example: A monostable system (stochastically)

k = 10α1 = 10−6

α2 = 10−5

ka = 10−5




Example: A monostable system (stochastically)

k = 103

α1 = 10−4

α2 = 10−3

ka = 10−3




Example: A monostable system (Conclusion)

For the second set of parameters there is a noticeablediscrepancy between the behaviour of the mean and that ofthe deterministic trajectory.

Stochastic excursions indicate a severe effect of noise on thesystem.

Such an effect indicates that a deterministic approach to theanalysis of such a system can be misleading and calls for athorough stochastic treatment.




Example: Circadian clock

To adapt to natural periodicity, such as the alternation of day andnight, most living organisms have developed the capability ofgenerating oscillating expressions of proteins in their cells with aperiod close to 24 hours (circadian rhythm).

The Vilar-Kueh-Barkai-Leibler (VKBL in short) description of thecircadian oscillator incorporates an abstraction of a minimal set ofessential, experimentally determined mechanisms for the circadiansystem.




Example: Circadian clock

The VKBL model involves two genes, an activator A and arepressor R, which are transcribed into mRNA andsubsequently translated into proteins.

The activator A binds to the A and R promoters and increasestheir expression rate.

Therefore, A implements a positive loop acting on its owntranscription.

At the same time, R sequesters A to form a complex C ,therefore inhibiting it from binding to the gene promoter andacting as a negative feedback loop.




Example: Circadian clock (cartoon)




Example: Circadian clock (deterministically . . . )




Example: Circadian clock (. . . and stochastically)




Example: Circadian clock (Conclusions)

For some parameter values a differential equation modelexhibits autonomous oscillations.

These oscillations disappear from the deterministic model asthe degradation rate of the repressor δR is decreased.

The system of ODEs undergoes a bifurcation at this point andthe unique deterministic equilibrium of the system becomesstable.

However, if the effects of molecular noise are incorporated theoscillations in the stochastic system pertain.

This phenomenon is a manifestation of coherence resonance,and illustrates the crucial interplay between noise anddynamics.




Comparing stochastic simulation and ODEs

It is relatively straightforward to contrast the results of the twomethods. We compare the results of 2000 runs of the stochasticalgorithm simulating a system with initial molecular populationsS0 = 100,E0 = 10,C0 = 0,P0 = 0 and a volume of 1000 units.




Results for S0 = 100, E0 = 10, C0 = 0, P0 = 0 (vol 1000)




Comparing stochastic simulation and ODEs

It is clear that there is a close correspondence between thepredictions of the deterministic approach and the stochasticapproach, with the deterministic curve falling well within onestandard deviation (S.D.) of the stochastic mean.

This is a very close match, especially considering our stochasticsimulation is modelling a system containing just 110molecules—well within what we might consider to be themicroscopic domain.




The variance of the stochastic approach

However, it is worth bearing in mind that an actual in vivobiochemical reaction would follow just one of the many randomcurves that average together producing the closely fitting mean.This curve may deviate significantly from that of the deterministicapproach, and thus call into question its validity.

Hence, it is perhaps most important to consider the variance of thestochastic approach—with a larger variance indicating a greaterdeviation from the mean and hence from the deterministic curve.




Comparing results at lower population sizes

Consider exactly the same simulation setup, except this time weare modelling a system consisting of just 11 molecules within avolume of 100 units [thus the molecular concentrations are equalto those earlier].




Results for S0 = 10, E0 = 1, C0 = 0, P0 = 0 (vol 100)




Compatibility of the two approaches

On average, the stochastic approach tends to the same solution asthe deterministic approach as the number of molecules in thesystem increases, and we hence move from the microscopic to themacroscopic domain.




Mean results for 11, 110 and 1100 molecules




From the microscopic to the macroscopic domain

To confirm the compatibility of the two approaches we plot theglobal S.D.’s (i.e. the mean over the whole simulation time of theS.D. at each time-point) of molecular concentration data from2000 runs of the stochastic algorithm for four different simulationvolumes with equal initial molecular concentrations.




Global S.D.’s for four different simulation volumes




From the microscopic to the macroscopic domain

Each specific run is individually in closer and closer agreement withthe deterministic approach as the number of molecules in thesystem increases.

This is a direct effect of the inherent averaging of macroscopicproperties of a system of many particles.




Conclusions from the comparison

1 These results provide clear verification of the compatibility ofthe deterministic and stochastic approaches.

2 They also illustrate the validity of the deterministic approachin systems containing as few as 100 molecules.





Gillespie’s Stochastic Simulation Algorithm (SSA) is essentially anexact procedure for numerically simulating the time evolution of awell-stirred chemically reacting system by taking proper account ofthe randomness inherent in such a system.

It is rigorously based on the same microphysical premise thatunderlies the chemical master equation and gives a more realisticrepresentation of a system’s evolution than the deterministicreaction rate equation (RRE) represented mathematically by ODEs.

As with the chemical master equation, the SSA converges, in thelimit of large numbers of reactants, to the same solution as the lawof mass action.




Gillespie’s exact SSA (1977)

The algorithm takes time steps of variable length, based onthe rate constants and population size of each chemicalspecies.

The probability of one reaction occurring relative to another isdictated by their relative propensity functions.

According to the correct probability distribution derived fromthe statistical thermodynamics theory, a random variable isthen used to choose which reaction will occur, and anotherrandom variable determines how long the step will last.

The chemical populations are altered according to thestoichiometry of the reaction and the process is repeated.




Computational cost of Gillespie’s exact algorithm

The cost of this detailed stochastic simulation algorithm is thelikely large amounts of computing time.

The key issue is that the time step for the next reaction can bevery small indeed if we are to guarantee that only one reaction cantake place in a given time interval.

Increasing the molecular population or number of reactionmechanisms necessarily requires a corresponding decrease in thetime interval. The SSA can be very computationally inefficientespecially when there are large numbers of molecules or thepropensity functions are large.




Gibson and Bruck (2000)

Gibson and Bruck refined the first reaction SSA of Gillespie byreducing the number of random variables that need to besimulated.

This can be effective for systems in which some reactions occurmuch more frequently than others.




Enhanced stochastic simulation techniques

If the system under study possesses a macroscopically infinitesimaltimescale so that during any dt all of the reaction channels can firemany times, yet none of the propensity functions changeappreciably, then the discrete Markov process as described by theSSA can be approximated by a continuous Markov process.

This Markov process is described by the Chemical LangevinEquation (CLE), which is a stochastic ordinary differential equation(SDE).




Gillespie’s tau-leap method (2001)

Gillespie proposed two new methods, namely the τ -leap methodand the midpoint τ -leap method in order to improve the efficiencyof the SSA while maintaining acceptable losses in accuracy.

The key idea here is to take a larger time step and allow for morereactions to take place in that step, but under the proviso that thepropensity functions do not change too much in that interval.

By means of a Poisson approximation, the tau-leaping method can“leap over” many reactions.




Gillespie’s tau-leap method (significance)

For many problems, the tau-leaping method can approximate thestochastic behaviour of the system very well.

The tau-leaping method connects the SSA in the discretestochastic regime to the explicit Euler method for the chemicalLangevin equation in the continuous stochastic regime and theRRE in the continuous deterministic regime.




Gillespie’s tau-leap method (drawback)

The use of approximation in Poisson methods leads to thepossibility of negative molecular numbers being predicted —something with no physical explanation.




Binomial leap methods (2004)

Independently Tian and Burrage, and Chatterjee and Vlachos,proposed replacing the use of the Poisson distribution with thebinomial distribution.

Unlike Poisson random variables whose range of sample values isfrom zero to infinity, binomial random variables have a finite rangeof sample values.




Gillespie’s modified Poisson tau-leap methods (2005)

Gillespie’s modified Poisson tau-leaping method introduces asecond control parameter whose value dials the procedure from theoriginal Poisson tau-leaping method at one extreme to the exactSSA at the other.




Comparing Poisson and Binomial leap results




Computation time

Original Poisson Binomial Mod. Poissonε Time (s) Leaps Time (s) Leaps Time (s) Leaps

0.03 57 5.15× 105 89 7.75× 105 72 6.31× 105

0.05 36 3.20× 105 85 7.73× 105 47 4.13× 105




Modelling challenges: stiffness

A problem for modelling temporal evolution is stiffness. Somereactions are much faster than others and quickly reach a stablestate. The dynamics of the system is driven by the slow reactions.

Most chemical systems, whether considered at a scale appropriateto stochastic or to deterministic simulation, involve several widelyvarying time scales, so such systems are nearly always stiff.




Modelling challenges: multiscale populations

The multiscale population problem arises when some species arepresent in relatively small quantities and should be modelled by adiscrete stochastic process, whereas other species are present inlarger quantities and are more efficiently modelled by adeterministic ordinary differential equation (or at some scale inbetween). SSA treats all of the species as discrete stochasticprocesses.




Gillespie’s multiscale SSA methods (2005)

SSA is used for slow reactions or species with small populations.The multiscale SSA method generalizes this idea to the case inwhich species with small population are involved in fast reactions.




Gillespie’s slow-scale SSA methods (2005)

The setting for Gillespie’s slow-scale SSA method is

S + Ek1

k−1

Ck2

→ E + P

wherek−1 � k2

Slow-scale SSA explicitly simulates only the relatively rareconversion reactions, skipping over occurrences of the other twoless interesting but much more frequent reactions.




Comparing SSA and Slow-Scale SSA results




Conclusions

Stochastic simulation is a well-founded method for simulatingin vivo reactions.

Gillespie’s SSA can be more accurate than ODEs at lowmolecular numbers; compatible with them at large molecularnumbers.

Recent explosion of interest in the subject with many newvariants of the SSA algorithm.




Acknowledgements

Many of the graphs and much of the explanation in this talk camefrom the excellent survey paper [TSB04]. The monostable systemand the circadian clock examples came from the paper [SKPG05].




Y. Cao, D.T. Gillespie, and L. Petzold.Accelerated stochastic simulation of the stiff enzyme-substratereaction.Journal of Chemical Physics, 123:144917–1 – 144917–12,2005.

M.A. Gibson and J. Bruck.Efficient exact stochastic simulation of chemical systems withmany species and many channels.J. Comp. Phys., 104:1876–1889, 2000.

D. Gonze, J. Halloy, and A. Goldbeter.Deterministic versus stochastic models for circadian rhythms.Journal of Biological Physics, 28:637–653, 2002.

D.T. Gillespie.A general method for numerically simulating the stochastictime evolution of coupled chemical species.




J. Comp. Phys., 22:403–434, 1976.

D.T. Gillespie.Exact stochastic simulation of coupled chemical reactions.Journal of Physical Chemistry, 81(25):2340–2361, 1977.

D.T. Gillespie and L.R. Petzold.Improved leap-size selection for accelerated stochasticsimulation.J. Comp. Phys., 119:8229–8234, 2003.

P. Mendes.Gepasi: A software package for modelling the dynamics, steadystates and control of biochemical and other systems.Comput. Applic. Biosci., 9:563–571, 1993.

P. Mendes.Biochemistry by numbers: simulation of biochemical pathwayswith Gepasi 3.




Trends in Biochemical Sciences, 22:5361–363, 1997.

P. Mendes and D.B. Kell.Non-linear optimization of biochemical pathways: applicationsto metabolic engineering and parameter estimation.Bioinformatics, 14:869–883, 1998.

S. Ramsey, D. Orrell, and H. Bolouri.Dizzy: stochastic simulation of large-scale genetic regulatorynetworks.J. Bioinf. Comp. Biol., 3(2):415–436, 2005.

M. Rathinam, L.R. Petzold, Y. Cao, and D.T. Gillespie.Stiffness in stochastic chemically reacting systems: Theimplicit tau-leaping methodb.Journal of Chemical Physics, 119(24):12784–12794, December2003.

H.E. Samad, M. Khammash, L. Petzold, and D. Gillespie.




Stochastic modelling of gene regulatory networks.International Journal of Robust and Nonlinear Control,15(15):691–711, October 2005.

T.E. Turner, S. Schnell, and K. Burrage.Stochastic approaches for modelling in vivo reactions.Computational Biology and Chemistry, 28:165–178, 2004.

Carel van Gend and Ursula Kummer.Stode: automatic stochastic simulation of systems describedby differential equations.In Proceedings of the Second International Conference onSystems Biology (ICSB2001), pages 326–332, CaliforniaInstitute of Technology, 2001.


A Beginner's Guide to Stochastic Simulationhomepages.inf.ed.ac.uk/jeh/SB/gilmore16-11-05.pdf ·...

Documents

Transcript of A Beginner's Guide to Stochastic Simulationhomepages.inf.ed.ac.uk/jeh/SB/gilmore16-11-05.pdf ·...