Connections, similarities, and differences betweenstochastic and deterministic models of biochemical reaction

systems

David F. Anderson∗

∗anderson@math.wisc.edu

SysBioM Seminar

April 29th, 2014

Outline of talk

1. Discussion of stochastic versus deterministic models.

2. Provide main ideas and results without technical details.

I Connections: deterministic models arise by limit of stochasticmodel.

I Similarities: large classes of systems have quite stable long-termbehavior for both stochastic and deterministic models.

I Differences: large classes of systems have very different long-termbehavior between stochastic and deterministic models.

3. Go back and fill in some of the details.

4. End with an open problem.

Stochastic versus deterministic models

A process is deterministic if its future is completely determined by its presentand past.

I solutions to differential equations.I solutions to difference equations.

ExampleConsider a growth process (i.e. Bacterial division):

B1/3→ 2B,

with associated ODE model

x(t) =13

x(t) x(0) = 2,

has the solutionx(t) = 2e

13 t .

�

Stochastic versus deterministic models

On the other hand, a stochastic process is a random process evolving in time.

Informally, even if you have full knowledge of the state of the system (and it’sentire past), you can not be sure of it’s value at future times.

More formally, a stochastic process is a collection of random variables, X (t)or Xt , indexed by time.

Example

I Consider rolling a die multiple times.I Let Fn denote the sum of the first n rolls.I Then, F0 = 0, and F1 ∈ {1, . . . , 6}, F2 ∈ {2, . . . , 12}, etc.I Knowing that F2 = 8 only guarantees that F3 ∈ {9, . . . , 14}.

Example: Bacterial Growth

Let’s consider two oversimplified models for growth of a bacterial colony:

I one deterministic

I one stochastic.

Deterministic model: a reasonable model for

B1/3→ 2B

would beddt

x(t) =13

x(t) x(0) = 10, (1)

with solutionx(t) = 10et/3.

Example: Bacterial Growth

Stochastic Model: Without going into the finer details, assume

1. Each bacteria divides after a random (independent, exponential) amountof time with an average wait of 3 hours.

Similar to equation (1) for the deterministic model, it is possible to write downsystems of equations describing the time evolution of model

1. Evolution of individual sample paths – instance of experiment (like theODE model)

2. Evolution of the distribution (probability of being in certain states)

I chemical master equation or Kolmogorov’s forward equation.

Example: Bacterial Growth - evolution of sample paths

I Below is a plot of the solution of the deterministic system versus threedifferent realizations of the stochastic system.

0 1 2 3 4 510

20

30

40

50

60

70

Colo

ny s

ize

Time

I Stochastic realizations/experiments appear to follow the deterministicsystem in a “noisy” way.

Example: population growth - evolution of distribution

10 12 14 16 18 200

0.2

0.4

0.6

0.8

1

n

p n(0)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

n

p n(1)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

n

p n(2)

10 20 30 40 50 600

0.01

0.02

0.03

0.04

0.05

0.06

0.07

n

p n(3)

Example: Bacterial Growth and Death

Now suppose that we change the model “slightly” in that:

1. we allow bacteria to die as well as divide.

We suppose that they die after about five hours:

B1/3→ 2B,

B1/5→ ∅.

Our new deterministic model could be

x(t) =13

x(t)− 15

x(t) =2

15x(t), x(0) = 2,

with solutionx(t) = 2e2t/15.

Example: Bacterial Growth and Death

For the stochastic model, the next event is either

1. a growth event (via a division) or

2. a decrease event (via a death).

Example: Bacterial Growth and Death

I Deterministic vs. three realizations/experiments of stochastic system.

0 2 4 6 8 100

5

10

15

20

25

Time

Colo

ny S

ize

What we want from theorems: possible long term dynamicsDeterministic: stable fixed point

Stochastic: convergence of distribution to equilibria

What next?

So we know about stochastic and deterministic models in a general sense....

I I am going to try to say something sensible/interesting about biochemicalreaction systems, which are

networks (a static object) + dynamics.

I We have a choice in dynamics:

I Deterministic.

I Stochastic.

I I want to better understand how the models relate, when the modelingchoice leads to similar conclusions, and when the modeling choice leadsto vastly different conclusions.

Different Networks: from simple to complex

Example 1: A+B → C.

Example 2: A + B → 2B or S + I → 2I

B → A I → S

Example 3: Gene transcription & translation:

Gκ1→ G + M transcription

Mκ2→ M + P translation

Mκ3→ ∅ degradation

Pκ4→ ∅ degradation

G + Pκ5�κ−5

B Binding/unbinding of Gene

Cartoon representation:

1

1J. Paulsson, Physics of Life Reviews, 2, 2005 157 – 175.

Model of viral infection: Srivistava, You, Summers, and Yin, J. Theor. Biol., 218(2002)Example 4:

R1 : T 1→ T + G, R2 : G 0.025→ T , R3 : T 1000→ T + S,

R4 : T 0.25→ ∅, R5 : S 2→ ∅, R6 : G + S 7.5×10−6→ V ,

0 20 40 60 80 100 120 140 160 180 200100

101

102

103

104

105

Time (Days)

Cou

nts

template

genome

virus

structure

Figure: A single trajectory.

4

Different Networks

Example 5: EnvZ/OmpR signaling system

2

2Guy Shinar and Martin Feinberg, Structural Sources of Robustness in Biochemical ReactionNetworks, Science, 2010

Different Networks

Hanahan and Weinberg, The Hallmarks of Cancer, Cell, 2000.!

Different Networks

Metabolic Pathways�

Roche Applied Science!

Big picture

These networks can be huge! However, we want to understand the behaviorof these models.

I For complex models, simulation is often used to explore the possibledynamics (for both deterministic and stochastic models).

I Key system parameters are oftentimes unknown, or known only up to anorder of magnitude.

I Want an alternative approach: discover what pieces of the networkarchitecture determine overall system behavior.

I Best if results do not depend upon particular choices of rate constants(which are never really known).

I how can we cut through the bewildering complexity of biological systemsand say something sensible?

I Mathematics.

I This story is far from complete – really just at beginning stages.

Big pictureWill tell three stories:

1. How are the relevant deterministic and stochastic models related?

2. (2010 - A., Craciun, Kurtz, Bulletin of Mathematical Biology)

Provide network conditions that guarantee both an “especially stable”deterministic model and an “especially stable” stochastic model.2.1 When will deterministic and stochastic models give similar behavior?

3. (2014 - A., Enciso, Johnston, Royal Society Interface)

Provide network conditions that guarantee deterministic model hascomponent which is “Absolutely robust”, stochastic model has extinctionevent!

3.1 When will deterministic and stochastic models give different behavior?

3.2 Isolated examples abound (Keizer’s paradox, models in ecology, etc).We characterize broad class.

This research is part of chemical reaction network theory, which is part ofsystems biology.

Reaction Networks: {S, C,R}

Example:

A→ B

I S = {A,B}.I C = {A,B}.I R = {A→ B}.

Example:

A + B → 2B

B → A

I S = {A,B}.I C = {A + B, 2B, B, A}.I R = {A + B → 2B, B → A}.

Reaction Networks: {S, C,R}

R1) G→ G + M (Transcription)

R2) M → M + P (Translation)

R3) M → ∅ (Degradation of mRNA)

R4) P → ∅ (Degradation of protein)

I S = {G,M,P}.I C = {G, G + M, M, M + P, P, ∅}.I R = {G→ G + M, M → M + P, M → ∅, P → ∅}.

Dynamics: stochastic

Example:

A + B α→ 2B (R1)

B β→ A (R2)

X (t) = X (0) + R1(t)([

02

]−[

11

])+ R2(t)

([10

]−[

01

])

= X (0) + R1(t)[−11

]+ R2(t)

[1−1

].

For Markov models can take

R1(t) = Y1

(α

∫ t

0XA(s)XB(s)ds

)R2(t) = Y2

(β

∫ t

0XB(s)ds

)where Y1,Y2 are independent unit-rate Poisson processes.

Dynamics: stochasticFor general system, we have S = {X1, . . . ,Xd}, with

R :d∑

i=1

ykiXiκk→

d∑i=1

y ′kiXi or yk → y ′k

and complexes found at either end of reaction arrow: C = {yk , y ′k}.

I The intensity (or propensity) of k th reaction is λk : Zd≥0 → R.

I As before:X (t) = X (0) +

∑k

Rk (t)(y ′k − yk ),

with

X (t) = X (0) +∑

k

Yk

(∫ t

0λk (X (s))ds

)(y ′k − yk ),

Yk are independent, unit-rate Poisson processes.

I Kolmogorov’s forward equation (chemical master equation)

p′t (x) =∑

k

λk (x − ζk )pt (x − ζk )− pt (x)∑

k

λk (x)

Mass-action kinetics

The standard intensity function chosen is mass-action kinetics:

Reaction Intensity Function∅ κ1→ S1 λ1(x) = κ1

S1κ2→ S2 λ2(x) = κ2x1

S1 + S2κ3→ S3 λ3(x) = κ3x1x2

2S1κ4→ S2 λ4(x) = κ4x1(x1 − 1)

Dynamics: deterministic model

Example:

A + B α→ 2B (R1)

B β→ A (R2)

x ′(t) = α · xA(t)xB(t)[−11

]+ β · xB(t)

[1−1

].

or

x ′A(t) = −α · xA(t)xB(t) + β · xB(t)

x ′B(t) = α · xA(t)xB(t)− β · xB(t)

Dynamics: Stochastic Versus Deterministic

A + B α→ 2B (R1)

B β→ A (R2)

Stochastic equations

X (t) = X (0) + Y1

(α

∫ t

0XA(s)XB(s)ds

)[−11

]+ Y2

(β

∫ t

0XB(s)ds

)[1−1

].

Deterministic equations

x(t) = x(0) + α

∫ t

0xA(s)xB(s)ds

[−11

]+ β

∫ t

0xB(s)ds

[1−1

],

or

x(t) = αxaxb

[−11

]+ βxB

[1−1

]or

xA(t) = −αxAxB + βxB

xB(t) = αxaxB − βxB.

Example: ODE Lotka-Volterra predator-prey model

Think of A as a prey and B as a predator.

Aκ1→ 2A, A + B

κ2→ 2B, Bκ3→ ∅,

with κ1 = 2, κ2 = .002, κ3 = 2.

Deterministic model. Let x(t) = [#prey,#predators]T

x(t) = x(0) + κ1

∫ t

0x1(s)ds

[10

]+ κ2

∫ t

0x1(s)x2(s)ds

[−11

]+ κ3

∫ t

0x2(s)ds

[0−1

]

Stochastic model. Let X (t) = [#prey,#predators]T

X(t) = X(0) + Y1

(κ1

∫ t

0X1(s)ds

)[10

]+ Y2

(κ2

∫ t

0X1(s)X2(s)ds

)[−11

]

+ Y3

(κ3

∫ t

0X2(s)ds

)[0−1

]

Lotka-VolterraThink of A as a prey and B as a predator.

Aκ1→ 2A, A + B

κ2→ 2B, Bκ3→ ∅,

with A(0) = B(0) = 1000 and κ1 = 2, κ2 = .002, κ3 = 2.

400 600 800 1000 1200 1400 1600 1800

600

800

1000

1200

1400

1600

1800

Predator

Prey

Stochastic modelODE model

0 5 10 15 20 25 30 35 40700

800

900

1000

1100

1200

1300

1400

1500

PreyPredator

0 5 10 15 20 25 30 35 40400

600

800

1000

1200

1400

1600

1800

2000

PreyPredator

Gene network

R1) G 200→ G + M (Transcription)R2) M 10→ M + P (Translation)R3) M 25→ ∅ (Degradation of mRNA)R4) P 1→ ∅ (Degradation of protein)

0 1 2 3 4 5 6 7 80

20

40

60

80

100

120

Time

Cou

nts

mRNA − stochasticProtein − stochasticProtein − deterministicmRNA − deterministic

Gene network

R1) G 200→ G + M (Transcription)R2) M 10→ M + P (Translation)R3) M 25→ ∅ (Degradation of mRNA)R4) P 1→ ∅ (Degradation of protein)

R5 & R6) P + P0.01�1

D (Dimerization)

0 1 2 3 4 5 6 7 80

10

20

30

40

50

60

Time

Cou

nts

mRNA − stochasticProtein − stochasticDimer − stochasticProtein − deterministicDimer − deterministicmRNA − deterministic

Gene network

R1) G 200→ G + M (Transcription)R2) M 10→ M + P (Translation)R3) M 25→ ∅ (Degradation of mRNA)R4) P 1→ ∅ (Degradation of protein)

R5 & R6) P + P0.01�1

D (Dimerization)

R7 & R8) D + G2�0.1

B (Binding)

0 10 20 30 40 50 600

5

10

15

20

25

30

Time

Cou

nts

Protein − deterministicmRNA − deterministicDimer − deterministicProtein − stochasticmRNA − stochasticDimer − stochastic

What is the main connection?

Assuming:I V is a scaling parameter (volume times Avogadro’s number),

I Xi = O(V ), and X V (t) def= X (t)/V ,

Then,

X V (t) ≈ 1V

X0 +∑

k

1V

Yk

(V∫ t

0κk X V (s)yk ds

)(y ′k − yk )

Apply the Law of Large Numbers

x(t) = x(0) +∑

k

∫ t

0κk x(s)yk ds · (y ′k − yk ),

whereuv = uv1

1 · · · uvdd ,

is standard mass-action kinetics. OR

x ′(t) =∑

k

κk xyk · (y ′k − yk ).

LLN: Example

I Stochastic models:

A + B2/V→ 2B (R1)

B 1→ A (R2)

with X (0) = [3V ,V ] so that [AV ,BV ] = X/V satisfies

AV (0) = 3, BV (0) = 1.

I ODE model of

A + B 2→ 2B

B 1→ A,

with x(0) = [3, 1].

LLN: Example, A + B → 2B B → A

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4

4.5V=1

AB

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4

4.5V=1

BA

LLN: Example, A + B → 2B B → A

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4

4.5V=10

AB

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4

4.5V=50

AB

LLN: Example, A + B → 2B B → A

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4

4.5V=100

AB

0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4

4.5V=1000

AB

Result 1: special stability for both models

Network reversibility conditions

DefinitionLet S = {Si}, C = {yk}, and R = {yk → y ′k} denote the sets of species,complexes, and reactions, respectively. The triple {S, C,R} is called thechemical reaction network.

DefinitionThe connected components of the reaction network are called the linkageclasses.

Example

A + B α→ 2B (Linkage Class 1)

B β→ A (Linkage Class 2)

Has two linkage classes.

Network reversibility conditions

DefinitionA chemical reaction network, {S, C,R}, is called weakly reversible if eachlinkage class is strongly connected.

A network is called reversible if y ′k → yk ∈ R whenever yk → y ′k ∈ R.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

C3 C2

C1

Weakly Reversible

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

C3 C2

C1

Reversible

Network propertiesDefinition

S = span{yk→y′k∈R}{y ′k − yk}

is the stoichiometric subspace of the network.

Denotedim(S) = s.

Implication: Solutions bound to translations of S.

Example: Reaction network

A + B � 2B

B � A

0 2 4 6 8 100

1

2

3

4

5

6

7

8

9

10

# A Molecuels

# B

Mol

ecul

es

Network properties

DefinitionThe deficiency of a chemical reaction network, {S, C,R}, is

δ = |C| − `− s,

where |C| is the number of complexes, ` is the number of linkage classes ofthe network graph, and s is the dimension of the stoichiometric subspace ofthe network.

Example

A + B � 2B (R1)

B � A (R2)

n = 4, ` = 2, s = 1 =⇒ δ = 1. But,

A + B � C (R1)

B � A (R2)

n = 4, ` = 2, s = 2 =⇒ δ = 0.

Deficiency zero theorem of Horn, Jackson, and Feinberg

Theorem (Deterministic - Horn, Jackson, Feinberg, 1970’s)Suppose we have a biochemical reaction system whose network satisfies thefollowing two conditions,

I Deficiency of zero,I weakly reversible.

Then, the associated deterministic model satisfies:

I for any choice of rate constants κk , within each invariant manifold thereis precisely one equilibrium value c, and

I that equilibrium value is locally asymptotically stable.

Actually have stronger result: for each η ∈ C,∑k :yk=η

κk cyk =∑

k :y′k=η

κk cyk . (2)

c is said to be a complex balanced equilibrium.

Deficiency Zero Theorem - stochastic

Theorem (Anderson, Craciun, Kurtz, 2010)Suppose we have a biochemical reaction system whose network satisfies thefollowing two conditions,

I Deficiency of zero,I weakly reversible.

Then, the associated stochastic model satisfies:

I There is a stationary distribution which is the product of Poissondistributions:

π(x) = Mm∏

i=1

cxii

xi !, x ∈ Γ, (3)

where M is a normalizing constant.

Enzyme kinetics

Consider the possible enzyme kinetics given by

E + S � ES � E + P , E � ∅ � S

In distributional equilibrium

I the specie numbers are independent and

I have Poisson distributions.

Story 2: Absolute Concentration Robustness

Guy Shinar and Martin Feinberg, Structural Sources of Robustness inBiochemical Reaction Networks, Science, 2010.

A + B α→ 2B (R1)

B β→ A (R2)

xA(t) = −αxA(t)xB(t) + βxB(t)

xB(t) = αxA(t)xB(t)− βxB(t)

M def= xA(0) + xB(0),

Solving for equilibria:

xA = β/α,

xB = M − β/α,

Network has absolute concentration robustness in species A.

Differing in one species

Examples:

1.A, A + B

differ in species B.

2.XT , XT + Yp

differ in species Yp.

3.T , T + G

differ in species G.

Terminal and non-terminal complexes

XD X XT Xp

Xp+Y XpY X+Yp

XD+Yp XDYp XD+Y

k1

k2[ ]

k3

k4

k5

k6

k7

k8

k9

k10

k11

[ ]T

D

I The orange complexes are called terminal.

I The blue complexes are called non-terminal.

Theorems

Theorem (Marty Feinberg and Guy Shinar, Science, 2010 –deterministic)Consider a deterministic mass-action system that

I has a deficiency of one.I admits a positive steady state andI has two non-terminal complexes that differ only in species S,

then the system has absolute concentration robustness in S.

Theorem (Anderson, Enciso, Johnston – stochastic)Consider a reaction network satisfying the following:

I has a deficiency of one,I the deterministic model admits a positive steady state,I has two non-terminal complexes that differ only in species S,I (new) is conservative,

then with probability one there is an extinction event.

Proof uses linear algebra and stochastic processes: Math 632 or 605.

ExampleReaction network

A + B → 2B

B → A

has state space

0 2 4 6 8 100

1

2

3

4

5

6

7

8

9

10

# A Molecuels

# B

Mol

ecul

es

Example

Extinction can be rare event: quasi-stationary distribution

A + B2/V→ 2B

B 1→ A

XA(0) + XB(0) = M,

0 2 4 6 8 100

1

2

3

4

5

6

7

8

9

10

# A Molecuels

# B

Mol

ecul

es

Find πQM so that for τ absorption time and x ∈ transient states,

limt→∞

Pν(X (t) = x | τ > t) = πQM (x).

SatisfiesπQ

M (x) = PπQM

(X (t) = x | τ > t).

Can show that quasi-stationary distribution for A converges to Poisson

πQM (x)→ e−(β/α) (β/α)x

x!, as M →∞.

Quasi-stationary distribution: EnvZ-OmpR signaling system

3

3Guy Shinar and Martin Feinberg, Structural Sources of Robustness in Biochemical ReactionNetworks, Science, 2010

Quasi-stationary distribution

0 10 20 30 40 500

0.02

0.04

0.06

0.08

Molecules of Yp

Qua

si−s

tatio

nary

pro

babi

litie

s Xtot = 100Ytot = 3500

Xtot = 1000Ytot = 35000

Xtot = 10000Ytot = 350000

Poisson

Open question: are all such distributions well approximated by a Poisson?

That is the story. Thanks!

Collaborators: Matthew Johnston, German Enciso, Gheorghe Craciun, andTom Kurtz

References:

1. Guy Shinar and Martin Feinberg, Structural Sources of Robustness inBiochemical Reaction Networks, Science, 2010.

2. David F. Anderson, German Enciso, and Matthew Johnston, Stochasticanalysis of biochemical reaction networks with absolute concentrationrobustness, Journal of the Royal Society Interface, Vol. 11, 20130943,February 12, 2014.

3. David F. Anderson, Gheorghe Craciun, Thomas G. Kurtz, Product-formstationary distributions for deficiency zero chemical reaction networks,Bulletin of Mathematical Biology, Vol. 72, No. 8, 1947 - 1970, 2010.

Funding: NSF-DMS-1009275, and NSF-DMS-1318832.