Jehoshua (Shuki) Bruck

From Screws to Systems…

The Lineage of BMW

It happens in biological systems!!!

C. Elegans Lineage

total of 959 cells302 nerve cells131 cells are

destined to die

C. Elegans Lineage – Simple Questions

total of 959 cells302 nerve cells131 cells are

destined to die

Dealing with identity:How do cells remember what to do?

Dealing with time:How do cells know when? No clock…

Dealing with order:How do cells coordinate their actions?

Control viaStochastic Chemical Reactions

AB

C

D

E

F

G

1

2

5

4

3

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1

Chemical Reactions Networks

1

2

5

4

3

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1

Chemical Reactions Networks

1

2

5

4

3

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1 1

2

5

4

3

Chemical Reactions Networks

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1 1

2

5

4

3

Chemical Reactions Networks

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1 1

2

5

4

3

Chemical Reactions Networks

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1 1

2

5

4

3

Chemical Reactions Networks

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1 1

2

5

4

3

Chemical Reactions Networks

Solving the Puzzle

Mapping and Prediction Principles and Abstractions

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1

• What are the key players in in a gene regulatory system?

• What are their relevant interactions?

• Success: predictive model

• What are the key computational principles in gene regulations?

• A formal language for design and analysis

• Success: understanding / compressiona calculus for Biology

Mapping and PredictionGillespie, 1976; McAdams and Arkin, 1997

Gibson and Bruck, 2000; Riedel and Bruck 2005

Trajectories Physical chemistry

2

)1(__

*

**

*

*

*

*

**

**

10

1

0

,

1,,

0,

9

8

7

6

5

4

3

2

1

tkvolumeInitialVolumeproteinnoprotein

proteinmRNARibosomemRNARibosome

mRNARibosomemRNARibosome

RNasemRNARNase

mRNARNasemRNARNase

mRNARibosomemRNARibosome

mRNADNARNAPDNARNAP

DNARNAPDNARNAP

DNARNAPDNARNAP

k

freefreek

MAX

nk

n

k

kfree

kfree

freefreefreek

MAXopen

nopenk

nopen

openk

closed

+=⎯→⎯

++⎯→⎯

⎯→⎯

⎯→⎯

⎯→⎯+

⎯→⎯+

++⎯→⎯

⎯→⎯

⎯→⎯

+

+

Generating trajectories from stochastic chemical equations

We can “see” trajectories and know how compute them faster

Descriptive Biology: Is It Sufficient?

Early Work on Abstractions

Computing with neural circuits: a connection between logic and neural networks, 1943

Warren McCulloch1899 - 1969

Walter Pitts1923 - 1969

Neurophysiologist, MD

Warren McCulloch arrived in early 1942 to the University of Chicago, invited Pitts, who was still homeless, to live with his family.

In the evenings McCulloch and Pitts collaborated. Pitts was familiar with the work of Gottfried Leibniz on computing and they considered the question of whether the nervous system could be considered a kind of universal computing device as described by Leibniz.

This led to their 1943 seminal neural networks paper:A Logical Calculus of Ideas Immanent in Nervous Activity.

Logician, Autodidact

Solving the Biology Puzzle

Mapping and Prediction Principles and Abstractions

AGEGDFFEDDCBCBA

k

k

k

k

k

⎯→⎯+

+⎯→⎯

⎯→⎯+

⎯→⎯+

⎯→⎯+

5

4

3

2

1

• What are the key players in in a gene regulatory system?

• What are their relevant interactions?

• Success: predictive model

• What are the key computational principles in gene regulations?

• A formal language for design and analysis

• Success: understanding / compressiona calculus for Biology

Key to the Wonderful Progress in Design:Abstractions in Information Systems

Reasoning to Calculations to Physics

CircuitsBoolean CalculusReasoning

Key to the Progress in Design:Abstractions in Information Systems

Shannon1916-2001

1938Boolean Algebra to Electrical Circuits

Logic Design

1847Connected Logic

with AlgebraBoolean Algebra

Logical Calculation

Boole1815-1864

Logic to Boolean Calculus to Physical Circuits

S D

Text to Algebra George Boole, 1854

The Algebra (Boolean Calculus)Boole, DeMorgan, Jevons, Peirce, Schroder (18xx)

Postulate System: Huntington (1904)

Algebraic system: set of elements B, two binary operations + and B has at least two elements (0 and 1)

•

If the following postulates are true then it is a Boolean Algebra:

(i) identity

(ii) complement

(iii) commutative

(vi) distributive

1; 0a a a a+ = ⋅ =

0 ; 1a = a a = a+ ⋅

;a b b a a b b a+ = + ⋅ = ⋅

( ) ( ); ( )a b c a b a c a b c a b a c+ ⋅ = + ⋅ + ⋅ + = ⋅ + ⋅

Shannon MSc Thesis, 1938

sum

carry

Who invented the binary representationof numbers?

Leibniz – Binary System

Gottfried Leibniz1646-1716

Leibniz – Binary System

Gottfried Leibniz1646-1716

Binary addition algorithm

The First Digital AdderGeorge Stibitz, 1904-1995

He worked at Bell Labs in New York.

In the fall of 1937 Stibitz used surplus relays, tin can strips, flashlight bulbs, and other common items to construct his "Model K" (K stands for kitchen table).

Model K was designed to display the result of the addition of two bits.

Key to the Wonderful Progress in Design:Abstractions in Information Systems

Reasoning to Calculations to Physics

CircuitsBoolean CalculusReasoning

Key Challenge to the Progress in AnalysisAbstractions in Information Systems

Sensory Forms to Calculations to Reasoning

ReasoningCalculusSensoryForms

•Text•Images•Audio•Numbers•Figures•SW•…

Ask a design question:Is it a feature or a bug?

Sensory Forms to Calculations to Reasoning

+

∧

+

∧∨

xy

z

C

S

Biology Engineering

Key Challenge to the Progress in AnalysisAbstractions in Information Systems

?? ??

Abstractions

• Cyclic vs. acyclic (feedback)• Stochastic vs. deterministic

A Feature or a Bug?

??

Bio Circuits vs. Combinational Logic CircuitsJoint work with Marc Riedel

• Cyclic vs. acyclic (feedback)• Stochastic vs. deterministic

+

∧

+

∧∨

xy

z

C

S

Are Cycles a Feature or a Bug?

Hypothesis ?????

Cycles might help in

• Reducing cost

• Increasing performance

Circuits With Cycles

a b c

1f 2f 3f

Generally exhibit time-dependent behaviorMay have unstable/unknown outputs

Generally exhibit time-dependent behaviorMay have unstable/unknown outputs

01 1

? ? ?0: non-controlling for OR1: non-controlling for AND

Circuits With Cycles

Cyclic Circuits Can be Combinational McCaw’s 1963

Cyclic, 4 AND/OR gates, 5 variables, 2 functions:

ORORAND AND

Cyclic Circuits Can be Combinational McCaw’s 1963

Cyclic, 4 AND/OR gates, 5 variables, 2 functions:

ORORAND AND

X=0

Cyclic Circuits Can be Combinational McCaw’s 1963

Cyclic, 4 AND/OR gates, 5 variables, 2 functions:

ORORAND AND

X=1

Smallest possible equivalent acyclic circuit?5 AND/OR gates; improvement factor is 4/5

x

1f

ba

ORANDOR ORAND

McCaw’s Circuit (1963)

Cyclic Combinational Circuits

Cyclic circuits can be combinationalShort 1961, McCaw 1963, Kautz 1970, Huffman 1971, Rivest 1977

b c b c

f1 f2 f3 f4 f5 f6

a a

Improvement factor is 2/3 (Rivest 1977)

Improvement factor of ½ (Riedel & Bruck 2003)

The Role of Cycles in Circuit Design?Best paper award in 2003 Design Automation Conference

• Developed the theory and synthesis techniques for cyclic combinational circuitsSynthesis is based on symbolic analysis

• Caltech Cyclify = a software package for the design of combinational circuits with cycles

• Integrated Caltech Cyclify with the Berkeleydesign tools

• Evaluated benchmark circuits and compared with current design tools

Cycles in Circuits is a Feature!

Cycles help in

• Reducing cost

• Increasing performance

Optimization for Cost (Area)

Cost: Number of NAND2/NOR2 gates

7.47%10031084s148811.66%758858styr

20.54%673847duke26.03%483514s5103.91%393409pma4.36%329344cse

15.56%255302bw3.90%222231s3865.73%889943planet

21.65%152194ex610.34%1822035xp1

ImprovementCaltech CYCLIFYBerkeley SISBenchmark

Optimization for Performance (Delay)and Fixed Cost

15.22%391.01%1079461090s149420.93%342.07%995431016s148810.53%343.50%71638742duke213.89%314.24%54236566s1

14.29%241.77%44428452s51028.57%209.29%25428280bw

4.35%2214.29%180232105xp114.71%295.42%55834590in217.50%331.00%59340599in317.65%144.66%32717343t121.05%154.57%16719175p82

ImprovementDelayImprovementAreaDelayAreabenchmarkCaltech CYCLIFYBerkeley SIS

Cost: number of NAND2/NOR2 gatesDelay: 1 time unit/gate

Bio Circuits vs. Combinational Logic CircuitsJoint work with Cook, Soloveichik and Winfree

+

∧

+

∧∨

xy

z

C

S

• Cyclic vs. acyclic (feedback)• Stochastic vs. deterministic

??

Computing with Systems of Chemical Reactions

CBA+

1112

0022 3101

2011

4000

2ADC +

The Reachability Question

Given a system of chemical reactions, and an initial state A (1112). Also given is a state B (4000).

Starting at A, can the system of chemical reactions reach B?

- This question is decidable

- The state space is finite!!!

- Originally proved by Karp and Miller 1969 in the context of Vector Addition Systems (VAS)

Stochastic Chemical Reactions

CBA+

1112

0022 3101

2ADC +

The probability for a reaction to happenis a monotonic function in the number of molecules

(#A x #B) or (#C x #D)

Stochastic Reachability

Given a system of chemical reactions, and an initial state A. Also given is a state B.

Starting at A, is the probability to reach B bigger than 1-small ?

Stochastic chemical reactions are Turing universal – with high probability

- This question is undecidable

Stochastic Behavior is a Feature

Probability enables general (‘precise’) computationin biochemical systems!!

Stochastic Behavior is a Feature

Probability enables general (‘precise’) computation in biochemical systems, Proof?

Register Machines (Minsky 1967)

Register A

Register B

Programming Unit

Infinitely large

Idea: Simulate Register Machineswith Chemical Reaction Networks

Register Machinesare universal!!

general computing

Marvin Minsky1927 -

Register Machines

Programming Unit

Register Machines (Minsky 1967)

Register A

Register C

Inc(A) – increment A and go to the next instruction

Dec(A,k) – if A is not 0, decrement A and go to next instructionotherwise, if A is 0, go to instruction k

Register B

Register Machines - Example

Inc(A) – increment A and go to the next instruction

Dec(A,k) – if A is not 0, decrement A and go to next instructionotherwise, go to instruction k

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

•Three registers

•a and b are nonnegative integers

•Let A=a , B=b and c = 0

•What is the program computing?

Register Machines - Example

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

•Three registers

•a and b are nonnegative integers

•Let A=a , B=b and c = 0

•What is the program computing?

Output of program is in C

C=1 a is bigger than b

C=2 a is smaller or equal to b

Register Machines - Example

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

A B C

0

Register Machines - Example

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

A B C

0

Register Machines - Example

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

A B C

000

Output of program is in C

C=1 a is bigger than b

C=2 a is smaller or equal to b

Register Machines

Programming Unit

Register Machines (Minsky 1967)

Register A

Register C

Inc(A) – increment A and go to the next instruction

Dec(A,k) – if A is not 0, decrement A and go to next instructionotherwise, go to instruction k

Register B

Are Universal

Idea: Simulate Register Machineswith Chemical Reaction Networks

Simulation of Register Machines with CRNs

i: Inc(R) : Si -> R + Si+1

i: Dec(R,k): R + Si -> Si+1

If R=0 then Si -> Sk

Inc(A) – increment A and go to the next instruction

Dec(A,k) – if A is not 0, decrement A and go to next instructionotherwise, go to instruction k

CompilerRM to CRN

Simulation of Register Machines with CRNs

i: Inc(R) : Si -> R + Si+1

i: Dec(R,k): R + Si -> Si+1

If R=0 then Si -> Sk

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

A + S1 -> S2S1 -> S4

B + S2 -> S3S2 -> S5

C + S3 -> S4S3 -> S1

S4 -> C + S5

S5 -> C + S6

Simulation of Register Machines with CRNs

i: Inc(R) : Si -> R + Si+1

i: Dec(R,k): R + Si -> Si+1

If R=0 then Si -> Sk

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

A + S1 -> S2S1 -> S4

B + S2 -> S3S2 -> S5

C + S3 -> S4S3 -> S1

S4 -> C + S5

S5 -> C + S6

A Problem:This reaction can happen even if R is not zero…..

Simulation of Register Machines with CRNs

i: Inc(R) : Si -> R + Si+1

i: Dec(R,k): R + Si -> Si+1

If R=0 then Si -> Sk

1: Dec(A,4)2: Dec(B,5)3: Dec(C,1)4: Inc(C)5: Inc(C)

A + S1 -> S2S1 -> S4

B + S2 -> S3S2 -> S5

C + S3 -> S4S3 -> S1

S4 -> C + S5

S5 -> C + S6

The solution:Delay thisreaction using a “stochastic clock”

Simulation of Register Machines with CRNs

i: Inc(R) : Si -> R + Si+1

i: Dec(R,k): R + Si -> Si+1

If R=0 then Si -> Sk

Delay this reaction using a “stochastic clock”

When R>0, it is less likely to happen. DEC (R,i)

Case 1: R=0 Si -> Sk with probability 1

Case 2: R>0

Si -> Sk with small probability

R + Si -> Si+1 with probability close to 1

Stochastic Behavior is a Feature

Probability enables general (‘precise’) computationin biochemical systems!!

??

Is it a Feature or a Bug?

• Stochastic vs. deterministicProbability enables universal computation in chemical reaction networks (Cook, Soloveichik, Winfree, Bruck, 2005)

• Cyclic vs. acyclic Cycles enable cost savings in real combinational circuits(Riedel & Bruck 2003)

Current / future work:

• Relations vs. functions?

• The logic of computing probability distributions?

??

Shannon1916-2001

Turing1912-1954

Leibniz1646-1716

Boole1815-1864

• Logic and Binary system

• Calculus

Connected Logicwith AlgebraBoolean AlgebraLogical Calculation

Defined Computingvia universal machinesComputer Science

•Connected Boolean Algebra to Electrical Circuits Logic Design

•Connected probability to Communications Information Theory

Calculus for Biology??

We need to learn / teach about abstract systems forreasoning about information

Emil Post 1897-1954

Compositions of Boolean functionsUniversal Algebra

"The further back you look, the further forward you can see"

Winston Churchill