Protocol Design from Concolic Snippets

Milo Martin f lUniversity of Pennsylvania

Ongoing work with J. Deshmukh, S. Mador-Haim, R Al A R h d A UdR. Alur, A. Raghavan, and A. Udupa

Draft manuscript at: http://www.cis.upenn.edu/~alur/Synthesis12.pdfp p y p

Protocols

Design challenging due to asynchronous model of communication

Examples: Cache coherence protocols, Distributed coordination algorithms

Successful application domain for model checking SPIN: Distributed algorithmsSPIN: Distributed algorithmsMurphi, SMV: Hardware protocolsIndustrial adoption (Intel, IBM, …)

Protocol Model: Communicating Processes

Directory

CacheController1

CacheController2

CacheController3

Finite set of processes coordinating by exchangingmessages over network links

Processes as Extended Finite-State Machines

Control States + VariablesSharers: Set of process IDsAcks: int

Sh d

Acks: intTransitions labeled with input events & guarded updates

I lidShared InvalidTrigger: Msg on channel ReqMsg

Guard: Msg.Type = GetM

Busy ModifiedUpdate: Add Sender to Sharers

Acks--Send Data to Sender ySend Invalidate to others

How can we help the designer figure out “correct” guards and updates for each transition?

Protocol Specification using Scenarios

Directory Cache1 Cache2 Cache3

Shared Invalid

GetM

Inv

Acks=1

Inv-MBusy

Inv

InvInv MBusy

Protocol Design using Concolic Snippets

Describe EFSM transitions using examplesD EF M u g mp Such as message sequence charts, live sequence charts, etc. Natural for describing typical cases and corner cases, but ….

Cumbersome to describe the entire logic using only scenariosHow to incorporate data variables and updates?

Approach: allow for “Concolic” snippetspp pp Can be a mix of concrete and symbolic values

“Concolic” coined by Sen et al in the context of testing of programs y g p gusing both concrete and symbolic inputs

Concolic Snippet Example 1

Process Directory Transition

From Shared To BusyInput channel: ReqMsgInput channel: ReqMsgOutput channels: RespMsg, InvMsg

Guard:R M T G tM & R M S d 1 & Sh {2 3}ReqMsg.Type == GetM & ReqMsg.Sender == 1 & Sharers =={2,3}

Update:RespMsg.Acks = 2;p g ;RespMsg.Destination = 1;InvMsg.Destination = {2,3}

Values of all variables can be concrete

Concolic Snippet Example 2

Process Directory Transition

From Shared To BusyInput channel: ReqMsgInput channel: ReqMsgOutput channels: RespMsg, InvMsg

Guard:R M T G tM & R M S d 1ReqMsg.Type == GetM & ReqMsg.Sender == 1

Update:RespMsg.Acks = Size(Sharers);p g ( );RespMsg.Destination = 1;InvMsg.Destination = Sharers

Same snippet can mix concrete and symbolic values

Concolic Snippet Example 3

Process Directory Transition

From Shared To BusyInput channel: ReqMsgInput channel: ReqMsgOutput channels: RespMsg, InvMsg

Guard:R M T G tMReqMsg.Type == GetM

Update:RespMsg.Acks = Size(Sharers);p g ( );RespMsg.Destination = ReqMsg.Sender;InvMsg.Destination = Sharers – ReqMsg.Sender

Values of all variables can be symbolic:Classical EFSM description maps directly to such snippets

Protocol Design from Concolic Snippets

Computational Problem

Inputs:Variable types and corresponding expression grammarFor each process,

1. Control states of EFSM2. List of all variables , input/output messages2. List of all variables , input/output messages3. Set of concolic snippets

High-level requirements (invariants and temporal logic formulas)

Solution strategy:For each EFSM transition, generate possible expressions for guards

d d b l f d h h and updates, one by one, until you find a choice that is consistent with all the corresponding snippetsCheck if resulting protocol meets all requirements, using a model h k (M h ) d f lchecker (Murphi) and if not, report a counter-example

Iterative Design

From Shared To BusyInput channel: ReqMsgO t t h l R M I MOutput channels: RespMsg, InvMsg

Guard:ReqMsg.Sender = 1 & ReqMsg.Type = GetM & Sharers ={2,3}

Update:pRespMsg.Acks = 2;InvMsg.Destination = {2,3}

Based on this single example, synthesis tool computes update:

R M A k Si (Sh )RespMsg.Acks = Size(Sharers);InvMsg.Destination = Sharers

But this is incorrect, and protocol deadlocks

Iterative Design Continued

Designer adds another concrete example (corresponds to case when Sender is a Sharer)

From Shared To BusyInput channel: ReqMsgOutput channels: RespMsg, InvMsgOutput channels RespMsg, InvMsg

Guard:ReqMsg.Sender = 1 & ReqMsg.Type = GetM & Sharers ={1,2}

Update:R s Ms A ks 1RespMsg.Acks = 1;InvMsg.Destination = {2}

Based on two snippets, synthesis tool computes update:

RespMsg.Acks = Size(Sharers + ReqMsg.Sender) - 1;RespMsg. cks S ze(Sharers ReqMsg.Sender) ;InvMsg.Destination = Sharers – ReqMsg.Sender

Expression Enumeration

Goal: Generate expression for RHS for a single variable in update for a single EFSM transition (also for guards and fields in output msgs)

Empirical observation: typical expression size is small (max was size 13)

Current prototype: p yp Supports types: Int, Set, Bool, PID, Address, Value, EnumTypes Typical operations: Boolean connectives, arithmetic, set operations, if-then-else, …

E ti ith d d t i ti i ti Enumeration with redundant expression optimization Using concrete examples, generate all “indistinguishable” expressions of size 1 Two expressions are “indistinguishable” if they generate same output in all examples

Subsumes commutativity/associativity (x+y vs y+x) composition (~(~x) vs x) constant propagation etc Subsumes commutativity/associativity (x y vs y x), composition ( ( x) vs x), constant propagation, etc.

Recursively build expressions of size N from smaller expressions Repeat until expression that satisfies all examples is found

C did t i i th h k d b li l i SMT Candidate expression is then checked on symbolic examples via SMT Any counterexample is added to set of concrete examples, repeat enumeration above

Implementation Evaluation

Starting point: cache coherence protocols described inA Primer on memory Consistency and Cache Coherence

H ll W d 2011Sorin, Hill, Wood, 2011 Translated EFSMs for 2 protocols into (mostly concrete) snippets

VI Protocol MSI Protocol

Snippets used 19 77

Update expressions synthesized 49 157

Guard expressions synthesized 17 45

E l d 1K 44 5KExpressions explored 3.1K 44.5K

Synthesis time 5 sec 134 sec

States explored by Murphi 140K 154KStates explored by Murphi 140K 154K

Methodology Evaluation

Case study: Can a user with no prior experience in designing

h h l b ld cache coherence protocol build a correct protocol from textbook description?p

Case study A: Version of MSI protocol with non-blocking directory progress

Case study B: Augmenting MSI protocol with E state to obtain MESI

Experience Report

Case study A Case study B

Snippets used in first version 19 96pp

Time to implement first version 2 hrs 6 hrs

Snippets used in final version 86 108

Number of iterations 13 8

Total manual effort 6 hrs 13 hrs

Number of counterexamples examined 5 6Synthesis time in last iteration 52 min 15 min

N b f d h d 1 5 260Number of update expressions synthesized 175 260

Number of guard expressions synthesized 80 74

States explored in final protocol 7 7 million 1 5 millionStates explored in final protocol 7.7 million 1.5 million

Recap: Protocol Design from Concolic Snippets

Specification: Familiar Communicating EFSMs, but transitions can be described using a mix of concrete and symbolic examples

Computational effort needed to “complete” the protocol is not high

Case studies suggest that, at least to translate text-book descriptions to working implementations, the proposed methodology helps designer

“Model checker in the loop” is plausible for design environments

19

Checking Consistency

Problem: Is a given update consistent with corresponding snippets? Example update: z = x+ y

C 1 ( ) Case 1 (answer is yes):Snippet: Guard: y=1; Update: z = x+1

Case 2 (answer is no):Snippet A: Guard: y=1; Update: z=x+1Snippet B: Guard: y=2; Update: z=x+1

Checking consistency corresponds to checking validity Checking consistency corresponds to checking validityCase 1: ( z=x+y -> (y=1 -> z=x+1))Case 2: ( z=x+ y -> [(y=1 -> z=x+1) & (y=2 -> z=x+1)])

V lidi i h k d i SMT l CVC3 Validity is checked using SMT solver CVC3 Synthesizing guards consistently is a bit trickier, but again relies on

calls to SMT solver

Distribution of Expression Sizes

50

60s

40

xpressions

20

30

tage of E

x

10

20

Percen

01 2 3 4 5 6 7 8

Expression Size

Synthesis Time vs Expression Size

100

10

e (s)

0.1

1

thesis Tim

0.01

Synt

0.0011 2 3 4 5 6 7 8

Expression Size

Sample Transitions

Sample Scenarios

Transition from I to S

Current Research

Efficient algorithms for learning typed expressions from concolicsnippets

Synthesis from distributed scenarios to discover auxiliary states

Automatic analysis of counter-example traces to suggest missing cases

Protocol design environment for designer productivity studiesrotocol des gn env ronment for des gner product v ty stud es